When the CHI conference gathers in Atlanta next April, one of the young traditions of the event—the Student Design Competition—will be enjoying its seventh year. With each running of the competition, the bar has been raised on both the quality and depth of the projects.
One of the side-effects of a recurring competition, however, is rivalry. For the past two years, in particular, the feel of the final round presentations and aftermath has shifted. The focus on “winning” something fosters behavior (killer on-site work sessions, project comparisons) that seem more destructive than constructive. I speak from experience on this, having successfully participated in this track in 2005.
Last week, I wrote to the co-chairs for the SDC—Mike Glaser (Drexel University) and Stephen Brewster (University of Glasgow)—to ask them to consider a couple small but critical changes to the format. I was idealistic enough to send it, but practical enough to realize it doesn’t stand much chance of changing the 2010 event. However, I thought the ideas were worth posting, perhaps for some discussion leading up to reforms for 2011.
What About Winning?
There is no doubt that winning the SDC had some positive effect. It opened some doors for our team, including enhanced stature in our program. More importantly, it validated the insane amount of work we put into the project after it was accepted to CHI. We even overcame a faulty projector and answered some tough questions to sell our fringe interpretation of the design challenge.
However, the more final sessions I see, the less important that victory becomes. The process is somewhat arbitrary, and the evaluation format changes as frequently as the judges. The more experienced I become, the easier it is to see holes in everyone’s designs that make it difficult to truly rate one above all others. Four years later, I feel more lucky than good.
The big question is, what is to be gained by ranking the top projects in some order? I’m arguing: “Nothing.”
A Chance to Experience
The beauty of this event is the opportunity for students to be challenged with a real-world problem and apply their developing skills to address it. It is framed as a competition to make it attractive for entrants, but competing is not the main value. By participating, students get a taste of academic publication, poster talks, and (if fortunate enough to advance) a chance to present in a CHI session. This is great sample experience of what successful academics strive to do.
What is unrealistic—and as it is turning out, counterproductive—is the ranked judging of the presentations, something that wouldn’t occur in later professional work in industry or academia. I believe the competition would generate a significantly more constructive atmosphere with two simple changes:
- Eliminate the project judging and ranked order of the final presentations.
- Replace one of the final four team presentations with an open panel to discuss the design challenge.
These changes should have three important effects.
First, the focus of the presentations moves from strategies for “winning” to ones built for innovation and sharing. Full paper presenters do this when approaching their long talks. This shift also takes the pressure off of inexperienced speakers. Most of our finalists (myself included) spent a good chunk of their time at CHI iterating and practicing their talk, sacrificing the normal activities of the conference, such as attending sessions. That may be the wrong way to make use of an expensive week of professional networking.
Second, eliminating ranking will de-emphasizes the specifics of “a” solution and finds strengths in many solutions. The final panel can mix professional experience (judges) with practical experience (students who did the work) and allow us collectively to learn from each other.
Finally, the changes allow the session to be more about community. Even those teams not selected to present would be able to contribute to a discussion about their experiences and process. The discussion that arises from the panel Q&A would certainly focus on interpretation of the design challenge and the methodology to support it, rather than the strengths and weaknesses of a few select concepts. The final beat will be about collaboration and shared experiences, not celebration and sour grapes.
If a ranked order is mandated for reasons I cannot see, then allow it to occur when the presenters are announced after the poster session, based on the paper, visuals and oral defense of each project. If awards are necessary at all, perhaps they could mimic those of the full CHI conference (best paper, best poster) and leave presentation out of consideration.
I would love to see the SDC feature three teams selected to present a range of high-quality solutions to stimulate discussion. Then, everyone wins.
5 replies on “Iterating CHI Student Design Competition”
I nominate Kevin for chair of the student design competition in CHI 2011. These changes are a must.
I like this proposal. It made me reflect on my own experience. As devil’s advocate, I would point out that most participants will largely be driven by external motivators. Whether it’s winning in and of itself or the chance to be seen. Honestly, I was driven by both. I didn’t care about not winning, but I would have been disappointed not being in the finals. On the other hand, the final presentation factored heavily into me getting a foot in the door at Autodesk as my future boss saw me present.
I agree it forces you to miss out on some parts of CHI as well. However, I wouldn’t trade the experience of preparing the poster and that final presentation for the talks I missed. That experience will have a lasting, positive, impression on me forever. Staying up all night to prepare, getting help from all the other IU students, nervousness I’ve still never matched leading up to the speech, etc. In the end, I don’t think I’d call that destructive. It was an experience that provided much more foundation as a designer than any talks or workshops would.
In the end, what I’ve mentioned is still possible with your proposal. Furthermore, I like that you’ve sought to remove a lot of the arbitrary nature of the oft-questionable judging. Most importantly, I agree that there is hardly a clear winner and that discussion would be far more valuable.
As a 2010 design competition finalist, I agree that I had less time to experience the week event as I stressed about last-minute poster printing, dredged up half-remembered facts, fretted about what I was being judged on (especially after realizing mid-way through presenting that the judges hadn’t read the project papers or abstracts). I was sympathetically told that I had been “fifth place”, just missing being able to present in front of the community. I would have loved to have been on a panel about process, as my team used very unorthodox guerrilla testing techniques during our urban social gaming experiments that I think the community would have liked to hear.
[…] And here’s a post pointing out some potential flaws in the competition. A good reminder that winning is not everything, it’s what you learn from the project and the other teams that counts. [Iterating the CHI Student Design Competition] […]
[…] of fun, and I’ve paid close attention to the event as it has evolved over the years, making a few suggestions of my own on how to improve it. This essay by John Thackara critiques such competitions, making valid points […]