When the CHI conference gathers in Atlanta next April, one of the young traditions of the event—the Student Design Competition—will be enjoying its seventh year. With each running of the competition, the bar has been raised on both the quality and depth of the projects.
One of the side-effects of a recurring competition, however, is rivalry. For the past two years, in particular, the feel of the final round presentations and aftermath has shifted. The focus on “winning” something fosters behavior (killer on-site work sessions, project comparisons) that seem more destructive than constructive. I speak from experience on this, having successfully participated in this track in 2005.
Last week, I wrote to the co-chairs for the SDC—Mike Glaser (Drexel University) and Stephen Brewster (University of Glasgow)—to ask them to consider a couple small but critical changes to the format. I was idealistic enough to send it, but practical enough to realize it doesn’t stand much chance of changing the 2010 event. However, I thought the ideas were worth posting, perhaps for some discussion leading up to reforms for 2011.
What About Winning?
There is no doubt that winning the SDC had some positive effect. It opened some doors for our team, including enhanced stature in our program. More importantly, it validated the insane amount of work we put into the project after it was accepted to CHI. We even overcame a faulty projector and answered some tough questions to sell our fringe interpretation of the design challenge.
However, the more final sessions I see, the less important that victory becomes. The process is somewhat arbitrary, and the evaluation format changes as frequently as the judges. The more experienced I become, the easier it is to see holes in everyone’s designs that make it difficult to truly rate one above all others. Four years later, I feel more lucky than good.
The big question is, what is to be gained by ranking the top projects in some order? I’m arguing: “Nothing.”
A Chance to Experience
The beauty of this event is the opportunity for students to be challenged with a real-world problem and apply their developing skills to address it. It is framed as a competition to make it attractive for entrants, but competing is not the main value. By participating, students get a taste of academic publication, poster talks, and (if fortunate enough to advance) a chance to present in a CHI session. This is great sample experience of what successful academics strive to do.
What is unrealistic—and as it is turning out, counterproductive—is the ranked judging of the presentations, something that wouldn’t occur in later professional work in industry or academia. I believe the competition would generate a significantly more constructive atmosphere with two simple changes:
- Eliminate the project judging and ranked order of the final presentations.
- Replace one of the final four team presentations with an open panel to discuss the design challenge.
These changes should have three important effects.
First, the focus of the presentations moves from strategies for “winning” to ones built for innovation and sharing. Full paper presenters do this when approaching their long talks. This shift also takes the pressure off of inexperienced speakers. Most of our finalists (myself included) spent a good chunk of their time at CHI iterating and practicing their talk, sacrificing the normal activities of the conference, such as attending sessions. That may be the wrong way to make use of an expensive week of professional networking.
Second, eliminating ranking will de-emphasizes the specifics of “a” solution and finds strengths in many solutions. The final panel can mix professional experience (judges) with practical experience (students who did the work) and allow us collectively to learn from each other.
Finally, the changes allow the session to be more about community. Even those teams not selected to present would be able to contribute to a discussion about their experiences and process. The discussion that arises from the panel Q&A would certainly focus on interpretation of the design challenge and the methodology to support it, rather than the strengths and weaknesses of a few select concepts. The final beat will be about collaboration and shared experiences, not celebration and sour grapes.
If a ranked order is mandated for reasons I cannot see, then allow it to occur when the presenters are announced after the poster session, based on the paper, visuals and oral defense of each project. If awards are necessary at all, perhaps they could mimic those of the full CHI conference (best paper, best poster) and leave presentation out of consideration.
I would love to see the SDC feature three teams selected to present a range of high-quality solutions to stimulate discussion. Then, everyone wins.