Back to Wayne Gray's home page / To Research Interests Index / To Downloadable Pubs / The CogWorks Lab / Cognitive Science Department
The spring before joining George Mason University I attended the 1994 CHI conference where I was inspired by a terrible piece of research presented by Jakob Nielsen. During the following summer I organized and submitted a panel for CHIÍ95 entitled: Discount or Disservice? Discount Usability Analysis -- Evaluation at a Bargain Price or Simply Damaged Merchandise? (Gray, Atwood et al., 1995). The panel was accepted. In preparation for the panel I organized my first graduate reading group (Spring 1995, with three grad students). We carefully read and critiqued the major empirical research studies in the area of usability evaluation methods. We quickly determined that Nielsen was not the only or even the worst offender. What was worse, this very bad research was being cited by other people as justification for choosing one usability evaluation method over another. The panel, chaired by Mike Atwood, was a heady experience. I left CHI'95 determined to write a review article that would not simply beat up on an influential bunch of bad research, but would use an analysis of the research to explain to the human-computer interaction community why the research was flawed. Indeed, my hope was that my paper would help HCI and other applied communities to not only recognize the difference between good and bad research, but to conduct better research. Damaged Merchandise (DM) was circulated widely among friends in the HCI community before it was submitted. Indeed, their comments resulted in many significant rewrites so that by my count, the version we submitted to the journal was our fourth draft. The paper was submitted as a regular submission to the Human-Computer Interaction journal. It went through the usual editorial process with the result that the version accepted was the 6th draft. Somewhere towards the end of this process, the Editors turned DM into a special issue by soliciting 10 commentaries and asking us to write a response. Judging by the reaction of the HCI community and the citations that the paper has garnered, DM is an important paper. However, although it was a long, time consuming and interesting project, from the perspective of my research program, it was a diversion.
Gray, W. D., & Salzman, M. C. (1998). Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction, 13(3), 203-261
Gray, W. D., & Salzman, M. C. (1998). Repairing damaged merchandise: A rejoinder. Human-Computer Interaction, 13(3), 325-335.
Kirschenbaum, S. S., Gray, W. D., Ehret, B. D., & Miller, S. L. (1996). When using the tool interferes with doing the task. In M. J. Tauber (Ed.), Conference companion of the ACM CHI'96 Conference Human Factors in Computing Systems (pp. 203-204). New York: ACM Press.
See the Project Ernestine work in which we applied CPM-GOMS to save the phone company lots of money.
Back to Wayne Gray's home page / To Research Interests Index / To Downloadable Pubs / The CogWorks Lab / Cognitive Science Department
Last changed: 2002-11-16 wdg