Back to Wayne Gray's home page / To Research Interests Index / To Downloadable Pubs / The CogWorks Lab / Cognitive Science Department
Arguably the first validated case of applying cognitive modeling to a real-world problem.
Project Ernestine served a pragmatic as well as a scientific goal: to compare the worktimes of telephone company toll and assistance operators on two different workstations, and to validate a GOMS analysis for predicting and explaining real-world performance. Contrary to expectations, GOMS predicted and the data confirmed, that performance with the proposed workstation was slower than with the current one. Pragmatically, this increase in performance time translates into a cost of almost $2 million dollars a year to NYNEX. Scientifically, the GOMS models predicted performance with exceptional accuracy.
The empirical data provided us with three interesting results: proof that the new workstation was slower than the old, evidence that this difference was not constant but varied with call category, and (in a trial that spanned four months and collected data on 72,450 phone calls) proof that performance on the new workstation stabilized after the first month. The GOMS models predicted the first two results and explained all three.
In key Project Ernestine paper, we discuss the process and results of model building as well as the design and outcome of the field trial. We assess the accuracy of GOMS predictions and use the mechanisms of the models to explain the empirical results. Lastly, we demonstrate how the GOMS models can be used to guide the design of a new workstation and evaluate design decisions before they are implemented.
Gray, W. D., John, B. E., & Atwood, M. E. (1993). Project Ernestine: Validating GOMS for predicting and explaining real-world task performance. Human Computer Interaction., 8(3), 237-309.
Gray, W. D., John, B. E., & Atwood, M. E. (2002). An abridgement of Project Ernestine: Validating a GOMS analysis for predicting and explaining real-world task performance. In T. Polk & C. M. Seifert (Eds.), Cognitive Modeling (pp. 1085-1114). Cambridge: MIT Press.
Atwood, M. E., Gray, W. D., & John, B. E. (1996). Project Ernestine: Analytic and Empirical Methods Applied to a Real-World CHI Problem. In M. Rudisill, C. Lewis, P. B. Polson, and T. D. McKay (Eds.), Human-Computer Interface Designs: Success Cases, Emerging Methods, and Real World Context. San Francisco: Morgan-Kaufmann
Gray, W. D., John, B. E., Stuart, R., Lawrence, D., & Atwood, M. E. (1995). GOMS Meets the Phone Company: Analytic Modeling Applied to Real-World Problems. In R. M. Baecker, J. Grudin, W. A. S. Buxton, & S. Greenberg (Eds.), Readings in human-computer interaction: Toward the year 2000, (Second ed., pp. 634-639). San Francisco: Morgan Kaufmann Publishers, Inc.
Gray, W. D., John, B. E., & Atwood, M. E. (1992). The précis of Project Ernestine or an overview of a validation of GOMS. In P. Bauersfeld, J. Bennett, & G. Lynch (Eds.), ACM CHI'92 Conference on Human Factors in Computing Systems (pp. 307-312). New York: ACM Press.
Gray, W. D., John, B. E., & Atwood, M. E. (September 1990). An application and evaluation of GOMS techniques for operator workstation evaluation. In Proceedings of the 13th International Symposium on Human Factors in Telecommunications ë90. Turin, Italy.
Gray, W. D., John, B. E., Stuart, R., Lawrence, D., & Atwood, M. E. (1990). GOMS meets the phone company: Analytic modeling applied to real-world problems. In D. Diaper, D. Gilmore, G. Cockton, and B. Shackel (Eds.), Human-Computer Interaction -- INTERACT '90. (29-34). New York: Elsevier Science Publishers.
Back to Wayne Gray's home page / To Research Interests Index / To Downloadable Pubs / The CogWorks Lab / Cognitive Science Department
Last changed: 2002-11-16 wdg