Back to Wayne Gray's home page / To Research Interests Index / To Downloadable Pubs / The CogWorks Lab / Cognitive Science Department
Verbal protocol analysis is extremely time consuming, but it can yield some interesting and important insights that cannot be gotten at by any other method. Cases in which I have been able to publish the results of verbal protocol analysis include the following:
Gray, W. D., & Anderson, J. R. (1987). Change episodes in coding: When and how do programmers change their code? In G. M. Olson, S. Sheppard, & E. Soloway (Eds.), Empirical studies of programmers: Second workshop. Norwood, NJ: Ablex Publishing Corporation.
Ehret, B. D., Gray, W. D., & Kirschenbaum, S. S. (2000). Contending with complexity: Developing and using a scaled world in applied cognitive research. Human Factors, 42(1), 8-23. ABSTRACT
Gray, W. D., & Kirschenbaum, S. S. (2000). Analyzing a novel expertise: An unmarked road. In J. M. C. Schraagen, S. F. Chipman, & V. L. Shalin (Eds.), Cognitive task analysis (pp. 275-290). Mahwah, NJ: Erlbaum
Action protocol analysis collects the keystrokes, mouse clicks, or other actions that people exhibit as they engage in interactive behavior. After you derive an encoding scheme it can be much easier than verbal protocol analysis. The best way of doing action protocol analysis is to use some sort of semi-automated procedures such as the ACT-PRO system developed by Wai-tat Fu.
ABSTRACT
Discrete action protocol data consist of time-stamped discrete user actions such as mouse clicks and key presses. Analysis of action protocols often entails determining how well the data matches higher-level patterns (where these patterns are specified by the researchers). Unfortunately, the process of sorting through thousands of actions to find matching patterns is very labor intensive. To facilitate this process, we have developed an ACTion PROtocol analyzer (ACT-PRO) that provides two levels of pattern matching. Level one groups sequences of actions into sets of labeled strings. Level two matches these labeled strings to a hierarchical pattern. ACT-PRO can be used to determine how well the data fit the patterns specified by the experimenter. Complementarily, it can be used to focus the experimenterÍs attention on those data that do not fit the pre-specified patterns. We provide examples of both uses of ACT-PRO.
Examples where I have used action protocol analysis in my own work include the following. (Note that where ever I have used verbal protocol analysis I have also used action protocol analysis; however, the reverse is not true.)
Gray, W. D., & Fu, W.-t. (2001). Ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head: Implications of rational analysis for interface design. CHI Letters, 3(1), 112-119. Also in ACM CHI'01 Conference on Human Factors in Computing Systems
Gray, W. D., & Boehm-Davis, D. A. (in press). Milliseconds Matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experiment Psychology: Applied, 6(4), 322-335.
Fu, W.-t., & Gray, W. D. (2000). Memory versus Perceptual-Motor Tradeoffs in a Blocks World Task. Twenty-second Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.
Gray, W. D. (2000). The nature and processing of errors in interactive behavior. Cognitive Science, 24(2), 205-248.
Ehret, B. D., Gray, W. D., & Kirschenbaum, S. S. (2000). Contending with complexity: Developing and using a scaled world in applied cognitive research. Human Factors, 42(1), 8-23. ABSTRACT
Gray, W. D., & Kirschenbaum, S. S. (2000). Analyzing a novel expertise: An unmarked road. In J. M. C. Schraagen, S. F. Chipman, & V. L. Shalin (Eds.), Cognitive task analysis (pp. 275-290). Mahwah, NJ: Erlbaum
Kirschenbaum, S. S., Gray, W. D., Ehret, B. D., & Miller, S. L. (1996). When using the tool interferes with doing the task. In M. J. Tauber (Ed.), Conference companion of the ACM CHI'96 Conference Human Factors in Computing Systems (pp. 203-204). New York: ACM Press.
Gray, W. D., John, B. E., & Atwood, M. E. (1993). Project Ernestine: Validating GOMS for predicting and explaining real-world task performance. Human Computer Interaction., 8(3), 237-309.
Gray, W. D., & Anderson, J. R. (1987). Change episodes in coding: When and how do programmers change their code? In G. M. Olson, S. Sheppard, & E. Soloway (Eds.), Empirical studies of programmers: Second workshop. Norwood, NJ: Ablex Publishing Corporation.
Back to Wayne Gray's home page / To Research Interests Index / To Downloadable Pubs / The CogWorks Lab / Cognitive Science Department
Last changed: 2004-03-01 wdg