Why   |  Who   |   What   |   When   |   How   |   Articles   |   Forms   |  Resources

The ADDIE model (Analyze, Design, Develop, Implement, Evaluate) is the generic process traditionally used by instructional designers and training developers. The five phases represent a dynamic, flexible guideline for building effective training and performance support tools. Though ADDIE is not the only valid model, it is a commonly used approach that can be effective in almost every situation. Most of the current instructional design models are spin-offs or variations of the ADDIE model.

• Evaluation
  • Evaluation can be formative and summative. The latter is the more common at RPI and accomplished through the IDEA form evaluations. During this phase, the effectiveness and efficiency of the instruction are examined, areas of strenghts and weaknesses are identified, so that adjustments can be made. During this phase, the instructional designer
    • makes sure the materials achieved the desired goals
    • evaluates the plan from all levels for next implementation
  • Sample task: Record time data, interpret test results, survey students, revise activitie
  • Sample output: Recommendations, project report, revised prototype

The following was adapted from material at http://www.ion.illinois.edu/

Why Evaluate?

• Identify problems
• Identify strengths
• Suggest improvements
• Assess effectiveness

back to top

Who Should Evaluate?

• Participants
• Instructor
• A third-party

back to top

What to Evaluate?

• Use of technology
• Teaching methods - effectiveness of lecture, discussion, question and answer
• Quality of questions or problems raised during instruction
• Encouragement given students to express themselves
• Instruction atmosphere - conduciveness to participant’s learning
• Quantity and quality of interaction with other participants and with instructor
• Instruction content - relevancy, adequate body of knowledge, organization
• Assignments - usefulness, degree of difficulty and time required, timeliness of feedback, readability level of print materials
• Tests - frequency, relevancy, sufficient review, difficulty, feedback
• Support services - facilitator, technology, library services, instructor availability
• Participant achievement - adequacy, appropriateness, timeliness, participant involvement
• Participant attitude - attendance, assignments submitted, class participation
• Instructor - contribution as discussion leader, effectiveness, organization, preparation, enthusiasm, openness to student views

back to top

When to Evaluate?

• Throughout
  • Will enable the instructor to improve the course as he/she proceeds
  • Facilitates course and content adaptation
  • Will identify major gaps in the instructional plan or the need for minor adjustments
• At the end
  • Assesses overall effectiveness of the finished product or course
  • Can be a springboard in developing a revision plan
  • Can be a baseline of information for designing a new plan, program, or course
  • Will not help current students since it is conducted upon course completion. A while later
• A while later

back to top

How to Evaluate?

• Online forms
• Independent (third-party) service (i.e. course developers)
• IDEA (see the section on IDEA in Analyze)

back to top

Articles

An Introduction to Classroom Assessment Techniques
By Diane M. Enerson et al.
Provides overview of assessment techniques, makes general recommendations for assessment choices, and provides classroom examples.
http://www.schreyerinstitute.psu.edu/pdf/classroom_assessment_techniques_intro.pdf

Assessment that Promotes Learning
By John P. Lowe
In this article the author shares ways to use the consuming student concern about tests and grades and points as a vehicle not only to measure how well students have learned material, but also to assess and improve how they learn, how they think, and how they come to value certain ways of learning and thinking and knowing.
http://www.schreyerinstitute.psu.edu/pdf/Assessment_That_Promotes_Learning.pdf

Evaluating Training and Results
By Carter McNamara
A rich site that provides information on perspective on evaluating training
http://www.mapnp.org/library/trng_dev/evaluate/evaluate.htm

Evaluating Training - There is no "cookbook" approach
By Fred Nickols
An article on what to evaluate, and why.
http://home.att.net/~nickols/evaluate.htm

Formative Evaluation: A Practical Guide
By Lisa Neal, eLearn Magazine
http://www.elearnmag.org/subpage.cfm?section=tutorials&article=25-1

Measuring Training Effectiveness / Impact - Training can be measured in a variety of ways
By Dr. John Sullivan
Variety of ways of evaluating training.
http://ourworld.compuserve.com/homepages/gately/pp15js18.htm

Podcasting Lectures
By Sarah Brittain, Pietrek Glowacki, Jared Van Ittersum, and Lynn Johnson
Formative evaluation strategies helped identify a solution to a learning dilemma.
http://www.educause.edu/apps/eq/eqm06/eqm0634.asp?bhcp=1

ProfPort Webfolio System: Implementation, Curriculum and Assessment
By Paul Gathercoal, Douglas Love & Gerry McKean
This article argues how it is possible that webfolio systems can challenge the mystique and authority of standardized tests as the guiding force behind education today.
http://www.educause.edu/ir/library/pdf/EDU0329.pdf

Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses
By Charles Graham, Kursat Cagiltay, Byung-Ro Lim, Joni Craner and Thomas M. Duffy
An article on how the "Seven Principles for Good Practice in Undergraduate Education,", a popular framework for evaluating teaching in traditional, face-to-face courses, were applied to online courses.
http://sln.suny.edu/sln/public/original.nsf/0/b495223246cabd6b85256a090058ab98

The Ten Rules for Perfect Evaluations
By Jay McNaught
A sobering article on choosing between training excellence and great evaluations.
http://www.karinrex.com/tc_evals.html

back to top

Forms

IAS online: Office of Educational Assessment – Online Course Evaluation
http://depts.washington.edu/oeaias/iasquest.htm

Is Online Teaching for Me? TM - Self-evaluation Quiz
http://www.onlinelearning.net/InstructorCommunity/selfevaluation.html?s=526.5020h0455.007z328j70

Online Course Evaluation
http://www.ctdlc.org/evaluation/evaluation.cfm

Online Course Evaluation
http://www.uab.edu/dcdc/misc/sample_eval.html

University of Bridgeport Faculty Course Evaluation Form
http://www.bridgeport.edu/Indexhtml/Centers/Disted/faculty.html

back to top

Resources

Evaluation Tools
A list of links to evaluation tools and descriptions of each.
http://www.ceismc.gatech.edu/MM_Tools/evaluation.html

Instructional Design: Evaluation
Principles of instructional design evaluation.
http://www.fgcu.edu/onlinedesign/designDevd.html

The Proof's in the Quick Quiz
Company owner assesses efficacy of management training by giving employees yearly pop quizzes.
http://www.inc.com/magazine/19940301/2826.html

Student Evaluation: A Teacher Handbook
Table of contents with links to all the different chapters of Student Evaluation: A Teacher Handbook
http://www.sasked.gov.sk.ca/docs/policy/studeval/index.html 

Approaches to assessment and feedback that foster independent learning
A series of well designed modules that cover critical assessment issues such as the integration of assessment strategies with the specific aims and objectives for student learning and the implementation of both formative and summative assessment.
http://www.deakin.edu.au/itl/pd/tl-modules/assessment/index.php

back to top