 |
Implementation,
Evaluation and Revision |
The purposes of evaluation are to:
-
Assess learner's achievement of objectives and
-
Evaluate the effectiveness of instruction
In this section we will cover how to address the second purpose.
Formative Evaluation
| One-to-one
evaluation | Small-group
evaluation | Field trial |
| Materials | Draft | Prototype
| Near Final |
| Audience | One target
student | Few target
students | Many target
students |
| Who conducts | Developer or
Instructional
Designer | Developer or
Instructional
Designer | Developer or
Instructional
Designer |
Level of
simulation | Low | Moderate | High |
| Purpose | Verify:
Accuracy
Completeness
Relevance
Materials | Verify:
Lesson structure
Achievement
Relevance
Materials | Verify:
Field conditions
Timing
Attitudes
Instructors |
Conduct Formative Evaluation
After a draft of the instructional material is completed, evaluations are done to determine how to improve the instruction. There are often several steps in a formative evaluation.
Dry Run
Before the materials are tested with learners, a dry run may be conducted, where an expert in the content area and in instructional designer review the materials.
subject matter dry run
At this stage, someone familiar with the subject matter goes over the draft material to determine the accuracy and currency of the content. He or she may also evaluate if the lesson is relevant and complete, while excluding unnecessary material.
instructional design dry run
At this stage, someone familiar with instructional design goes over the material to evaluate the instructional adequacy of the instructional strategies.
One-to-one evaluation
This stage is when a student (member of the target population) goes through draft materials of the lesson with the instructional designer. The designer observes the learner for areas of the lesson which cause confusion or questions. Does the lesson help the learner accomplish the lesson goals and objectives? Are directions clear? Are the exercises and assessments effective? Was your instructional analysis accurate?
Small-group evaluation
At this stage, a few students who are members of the target population go through prototype materials to verify that the structure of the lesson is appropriate for students, the materials are relevant, and the materials are acceptable.
Field Test
At this stage the lesson is implemented with the target population of learners under actual teaching conditions. The designer can determine the actual length of exercises, assessments, and presentations or self-paced tutorials. Was the audience analysis accurate in terms of what the learners know about the subject matter? Did the students enjoy the instruction? If the developer is not the instructor, this may also be the opportunity to evaluate instructor skills and knowledge.
Practice
Using the multimedia quality checklist in your packet to conduct an instructional design dry run, look over your lesson. What looks good? What would you like to revise? Then exchange lessons with someone else and look over another lesson. This is particularly helpful if you are familiar with the subject matter of the other lesson, but it is not necessary for this exercise. What suggestions for improvement can you make?
Revise Instruction
Based on the formative evaluation, reexamine the validity of the instructional and audience analyses, and make appropriate changes in the instruction. It may be necessary to prioritize the revisions, and carry out high priority revisions first. High priority revisions are most often those which focus on crucial content which gets learners "up to speed" or "on the job" quickly and effectively. Make sure the instruction and technologies work effectively, then add desired bells and whistles
Implement Instruction
At this point, the lesson is ready for implementation. However, ongoing evaluation can help detect changes in the audience skills, knowledge and attitudes. It can also help you maintain quality instruction. If you used the rapid prototyping process, you will continue to revise to continuously improve and add bells and whistles. As you revise, you need to continue to evaluate the "new and improved" lesson.
Conduct Summative Evaluation
This step examines the relative worth of instruction. This is often conducted by an independent evaluator, rather than the instructional designer. This is the type of evaluation conducted when you select instructional materials for purchase. Some of the things you may examine during an evaluation for purchase are:
- technical quality of the materials
- accuracy, currency of the contents
- relevance of the instruction to your student's needs
- flexibility, durability, convenience of the materials (and the availability of equipment and facilities at your location)
- evidence of effectiveness
- cost-effectiveness
Businesses often continue to evaluate after implementing training to determine if learners are using their new skills and knowledge, and if there are operational results (savings or improved productivity) following training.
A cost/benefit analysis may be done at this time. This is done by computing the costs of the training, estimating the benefits, and comparing to determine if benefits exceeded the costs.
[Support Materials] [Presentation Support]
