Activities

Pilot Testing and Revision of the Module

Pilot Testing Model

The first level of pilot testing involves a one-on-one interview with 2-3 students.

What to look for:

• Students who have the entry behaviour (prerequisites) but not the terminal behaviour (the students can be paid or volunteers)
• Students who are highly communicative

What to do

Prepare the students for the pilot testing interview:

• Reduce the student’s text anxiety (“I want to test this material, not you. Any difficulties you run into will help me locate portions of the material which need improvement.”)
• Reduce the student’s aversion to criticizing (“This is a module that has been developed by a group of people. They are asking you to help them locate errors, difficulties and weak spots in it so that it is improved before it is used in an actual course.”)

During the interview:

• Observe for any sign that the student is having trouble: he/she stops working, frowns, deletes something, gazes into space, flips back in the material.
• Ask the student to help you revise awkward directions on the spot.
• You can record your comments or possible revisions on a student tryout evaluation sheet or a blank sheet of paper, but the easiest by far is record everything on a duplicate copy of the material.
• Be careful not to give additional explanations, known as over cueing, since nothing you say in the test session to help the student will be there to help the actual students.

During the same time frame as the one-on-one pilot testing with a couple of students, you can also give the material to several content experts to verify the accuracy of the content and to get their feedback. For example, you could give your draft objectives to several colleagues and ask them what they think they would have to do to demonstrate their mastery of the objective. If their description matches what you have in mind, then the objectives will pass to the next level of pilot testing. If not, they will need to be revised.

The next level of pilot testing involves a limited group of 8-20 students and is based on:

• how students perform on the post-test (pre/post-test comparisons);
• how students respond to activities within the teaching material;
• comments they make on a feedback form.

The following level of pilot testing in this model targets 30 or more real students under authentic conditions to field test the module. This often means limiting the use of the module to just one section of the course and allows for more data to be gathered on the effectiveness of the module.

The final level of pilot testing takes place once the module is being used on a regular basis. Evaluation at this stage involves periodic reviews of the objectives and teaching material to verify that everything is still up to date and appropriate for that module.

As Russell (1974) points out, however, there is no magic or sacred number of tests that should be conducted. The student tryout cycle is one of: design, test, revise, test…revise, test. The ultimate goal is to revise the module until most of the students are able to master the objectives.

Helpful Tips

• Keep the material in rough form. Developers tend to be very reluctant to change or modify the material even when the student tryout dictates it if they spent a lot of time and money to produce professional products before the student tryout.
  
  • e.g. If you plan to use an audio clip but do not want to record the information during the early stages of student tryout, you can put the instructions on cue cards and read the material to the students. (Russell, 1974, p. 80, 82)
• Keep the study guide and any other handouts in small, manageable pieces so that if the material need to be revised, it’s easy to rearrange or re-sequence them.
  
• Keep the original student responses given during the pilot testing. They can be used later as models or as the correct answers in the final version of the module.

Final Summary

This pilot testing model is used at the end of the Systematic Design of Modular Instruction that has been presented here yet it is a dynamic and iterative process that is continually refining the module until the targeted objectives can be mastered satisfactorily. It is a process that is based on proven pedagogical principles and best practices that leads to a more systematic and effective approach to the design and implementation of modular instruction.