8

The Pilot Class and Beyond

 

At this point, your course has been accepted, and you’ve made it to the pilot class stage. As happy as you may be, your job is not quite done. You need to first be able to effectively transfer the project to the instructor of the course, and then you need to observe and analyze the course in action to determine whether your objectives and design are sound. In addition, you should put some steps in place so that your material can be more easily updated and kept relevant. These items will be discussed in this chapter.

Helping Out the SME-Turned-Trainer

Typically, subject matter experts (SMEs) also double as instructors of technical training classes. While there are rare times when a complete nonexpert teaches a technical topic or when an SME is permanently paired with someone who specializes in classroom instruction, you will find these to be the exceptions, rather than the rule. Most of the time, an SME will instruct the class him-or herself. As such, there are certain things you can do to help out the SME-turned-trainer both before and during the pilot class.

Course “Dry Run” Before the Pilot

You can help out the SME-turned-trainer by having a run-through of the course materials before the pilot class. Explain the rationale of the organizational structure, the exercises, and any other particulars of the class. If possible, have the instructor practice the presentation or complete portions of the exercises him- or herself.

You should go over course content verbally with the SME-turned-trainer, but you might also consider having an instructor’s guide printed out. Creating an instructor’s guide can really help ensure that your course will continue to be delivered as originally designed and developed. This instructor’s guide can be as simple as a one-page typed schedule, or it can be as detailed as a full-blown facilitator’s guide with descriptions, rationale and answers to exercises, notes included for the PowerPoint slides, and so on.

The course dry run is also a good time to review adult learning principles. If your instructor seems hesitant about interactivity or any of the other instructionally related features of the course, you might provide resources that prove that adult learning principles are tested and sound. It can ease your instructor’s mind if you explain that you will be there to assist in the class (for example, by helping to explain exercise instructions or to provide guidance in keeping the course on track). This not only can help calm your instructor’s nerves about trying something new, but will prepare the instructor for you to stand up and help out during the class.

Resist any urges to change material—unless it is incorrect—at this stage if the pilot class will be happening sooner than would allow for completion of the requested changes. Ask the SME to give the course a try as it is and then explain that you’ll be open to removing anything that doesn’t work after the pilot.

Assistance During the Class

Don’t be afraid to stand up and help out the instructor during the pilot course. Think of yourself as a co-facilitator, especially when it comes time to explain the instructions for participant exercises. If you notice the instructor getting completely off track, ask if the class can have a break and privately discuss this with the instructor.

If you have to jump in to help, it will usually be when the exercises occur. Flag the exercises the instructor has trouble with during the pilot. Make a note to add additional detailed written instructions and possibly even a sample completed problem and answer for the learners to view before completing the exercise themselves. These types of things can augment some of the instructor’s verbal directions.

In addition, you may need to rethink whether to include certain kinds of exercises in the course. You may have created the best exercise ever, but if the instructor doesn’t use it, the exercise will not be effective. The intricacies and detail included in administrating the exercises you design should match the facilitation skills of the instructor. Some instructors just may not have the skills or motivation to try an exercise with a lot of steps. In that case, the exercise may need to be replaced with something else.

Approach instructor tangents just as you did when developing and gathering material with your SME: warily, but with some leeway available. Sometimes tangents occur because the instructor has noticed a hole in the content and that a new objective is necessary. Sometimes instructor tangents are OK—they may answer learners’ questions and aid in their understanding. It will be up to you to determine whether the tangents are ambling mindlessly or are fit-for-purpose.

Effectively briefing and supporting the SME-turned-trainer is just one aspect of the pilot class. The next section will discuss development opportunities from the pilot class.

Development Opportunities from the Pilot Class

During the pilot course, both you and the instructor should be analyzing whether

• course objectives are met through the training content provided
• there is a logical flow of material
• technical explanations are complete
• unnecessary information is removed
• course exercises are challenging, useful, and relevant.

Attendees in the course can be great sources to help you determine whether the conditions listed above describe your course. Attendees can be polled or pre-and post-tested to determine whether your course objectives really are on track and whether your design of material actually helps to meet course objectives.

When considering input, you have a couple of choices for the pilot class: You can tell the learners the course is a pilot run, or you can avoid letting on to anyone that this is the first time that this course has been taught.

Option 1: Tell the Learners It Is a Pilot Run

Sometimes, if a course is highly anticipated, it will be next to impossible to keep it a secret that this is the first time it has been run. In addition, many instructors choose to explain to learners that the course is a pilot run because those instructors are not confident with the material, so saying it is a pilot serves as a disclaimer that makes the instructors feel better and above potential criticism.

A benefit of sharing that it is a pilot class is that you can request feedback and hold periodic open and honest discussions about whether course content is effective and working. You are basically attempting to enlist the learners as co-creators and trying to make them feel invested in the outcome of the course. If you are able to achieve this, learners’ feedback will be especially valuable.

A possible drawback of this is that the learners may not be as objective with their comments and evaluations; they may be more forgiving because they may not think the content is “final.”

Option 2: Don’t Let On That It Is a Pilot Course

This allows you to get more objective reactions of course content from learners. You get to see what their reactions would be if the content presented was the finalized version, including what portions learners like and what portions may need more work. Of course, you will probably already have ideas about these things, but the feedback is still quite valuable and more “pure.” Often, you will be surprised with at least some aspects of what learners thought was effective or not effective.

Regardless of which route you choose, it is important that you tap into learners’ reactions during the pilot course. This will give you a chance to see how learners respond to the material, both positively and negatively. Keep an eye out during the class to see where they struggle and where they are not engaged. You can document questions learners have and different content areas that appear difficult to understand. Don’t be afraid to ask the learners directly for feedback. Speaking to the target audience firsthand can help you to figure out what works and what does not, and which objectives are needed and which are not. You will probably find that the learners have good suggestions for activities and content, and many will even possess hidden gems of relevant technical information.

As the pilot class proceeds, you should be analyzing objectives, content, areas for interactivity, learners’ reactions, their questions, and extra things the instructor says (things not mentioned in the presentation or the materials). Below are specific items to consider when sitting through a pilot class:

• Are learners engaged in all sections of the course? Is there something we could add to further engage learners?
• Does the target audience feel all objectives were met? That all objectives were necessary? That any important objectives were missing?
• Do learners express positive ratings or comments about the course structure and content?
• In which areas of the course did learners have a lot of questions? Were these questions answered by the current course material?
• What new explanations did the instructor include as the class occurred? Do those explanations need to be formally included in the content?
• Did learners find any inconsistencies or mistakes in the material?
• Did learners reference other resources that might be useful to include in the course?
• Are there any areas in which we can include a more complete or more updated example in the material?
• Are there any ways we can make the learners more responsible for their own learning?
• Do we make any general statements or assumptions in the material that are confusing to learners? How should we adjust the text, or what specific examples can we include?

From the information you gather from the pilot class, you should be able to make one more major update and then be done with your course (or at least be done with the first version of it!).

Other Opportunities from the Pilot

There is nothing like a pilot class to find the final pieces of missing information you need. For example, in pilot classes you could do the following:

• Give learners exercises to complete that—despite your best efforts—you were not able to get answers for yet. In cases in which you have extreme trouble getting answers, it is possible you will attain these answers from learners during the pilot. After they complete the exercise and share their answers, you can document the correct input and make these answers part of the permanent instructor’s guide.
• Ask learners in the pilot course to create questions for a needed final exam. At the end of the class, divide the learners into groups and assign each group a course objective. Ask learners to create five test questions based on what they have learned about their assigned course objective. For the pilot class, you can use these questions as a trivia game, which temporarily functions as the final exam. After the pilot, however, you should be able to use these test questions as a starting point for creating a permanent written exam for the next class.

These examples are a little scrappy, but sometimes measures like this are necessary. You can use the pilot as a way to finally find the rest of the technical information you are seeking for a course. It should be noted—importantly—that if you are going the route of an unfinished or unpolished pilot course, you must be aware of how you advertise the course, who attends, and what kind of “course credit” you give!

After you have updated your course accordingly based on the pilot class, your next step will be saying good-bye to your course for good . . . or at least for a little while.

Postcourse Maintenance

It is important to keep your course updated and relevant; yet it is also important that you be able to say “good-bye” to a project so that you can move on to your next priority. Postcourse maintenance can be a tricky thing, but there are measures you can take in order to keep a project “closed” for an appropriate amount of time, while still providing updated content to the learners.

Dealing with “Tweaks” in Material

It can be next to impossible to get your course absolutely perfect. Your SME (or you) will probably want to make a tweak here or there. Constantly opening and closing a project for small details, however, can be a major disruption in your work process and efficiencies.

You can meet this “tweak” challenge by logging all the small changes needed for a course and then waiting to deal with them until enough have built up to validate a major overhaul of the course. For example, as your instructor continues to teach the course, provide a worksheet for your instructor to log any improvements he or she thinks could be made. Set up an electronic file in which the instructor can enter new information and requested changes. Keep this information as long-term data to be incorporated into the next major redevelopment of the course.

Basically, you should set up a location for the instructor to keep track of eventual changes he or she wants made, and then periodically return to the class once enough major changes to the material are needed—that is, when enough changes are needed to justify reopening the project.

Keeping Content Updated

It is important that technical content stay up-to-date. One suggestion is to provide your instructor with a sample “addendum” worksheet template, so that he or she can list additional important updates, changes, and so on that come up and give this addendum worksheet to learners along with the “regular” course materials. This allows the instructor some control over keeping the content up-to-date, and it also prevents you from having to jump at every minor edit.

Getting It Done

As happy as you may be to have made it to the pilot course stage, your job is not done. You need to first be able to transfer the project to the instructor of the course, and then you need to observe the course in action to determine that your objectives and design are sound and relevant. The pilot course is an opportunity to test your development project. As the pilot class proceeds, you should be analyzing objectives, content, areas for interactivity, learners’ reactions, their questions, and extra things the instructor says. In addition, you should put some steps in place so that your material can be more easily updated and kept relevant.

Saying good-bye to technical training projects should be a happy event; from this day forward, you will have successfully provided the organization with a useful and relevant training course that can assist employee performance.

 

Worksheet 8–1. Pilot Course Evaluation—Developer and Instructor

Use the following worksheet to evaluate the pilot course. You can ask your instructor/SME to fill out the worksheet as well.

Based on the Pilot Course

Objectives are appropriate to the target audience relevant to the organization and business strategy specific, measurable, achievable, realistic, and testable.

Information is complete accurate concise.

Exercises meet course objectives engage learners include clear instructions are easy to administer.

In regard to course structure, overall organization of the course works flow of each section makes sense length of the course is appropriate level of content is appropriate.

Additional comments:

 

Worksheet 8–2. Pilot Course Evaluation—Learner

Hand out the following worksheet to learners after your pilot course in order to help you gauge learner reaction and capture learner comments. Adjust the worksheet as necessary to meet your own specific needs.

Please rate your knowledge level before attending the course.
Novice				Expert
1	2	3	4	5
Please rate your knowledge level after attending the course.
Novice				Expert
1	2	3	4	5
Course materials are organized and easy to follow.
Strongly Disagree			Disagree	Neutral	Agree	Strongly Agree
Course exercises helped me learn the content.
Strongly Disagree			Disagree	Neutral	Agree	Strongly Agree
The course will help me do my job better.
Strongly Disagree			Disagree	Neutral	Agree	Strongly Agree
Comments:
What did you like about the course?
How could the course be improved?
Would you recommend this course to others?

 

Worksheet 8–3. Sample "Updates to Content" Worksheet

The following is an example of a worksheet that you can give to your instructor/SME in order to allow him or her to provide learners with updated information related to course content that has not yet been formally incorporated in the course materials.

As you know, technology is always changing. Please note the following updates to the course material. Thank you.   Section 1

Section 2

Section 3

Additional comments: