We know it is important to measure the effectiveness of a training programs, yet we don’t always take the time to do it right. Why is that? Are we too busy and just don’t have the time to do so? Maybe we don’t think it is all that important after all. If the participant in the class believes they learned new skills and tell us they increased their knowledge – maybe it’s sufficient? Hmmm..…I don’t think that is what we believe. Maybe we just don’t know where to start! We want to start slowly for sure – let’s not jump into doing a business impact and ROI study just yet! (We do want to get there at some point though.)
Based on the last poll (which is certainly not statistically accurate but interesting nonetheless) it seems that much of the time we gauge the effectiveness of training through a survey. If some of the respondents to the poll are similar to many clients I work with, they are thinking of a simple survey that asks questions such as:
- How was the training?
- Did you learn new skills in this class?
- Are you pleased overall with the training you received?
Or, as it is frequently called, “the smile sheet.” Not much good information coming out of these surveys.
I can understand the issue of not having enough time to measure the effectiveness of a training program; especially if you have quite a few programs to oversee or running training programs is just a small part of everything you do (we all wear so many hats!). However, you can do better than just the standard smile sheet without increasing your workload significantly and getting some better information to share with others in the organization. And, moving yourself closer to your goal of effective measurement of your training programs.
Some additional questions on the survey will provide you better data on the effectiveness of the training program. For example, add the following questions to your survey:
- What percent of your total work time will require the knowledge and skills presented in this training?
- On a scale of 1 to 10, how critical is applying the content of this program to your success on the job?
- What potential barriers could prevent you from applying the knowledge and skills learned in the course?
- What specific actions will you take to implement your new skills and knowledge on the job?
- What are the benefits the company will realize based on you applying your new skills and knowledge on the job? (Be as specific as possible.)
- What would happen with respect to costs and/or productivity if you were not able to apply your new skills and knowledge to your job? (Be as specific as possible.)
I would also recommend that you ensure each participant develop an action plan after each training course so that they have a path to applying those skills. See my previous post, Make the Training “Stick,” for information on action planning and a sample completed action plan.
As you can see, just adding some additional questions to your current survey will help you gather more significant data to determine the effectiveness of the training program. Not much more time commitment on your part, but the data you gather will enable you to better determine what improvements need to be made in your training programs. You may also learn what needs to be done in the organization to better support the training. For example, the question, “What potential barriers could prevent you from applying the knowledge and skills learned in the course,” will provide insight as to why skills may not be applied back on the job. You may need to address some organizational issues or management barriers that are prohibiting the participant from applying his/her new skills.
Follow-Up
A follow-up survey, 2 – 4 months after the training program ends (or when the individual will have the opportunity to apply the skills), will enable you to find out how successful the individuals have been in applying their skills back on the job.
The follow-up survey should focus on questions such as:
- What is enabling you to provide your skills back on the job?
- What barriers have you faced on applying your skills?
- How have you applied your skills? (Be specific.)
- What impact have you had on the business because you have applied your new skills and knowledge? (Be specific – explain how you calculated/determined that impact.)
How About a Manager Survey?
Can I push you just a bit more? How about at the same time you send out the follow-up survey you also include a brief survey to the manager of the participant? This survey is to gather data from the manager on how they are seeing the application of the skills by the participant. For example, questions you ask the manager might include:
- What improvement in skills have you seen based on the training program the participant attended?
- What business impact have you seen because of the application of these skills?
- What percentage of the new skills and knowledge is directly applicable to the participant’s job?
- Has the participant been assigned any new projects or responsibility due to this training program? Please explain.
Your thoughts? How easily might you incorporate just a few extra questions into your survey to improve the data on the effectiveness of your training? How easily can you launch a follow-up survey at your organization? Please provide your thoughts in the Comments field below. Thanks!
Next post: Hey Project Sponsor – We Need Your Help Please!
Hello Andrew,
Thanks for your comment. Regarding your questions:
1. I am referring to either internal or external training – it would apply equally well to either, and I have applied it to both. For the survey question – “What specific actions…” for the clients I am working with it is not too probing a question – likely you have already built a trusted relationship with them. However, let’s assume that the client did feel it was too probing. Then I might reword it to state: “What are some ways you will apply your new skills and knowledge on the job?” and have them pick from a list that is associated with the objectives of the course (so I am removing the “specific ways” part of the question). If we use product training classes for clients as an example I might include on the list for them to choose how they will apply the skills the following:
– Will implement product X organization-wide
– Will hold an internal training of others in my department or in the organization to use product X
– Will use product X for all virtual meetings and/or virtual training sessions
2. You can use a multiple choice option to better roll up your results for larger audiences. For example, for one of the suggested questions in the post: “What potential barriers could prevent you from applying the knowledge and skills learned in the course,” I would provide them options to choose from such as:
– no management support
– current processes do not enable me to apply the skills
– my current job does not require the use of these skills
If you are considering doing business impact and ROI, I would suggest you purchase the book: How to Measure Training Results : A Practical Guide to Tracking the Six Key Indicators by Jack Phillips and Ron Drew Stone.
Great reference and easy “how to” guide.
If you’d like to discuss further – I’d be happy to talk to you and help you brainstorm how you might improve your measurement.There are also some templates I have used that you might find helpful. Happy to share with you. Feel free to contact me via the “Contact Me” link on the left.
Thanks again for your comments! I am glad you found the post thought provoking.
Best regards,
Gina
Pleasure reading your Post – quite useful. I totally agree about how incomplete the “smile sheets” are for training feedback. Most people have no desire to trash their trainer and certainly don’t we do be bothered with some (how can we do it better) follow-up call if they did, so they typcially whisk through the feedback form with a series of 3s,4s and 5s with nary a 1 or 2.
I have two questions.
1) I have to ask what type of training program(s) are you referring to — there are two major kinds, “internal” and “external”. What I draw from your perspective is that your intended audience for this post is those who are receiving workplace training within the enterprise. Are your training measurement suggestions as applicable to “external” training programs, such as product training classes for clients?
Choosing one of your survey question examples, would you agree (or disagree) that the following question, “What specific actions will you take to implement your new skills and knowledge on the job,” is a bit too probing a question for a client/customer? How would you phrase your questions differently for external trainings.
2) By adding all of these more subjective questions, it’s certainly true the reader of those responses will gain more insight into the general effectiveness of the training program. But what if we had to scale this survey to vastly higher volume of attendance making the reading of comments too time consuming. With these multiple datapoints from the survey, is there way to translate them into some type of overal “score” of effectivness. Managers always like numerical meausurements. Are your suggested additions to the “smile sheet” well suited for scoring?