Read Part I, which covered Evaluation Planning, Part II, which covered Data Collection, and Part III, which covered Data Analysis prior to reading Part IV: Reporting
Case Study
Using the Phillips ROI Methodology™ to Evaluate the XYZ Widget Manufacturing Company Leadership Development Program
Phase 4: Reporting
In Phase 4, we completed the final report and communicated the results of the Business Impact and ROI Study to the executives, Board of Directors, and other interested parties of XYZ Widget.
A detailed report was compiled that included the following information:
- General background information about the Business Impact and ROI Study, including the objectives behind doing the study and our expectations of an outcome
- General information about the ROI process
- Information about what data was collected and how the data was collected (surveys, etc.)
- The results from all six measures: Level 1 – 5 and intangible benefits realized
- Information about how the effects of the program were isolated
- Details about converting the data and detailed costs of the solution
- How ROI was calculated
- Interpretation of the results – conclusions and recommendations for moving forward
- All of the backup documentation: copies of surveys, worksheets, raw data collected, etc.
For details of the data analysis, review Part III of the post.
Presenting the Results
There were three distinct groups who were the target for the communication of the results:
- Executives and Board of Directors: These individuals were presented with a 10 page executive summary that focused on the process used to measure the business impact and ROI of the leadership development program and the results of that measurement. A 2 hour meeting focused on the ROI component of the study (the process used, how ROI was calculated, etc.) and discussed the value of the program to the organization and plans for ongoing development of the high potentials. The executives and board were most interested in the return on their investment in the program. Access was provided to the complete report and all appendices.
- Functional area managers, vice presidents, immediate managers of the participants, and mentors: These individuals were presented with a 20 page report that focused on all of the data collected and analyzed. They were also presented with the appendices of the results of the surveys and other data that was collected. Discussion focused on next steps in the program and support needed from the participants’ immediate management and their mentors to continue their development. Access was provided to the complete report and all appendices.
- Learning & Development staff: These individuals were presented with the complete report and all appendices. The focus was on the process used to evaluate the leadership development program, how data was collected and analyzed, and the final results of the study. Discussion focused on the use of business impact and ROI for other relevant programs within XYZ Widget. Additionally, processes were discussed and agreed upon for future ROI studies. This first study was a learning process for many of the learning and development staff.
Prior to beginning the study, and throughout the study, all of these groups mentioned above were part of the original project communication plan – they were all stakeholders in the project. This “final” communication was to wrap up the study and show them the benefits of such a study for a leadership development program or any other program with links to the organization’s long-term strategy and growth objectives.
The executives and board of directors were particularly interested in the study and the results for a variety of reasons, including:
- The need for a strategic plan to groom high potentials to take over leadership roles within the company. You’ll recall from the original post,Developing a Leadership Training Program for High Potentials: A Case Study, that many of the executives were expected to retire within a short time frame.
- The training program developed for the high potentials provided what was needed to ensure these individuals had the skills and knowledge to lead the company in an increasingly competitive global economy.
- Their investment in the program, which was not insignificant, was a worthwhile investment and needed to show a positive ROI for the company.
Continued Investment in the Leadership Development Program for High Potentials
Given the ROI realized from the program (2,164%, see Part III for the details), the executives and board of directors approved the program for all high potentials within the company. Some adjustments were made to the program itself, including:
- The mentoring program would begin on day one of the program, rather than having mentors assigned at the end. Since the mentors were such a key part of the high potential’s development, it was determined that having them available and assigned at the very beginning would provide the participants further support and guidance.
- The portal would be part of the program since it was such a successful tool for the current group of participants to have a place to “gather” to brainstorm, share ideas and group problem-solve.
- A job-sharing component was built in so that each high potential was able to spend time learning about other function areas. As an example, manufacturing function area high potentials would also spend time working within the planning and procurement and materials management functions.
- Since the company was having more of a global presence, an additional course was added to the curriculum on cultural diversity. As the company grew globally, it was also expected that a future component would include spending at least a year or two overseas working in a foreign facility.
- The program was changed to a three year program to accommodate the job-sharing component and increased mentoring support. Additionally, since it ran over time for the first run, it was felt that by extending it, there would be less pressure on people’s schedules.
Summary
XYZ Widget continued with their leadership development program for high potentials, completing a business impact and ROI study with each group that attended the program. Results were successful for each run of the program. The nomination process continued for the program, enabling a large variety of individuals to participate. Word of mouth about the program spread, and it was a “badge of honor” to be selected to participate. Individuals worked hard to be selected for the program and the program became a valuable way for XYZ Widget to groom their high potential employees to add value to the company and be prepared to take on leadership roles.
Reference: Return on Investment in Training and Performance Improvement Programs, Second Edition (Improving Human Performance)
