Make Your Evaluation System Proactive

Patti P. Phillips, Ph.D., CEO, ROI Institute and Jack J. Phillips, Ph.D., Chairman, ROI Institute

This article was originally published on October 14, 2020, on TD.orgย 

Recent surveys and reports suggest learning and talent development professionals are not stepping up to the evaluation challenge of showing the value of what they do in terms that sponsors, supporters, and funders can appreciate. In other words, they donโ€™t really connect their programs, projects, and initiatives to key business measures. Yet, they all confess that this is needed. Perhaps the more significant challenge is the concern that talent development programs may not be adding value at this level. If evaluation results are disappointing, not many talent professionals will want to measure it or report it. Essentially, this is a fear of disappointing results, including a negative ROI. To tackle this challenge, talent development professionals need to reframe their approach to evaluation. Evaluation must be seen as a proactive process, and not a process applied only after a program is complete. This article describes how a proactive approach to evaluation would design, measure, and deliver the desired impact and ROI of various programs.

The Concern

There is much confusion about what evaluation adds to the organization. This is because many evaluation systems are more reactive than proactive. In essence, they function like an auditing system. They check to see what has or has not happened, and then categorize the results as either โ€œit workedโ€ or โ€œit didnโ€™t work.โ€ When the evaluation results are distributed, those who own the โ€œit didnโ€™t workโ€ efforts and those who funded those programs are not so happy. This approach to evaluation must change because the number one reason that programs are not evaluated at the impact and ROI levels is the fear of negative results. We all understand this concern: If your programs are not working, why publish a report and tell everyone? With a proactive approach to program evaluation, this issue essentially goes away.

The Needed Approach

A proactive approach is a logical extension of some evaluation systems. After all, when you evaluate a program, you are doing it for the organization or the client who wants the program to be successful. Why not structure an evaluation model that helps you achieve this success? This requires a change in thinking and a determination to share the roles and responsibilities with the entire team. The key is to design for the results that you want, and the evaluation system will influence design and outcomes. When everyone has a part in the process, achieving the desired results is not that difficult. Talent development professions can do this by implementing five straightforward steps.

Five Steps to a Proactive Evaluation: Designing for the Results You Wantย 

Step 1. Start with Why. As you start to plan the evaluation, explore why you are implementing the program. This is determining the business impact or business needs. As our partner in Australia proclaims, โ€œStart where you want to finish.โ€ With this as your first step, you are focusing on impact through all the steps in the process. This proactive approach ensures, in the beginning, that the program connects to the business. Too many programs do not connect to business measures. In fact, most learning programs do not connect to the business. They are often based on something that participants need to know (learning) or need to do (application, behavior).

Recently, in a conversation with a talent development professional, we discussed a new agile management program that was being implemented in their organization. The programโ€™s funder was asking for the impact and ROI of this program. We asked, โ€œWhy are you implementing this program?โ€ Our colleague answered, โ€œOur management wants us to obtain the six skills of agile management.โ€ We then asked, โ€œWhy do managers need these skills?โ€ This question led to an uncomfortable conversation for some because just having the skills in place is behavior, not the business need. For most funders, the behavior is not enough.

If you begin with the behavior, you will end up with behavior, as highlighted in Stephen Coveyโ€™s Habit #2 โ€œBegin with the end in mind.โ€1 The individuals who initiate programs must take change their thinking and realize their role is to ensure the programs have an impact. The executivesโ€™ perspectives are well known, and research shows executives want to see the business connection of learning. They want to see the impact of learning and talent development programs.

Step 2. Make it Feasible: Select the right solution. Sometimes the suggested solution may not be the right solution. It happens. You must take steps to ensure the solution drives the business measure. This may require you to have performance discussions or conduct some analysis. The analysis may involve interviews, focus groups, examining organizational records, or using diagnostic tools. Taking these necessary steps ensures the solution will improve the particular business measure.

Step 3. Expect Success: Plan for the results. An evaluation system must help the organization be successful. You expect success by having specific objectives. When a new solution is implemented, objectives need to be developed. These objectives should include, not just routine learning objectives, but objectives for application (using the learning), and impact (the consequences of application). Objectives are set at each level, Level 1 (Reaction), Level 2 (Learning), Level 3 (Application), and Level 4 (Impact) and then provided to the different stakeholders (designers, developers, facilitators, managers of participants, and even the participants).

Step 4. Ensure that stakeholders are designing for the desired outcome. Challenge the stakeholders to design the program to achieve the objectives, including the impact objectives. This can be accomplished with tools, templates, processes, encouragement, and suggestions aimed at not only learning but the use of the learning and the impact from it. This pushes the success of learning from just acquiring knowledge and skills to actually applying the learning and ultimately delivering the impact. This is what executives and funders want to see and what the programโ€™s sponsors supporters need to see, the business connection. Taking this step ensures that each stakeholder clearly understands their role in delivering the success that is needed.

Step 5. Measure results. With a proactive evaluation process in place, you can then can apply the classic evaluation strategies of collecting and analyzing data to examine the programโ€™s results. When things are not going well, the data will show what changes need to be made to improve the results. And if things are going well, the data will show how to make the program results even more successful.

The Payoff

When you take these five steps, your program is almost guaranteed to deliver positive business results because you have ensured it is connected to the business, itโ€™s the right solution, everyone knows what is expected, and everyone has designed for the results. This places you and your organization on the path to success. This proactive approach shifts the evaluation process from reporting to driving the results you want and need.

For some, this might be fundamentally in conflict with evaluation, but this should not be the case. What you actually want from an evaluation system is process improvement. You want to make improvements if the program is not working. Why not have processes in place to avoid barriers, breakdowns, and stumbling points along the way? Removing those barriers paves the path to success. A proactive approach to evaluation creates a significant shift in the way programs are developed and implemented. For more detail on this approach and a case study showing how a proactive evaluation works, please contact ROI Institute.

Reference

1. Covey, Stephen R., The 7 Habits of Highly Effective People. New York, NY: Simon & Schuster, 1989.