Your cart is currently empty!
As funding becomes more difficult, executives need studies that show the value of engagement with a new initiative and this leads to the impact and ROI for the engagement expenditure. More studies are needed at the organizational level, comparing the monetary benefits of engagement with the investment to show the ROI, the micro-analysis in the figure.
The good news is this is being done and is the focus of our ATD book, Measuring the Success of Employee Engagement.1 The ROI studies presented in the latter part of the book show how this is being accomplished. Hereโs a summary of the steps.
ROI Analysis
At the heart of ROI analysis is the variety of data collected throughout the process and reported at different intervals. Most projects and programs use logical steps of value, and their sequential use can be linked to a variety of guidelines and models. The ROI Methodology, based on five levels of evaluation, is shown in Figure 2. This logical flow of data is appropriate for engagement as employees react to the engagement process, learn how to be engaged, apply and use engagement actions and concepts, and have an impact in their work. When impact is converted to money and compared to the cost of the engagement program, the ROI is calculated.
As evaluation moves to the higher levels, the value ascribed to the data by the sponsor/client increases. Accordingly, the degree of effort and cost of capturing the data for higher levels of evaluation also generally increase. With proper project planning and preparation, costs can be minimized.
Input Data
Level 0, Input, represents a category of data that reveals the volume, time, and cost of engagement programs. It includes the number of people involved and the time of their involvement, representing a fully loaded cost profile. It reflects all direct and indirect costs. Level 0 data do not represent outcome data, but they are important because they represent the scope and investment in engagement.
Reaction Data
The first category of outcome data collected is the perceived value of the concept of engagement (Level 1). At this level, a variety of key reaction measures are taken to gain insight into the value, importance, relevance, usefulness of the engagement programs, and the intent to โengageโ more with the team and the work.
Learning Data
As the engagement program is implemented, new information is acquired and new skills are learned. This level of measurement (Level 2) focuses on the changes in knowledge and skill acquisition. Some engagement programs have a high learning component, such as major changes in job design and work flow. Others may have a low learning component, such as needed brief engagement actions and behaviors.
Figure 2. Six Categories of Data
Application and Implementation Data
Application and implementation (Level 3) are key measures that show the extent to which employees are engaged, behavior has changed, and job performance has improved. This data reflects the extent of actions taken, skills applied, habits changed, and new processes are initiated, because of the engagement programs.
This is one of the most powerful categories because it uncovers not only the extent to which the employee engagement programs are successful, but also the reasons for lack of success. At this level, barriers and enablers to application and implementation are detailed.
Impact Data (Tangible and Intangible)
As employees become more engaged, consequences are monitored. These can be described in one or more measures in the work unit representing output, quality, time, and costs.
This level of data (Level 4) reflects the specific business impact and include measures such as sales, production, services delivered, errors, waste, accidents, retention, costs, innovation, job satisfaction, and customer satisfactionโthat have been influenced by the engagement program. A direct link between the business impact and the engagement program must be established to have credible results. This requires using a technique to isolate the effects of the program from other influences that may be driving the same measure. Answering this question is critical: How do you know it was the employee engagement program that caused the improvement and not something else?
Intangible benefits are impact measures that are not converted to monetary value, because the conversion to monetary values is not credible with a reasonable amount of resources.
ROI Data
This level of measurement (Level 5) compares the monetary value of the business impact measures with the total cost of the program. ROI is the ultimate level of accountability and represents the financial impact directly linked with the program, expressed as a benefit-cost ratio (BCR) or return on investment percentage.
Figure 3 provides an example of a small banking system with 700 employees. The engagement program addressed turnover reduction in the branches.2
The analysis along the chain of value is expanded to highlight the steps shown in Figure 4, The ROI Process Model. The most important data set for those who sponsor projects is the impact, which is the consequence of application expressed in business terms. However, showing the impact of a program isnโt enough for some executives. They want the ultimate level of accountability: ROI. To develop a credible analysis for impact and ROI, conservative assumptions and proven processes must be used. The ROI Methodology follows guiding principles as shown in Figure 5.
Figure 3. Secor Bank Example
Figure 4. ROI Methodology Model
Figure 5. ROI Standards, Twelve Guiding Principles
- When conducting a higher level evaluation, collect data at lower levels.
- When planning a higher level evaluation, the previous level of evaluation is not required to be comprehensive.
- When collecting and analyzing data, use only the most credible sources.
- When analyzing data, select the most conservative alternative for calculations.
- Use at least one method to isolate the effects of a program.
- If no improvement data are available for a population or from a speci๏ฌc source, assume that little or no improvement has occurred.
- Adjust estimates of improvement for potential errors of estimation.
- Avoid use of extreme data items and unsupported claims when calculating ROI.
- Use only the ๏ฌrst year of annual bene๏ฌts in ROI analysis of short-term solutions.
- Fully load all costs of a solution, project, or program when analyzing ROI.
- Intangible measures are de๏ฌned as measures that are purposely not converted to monetary values.
- Communicate the results of the ROI Methodologyย to all key stakeholders.
Final Thoughts
Every employee engagement program should be evaluated in some way, even if itโs only collecting reaction data from those involved. The challenge is to collect additional data at higher levels, and to do so only when it is relevant and feasible. Level 3 evaluation, which shows how well employees are engaged in the workplace, is very common with standard and customized engagement surveys. If the cost of the program increases, sponsors may ask for an impact (Level 4) or even ROI (Level 5) evaluation of the program.
Because of the resources required and the realistic barriers for ROI implementation, ROI analysis should be used only for those programs that are very expensive, linked to strategic objectives, important for solving organizational problems, and highly visible. A comprehensive employee engagement program usually meets most of these criteria.
References:
1. Phillips, Patti P., Jack J. Phillips, and Rebecca Ray. Measuring the Success of Employee Engagement. Alexandria, VA: ASTD Press, 2016.
2. Phillips, Patti P., and Jack J. Phillips. โMeasuring ROI for Engagement Linked to Retention Improvement: Southeast Corridor Bank,โ in Measuring the Success of Employee Engagement by Phillips, Patti P., Jack J. Phillips, and Rebecca Ray. Alexandria, VA: ASTD Press, 2016.
