Your cart is currently empty!
“Now that we’ve launched this incredible program, let’s figure out how to measure success!” No, no, no. While a late measurement discussion is better than no measurement discussion, the missed opportunity here is exponential. If you’re figuring out how to measure success post-implementation, then that means you missed the opportunity to fully level set on program sponsor/key stakeholder expectations for the programโs chain of impact and quantification of anticipated outcomes.
How to Avoid it: There are very few resources that I love as much as ROI Institute’s V model, which you can access for free by registering here. The purpose of this tool is to ensure that the program sponsor and other key players are all on the same page. It begins on the left-hand side of the V, where program needs are defined at each of the five levels of measurement.
I have found that we often define what we want the learner to learn, but we don’t do as great a job setting specific expectations along the rest of the chain of impact. What exactly should learners do with the information they’ve learned? What does success look like? What observable behaviors are expected? Then, when these behaviors are present, what is the business impact we expect as a result? Itโs important to be specific here. Just calling out that we want a particular KPI or two to improve isn’t enough. What is the target? How much improvement is expected? There is nothing worse than celebrating success only to find out your program sponsor was expecting a bigger impact. And I’m sure we’ve all heard of SMART goals at this point, so remember the “T” is for time bound, which also applies to program design. We want to discuss and align on when it is reasonable to expect the impact to manifest in our identified KPIs.
Lastly, it’s important to discuss ROI expectations and align on a target there, too. Believe it or not, I’ve been responsible for evaluating programs where the executive sponsor was not expecting a measurable ROI. All program benefits were anticipated as being intangible. While measurable, they could not be converted to reliable monetary values with reasonable time/effort. This is not an issue as long as all stakeholders are aligned with stopping at Level 4. Can you imagine not getting this alignment prior to program design and implementation? Trust me, I’ve been there, and it’s a place you don’t want to be.
This pitfall refers to something I see so frequently across many different initiatives- failure to disclose objectives beyond Level 1 to program participants. If you haven’t told participants what you expect them to learn, do, and impact, then you’re not properly setting them up for success, and you’re not going to get good Level 1 Reaction data.
For example, let’s say my husband comes home and tells me how to disassemble a transmission (he is a mechanic), and then he asks me, did I do a good job telling you how to disassemble a transmission? I would say yes, honey! Absolutely. You did a wonderful job; it was very interesting. Now, let’s say the following day, my husband tells me he needs me to disassemble a transmission by myself in one hour. Full disclosure: I have no idea if that’s reasonable because we haven’t ever discussed disassembling a transmission, but for argument’s sake, let’s pretend it’s possible to do, but only if you know what you’re doing. So now that my husband makes this request, I’m like, hold on. I am NOT equipped to do this on my own. You’re crazy. Telling me how to do it yesterday was not enough. If I had known how you wanted me to apply it, I would have taken notes, and when you asked me for feedback, I would have told you there was no way walking me through it once was going to equip me for success. Do you see where I’m going here?
It’s not enough to just tell or teach me about something. You need to be very specific about the expectations for application (frequency, duration, time period, etc.) and impact (what measurable outcome is expected to improve and exactly how much improvement is expected). I always see learning objectives, but I seldom see specific application objectives and their subsequent impact objectives.
How to Avoid it: Here, we are back at the V model. Before you can tell your participants about the program objectives, you’ve got to create them. How do you want people to react? What planned action are you hoping for? (Level 1) What do people need to learn? (Level 2) How should they apply it? (Level 3) What impact is expected? (Level 4) and while it’s good to have an ROI objective, that’s not one you necessarily need to go over with participants unless you’d like participants to understand the program cost and feel a sense of accountability for a positive ROI. But you’ve got to make sure the conditions are right for that to work- which is a whole other article in and of itself.
I’d also like to point out that if you need examples of what good objectives look like for Application (Level 3) and Impact (Level 4) then look no further than the ROI Methodology Application Guide, which you can access for free here.ย
“It’s too hard.” Is it really, though? I think I’ve heard this excuse used for at least 15, maybe even closer to 20 programs. The reality is that the ROI Methodology is simple to apply, understand, and communicate. Now, I will acknowledge that not all programs are candidates for ROI but that doesn’t mean you can’t still measure Levels 1 through 4. To be fair, there are situations where it was determined that the level of effort to obtain and analyze impact data (Level 4) was greater than the benefit of collecting it. However, there’s a big difference between writing off the process as being too hard vs following the process and making the judgment to stop below Level 5.
How to Avoid it: This one can be tricky. You’ve got to root cause it. Why are you getting the “it’s too hard” pushback? Is it because your sponsor and/or decision maker doesn’t understand there is a methodology available, or are they just not familiar enough with the methodology to consider it credible? Unfortunately, I’ve also been in a situation where the program sponsor was concerned the study might yield unfavorable results, and I was blatantly instructed NOT to perform an evaluation on the program in question.
If you think the program sponsor (or other key stakeholder with decision-making power) may be resistant because they’re afraid of a negative ROI, that’s an opportunity to articulate the benefits of the methodology beyond the ROI in and of itself. The collection of enablers, barriers, and even specific behavioral and knowledge data can identify points of failure and enable correction. Avoiding the results won’t make the results any better. Of course, I would recommend having this conversation 1:1.
“What we’ve got here is failure to communicate” – when you read this in your head, did it sound like the clip from Cool Hand Luke? When it comes to stakeholders, it should never be “Surprise! We measured this!”
How to Avoid it: For one thing, awareness of the measurement plan may drive accountability. When people know that you’ll be following up continuously for a period of time and that the results will be communicated to the program sponsor(s) and leadership, they’ll be more likely to continue application (participants) and/or reinforcement (leaders of participants). Now, this communication of results should never be a “gotcha!” moment that people resent you for. Be sure to provide helpful reminders to (leaders of) participants and be fully transparent about what will be reported and when.
I do have a quick cautionary tale here for you, though. Years ago, I was working as a representative in a sales/customer service call center. Customer satisfaction was an area of opportunity. After some research, it was determined that transfers were a big pain point (no kidding). So the directive came down from above that we need to reduce customer transfers. Well, just like a game of telephone, things get lost as they travel from person to person, right? By the time the message got to the front line, the true purpose (which was customer experience, of course) had been lost. The message had become, “reduce transfers.” Somebody had the “brilliant” idea to provide all of us representatives direct numbers to each department, and we were instructed to just give the customer the phone number and tell them to call themselves instead of transferring. Wallah! Transfer rates plummeted to record lows! Mission accomplished! Hopefully, you’re picking up on my sarcasm here.
My point is that when you focus people on the metric and lose the purpose, you risk a check-the-box, meet the metric at all costs situation. Be mindful of this when you’re collecting data and communicating about your data collection. Choose your words carefully and be mindful to drive the behavior, not the metric. Also, I can’t stress this enough: recognition, recognition, recognition, and a few sprinkles of shared success will go a long way. It’s important to close the loop with participants and show them their application leads to impact. Celebrate the wins and thank people for their contributions.
Stakeholder involvement is so important. Nothing can be successful if it is created or measured in a vacuum. Contribution drives ownership, and ownership drives success. Failure to get the right stakeholder involvement in your program design and measurement will impact engagement, response rates, and, therefore, the quality of data.
How to Avoid it: Some of my most successful programs have leveraged a team of champions. This is different than the core group that attends every project call. This group is representative of the participants- they will be participants themselves- and they provide feedback at several predetermined points. I publicly recognize this group because they deserve it, but also because they will then own it. It becomes their work, too. It’s like having a “friendly” in the rooms you can’t be in. They will encourage, promote, embrace, own, and champion the program and even the data collection. This is not the right thing for every program, but for those programs where it is right, it makes all the difference.
The other thing I would suggest is pulling in the owners of your data source as early as possible. For example, let’s say I roll out a new process where customer service representatives will be sending an email to customers to confirm their transaction. Part of my Level 3 (Application) measurement plan would likely include data on the frequency that representatives are capturing a valid email address and sending this email to their customers. This data is likely available- I would not use a survey/questionnaire to ask the representatives about the frequency of successful email address collection and how often they’re sending the actual email. I do not recommend waiting until you need the data to talk to the data owner.
Connect with them upfront as you are designing your data collection plan. Build the relationship and make sure you have a clear understanding of what can be provided and at what level of granularity. For example, if you’re using a control group for your program, can you isolate the data to view control group vs trained, or can they only provide high-level data by region?
The point of measurement is not to say look at meโmy initiative was successful! Though that is a benefit, the point of measurement is to learn about your program. What worked? What didn’t? What factors drove success, and what got in the way? What did we learn about drivers and detractors of success that we can apply to future initiatives? Was this a worthwhile investment? Why or why not?
Part of what you want to learn should be about contributing factors across the board, not just the ones from your program. Success can often be the perfect storm of conditions, and re-creating just one condition won’t necessarily yield future success. Just because we observe a positive change in a target KPI does not mean that positive change can be attributed to our program. What’s more, you will quickly damage relationships if you’re taking credit for what others perceive to be their work.
How to Avoid it: I don’t want to turn this into a full-blown lesson on isolation (though if you’re interested in that, you can sign up for ROI Institute Boot Camp or Certification!), so I’ll just provide one example here.
I had a program where we saw significant improvement in the target KPI, and neither control group nor trend line analysis was an option for this particular project. First, I used my team of champions to identify everything that was going on in the business at the time that could have potentially impacted the metric. In addition to the training program, there were some recognition incentives happening, leaders were also holding coaching sessions with participants where they reinforced the training, and in this case, the natural learning curve meant that over time, participants would have improved somewhat regardless of receiving additional training.
Next, using what was deemed to be the most reliable source, participant direct managers, we asked them to estimate the % improvement that could be attributed to each of these factors. There was also an “other” option where, if chosen, respondents would state specifically what factor beyond those listed had influenced the program results. Now for those more familiar with this process, you know we also adjusted for confidence, etc., but as I said, this is not a lesson on isolation. Instead what I’d like to point out is that we were able to provide feedback that included how impactful these other factors were in addition to the program, and we only took credit for the % improvement leaders directly attributed to the training.
This was valuable information, and ultimately, senior leadership was thrilled with the added insight and felt that the training probably did even more than we were taking credit for. This information also served to provide insight into the combination of factors that led to success, and those involved in the recognition incentive program felt acknowledged, too.
I hope you found this article helpful. Hungry for more? Please join me at our upcoming ROI Boot Camp or enroll in ROI Certification to earn your CRP and take your program measurement strategy to the next level!
This article was originally published on LinkedIn, January 8, 2024.
About the Author
Katharine Aldana, a two-time “ROI Practitioner of the Year” award winner, is known for her creative program improvement strategies. Katharine spent seven years designing learning strategies and performing program evaluations for a Fortune Top 20 organization before transitioning to her current role as Senior Manager-Business Strategy within the same organization. Katharine’s enthusiasm for program effectiveness and real-world experience make her a dynamic facilitator and highly sought-after ROI implementation resource.
Jennifer Wingate is the Communication and Research Manager at ROI Institute. Jennifer is skilled in managing all marketing and public relations campaigns and brand messaging. Before joining ROI Institute, she was a marketing professional working for companies like Encompass Health and Bradley Arant Boult Cummings for over fifteen years. She assisted Encompass Health with the national re-brand in 2019.
Jennifer is a University of Alabama in Birmingham graduate with a degree in communications management and mass communications. She enjoys spending time with her husband and puppies and hunting for antiques in her spare time.
Jennifer can be reached at jennifer@roiinstitute.net
Klaas Toes MSc, is President Global Business Development and Director, ROI Institute-Europe, and boasts over 30 years of hands-on experience in shaping human capital strategies, leadership, change, and sales for diverse businesses and governmental agencies. A prominent learning economist, Klaas emphasizes the importance of concrete data when determining business impact and ROI.
Klaas founded ROI Institute-Europe in 1998 enabling him to support the measurement and evaluation of tangible and financial values of learning for individuals and organizations. This venture was instrumental in laying the groundwork for people analytics in Europe.
By 2012, Klaas published De Waarde van Leren. As a well-respected voice in his domain, Klaas often shares insights via lectures and keynotes on various subjects including people analytics, business alignment, and ROI. In 2024 Klaas was co author of The Return On Investment for Training and Performance Improvement programs.
In 2016, Klaas’s expertise earned him a pivotal role as the European Council Director for People Analytics at The Conference Board, a position he held till 2021. He’s not just a thought leader but a successful entrepreneur in the fields of HR and Learning & Development, with his ventures impacting organizations on a global scale.
Andy Vance is Director of Client Relationships and IT Manager for ROI Institute. In his role asย Director of Client Relationships, Andy generates sales, implements marketing strategies, and maintains connections with clients and participants. In his IT Manager role, Andy maintains ROI Instituteโs technology infrastructure, produces virtual events, designing and updating webpages, and ensures online security.
Andy is a graduate of Phoenix Online University and has a Practical Project Manager certification. Andy also is a Glenwood Junior Board member a non profit dedicated to the support of adults and children with autism.
In his spare time, he enjoys hiking, camping, swimming, and building models.
Andy can be reached at andy@roiinstitute.net.
Hope Nicholas is the Director of Publications for ROI Institute.ย In this role she coordinates all publications, including books, articles, and case studies.ย Hope has significant experience with transcription, editing, proofreading, and project planning.ย Additionally, she contributes to multiple research and business development projects by facilitating communications and developing a wide variety of documents.
Hope is a graduate of the University of Akron and has over 15 years of experience working the field of healthcare administration and documentation. In her spare time, she enjoys painting, golfing, and playing tennis with friends and family.
Hope can be reached at hope@roiinstitute.net.
Tim Brock,ย Ph.D., a senior consultant at ROI Institute, facilitates ROI Certification and ROI workshops and provides ROI consulting services. Tim leads ROI Instituteโs weekly live virtual group coaching sessions that support individuals working toward earning the Certified ROI Professionalยฎ (CRP) credential. He is also an invited presenter at multiple international conferences and has also authored chapters on the ROI Methodology in several books.
Tim is a retired Air Force field grade officer, having served in both the enlisted and officer ranks, serving as a nuclear weapons maintenance crew chief and a missile combat crew commander for both the Titan II and Minuteman II intercontinental ballistic missile weapon systems. During his later career at Lockheed Martin, Tim led their science of learning and performance improvement team as a human performance system engineer and architect. He has experience in the education profession as a K-12 private school principal, a college director of education, and associate professor. Tim earned his Ph.D. from Capella University in education with a specialization in training and performance improvement.
Tim enjoys spending time with his three grown children and three young grandchildren. He and his wife enjoy taking extended cruises to relax, read, and see the world.
Caroline Hubble is Vice President of Consulting Services with ROI Institute. She manages the implementation of complex program evaluation projects, including conducting research and evaluation studies in various areas such as leadership development, process improvement strategies, and program implementation. In previous roles, she managed evaluation, analytics, reporting, and operations for business line and enterprise-wide HR departments. Her operational, project, and relationship management expertise is noted for significantly contributing to improved business practices.
Caroline holds a BA in Psychology from Rollins College and received her Masters of Science in Organizational Development in May 2013. Her publications include: ASTD Handbook of Measuring and Evaluating Training, Chapter 4: Using Surveys and Questionnaires. ASTD 2010; Measuring ROI in Learning & Development: Case Studies from Global Organizations, Chapter 4 (co-authored). ASTD 2012โ Offering Money for School Pays Off, (co-authored) Chief Learning Officer Magazine, June 2012.
When not focused on evaluation efforts, she enjoys spending time on her small farm in the mountains of North Carolina, raising Savannah and Egyptian Mau cats, following NASCAR, and capturing memories through photography.
Caroline can be reached at caroline@roiinstitute.net.
Melissa Brown is the Director of Partnerships at ROI Institute. She manages ROI Instituteโs network of more than 70 international partners and associates, as they implement the ROI Methodologyยฎ globally. Melissa also oversees marketing by coordinating and designing marketing campaigns, the layout and design of course materials, research projects, promotional collateral, and other communication resources. Melissa works alongside clients, partners, and associates to ensure quality results. Melissa is a Certified Digital Marketing Professional and American Marketing Association Professional Certified Marketer in Digital Marketing, and she also holds a Qualtrics Platform Essentials Certification.
Melissa earned a bachelorโs degree in Communication and Information Sciences from The University of Alabama and a masterโs degree in Communication from Walden University. She also holds a teaching license in secondary language arts. Melissa and her family enjoy traveling, watching and playing sports, and spending time outdoors.
Melissa can be reached at melissa@roiinstitute.net.
Kylie McLeod manages research, assessment, and communication at ROI Institute. She develops data collection instruments, collects, and analyzes data, writes final evaluation reports, manages research projects, coordinates the communication process, and directs the ROI Institute Awards Program. She also edits ROI Institute books and articles, develops promotional and marketing materials, and assists with the proposal creation process. She served as editor, production manager, and coordinator of the book, Value for Money: How to Show the Value for Money for All Types of Projects and Programs In Governments, Nongovernmental Organizations, Nonprofits, and Businesses (Wiley, 2020).
Kylie graduated with a Bachelor’s in Communication and Information Sciences from The University of Alabama in 2015 and a Master of Arts in Journalism from the same institution in 2017. Kylie holds CoreXM and CoreXM Expert certifications from Qualtrics and earned the Certified ROI Practitioner credential in November 2019.
In her free time, Kylie cheers on the Alabama Crimson Tide, enjoys the (mostly) sunny weather in Northwest Florida, and dotes on her dog, Penny.
Kylie can be reached at kylie@roiinstitute.net.
Kylie McLeod manages research, assessment, and communication at ROI Institute. She manages proposals, develops data collection instruments, collects, and analyzes data, writes final evaluation reports, and manages research projects. She has previously edited ROI Institute books and articles and developed promotional and marketing materials. She served as editor, production manager, and coordinator of the book, Value for Money: How to Show the Value for Money for All Types of Projects and Programs In Governments, Nongovernmental Organizations, Nonprofits, and Businesses (Wiley, 2020).
Kylie graduated with a Bachelorโs in Communication and Information Sciences from The University of Alabama in 2015 and a Master of Arts in Journalism from the same institution in 2017. Kylie holds CoreXM and CoreXM Expert certifications from Qualtrics and earned the Certified ROI Professional credential in November 2019.
In her free time, Kylie cheers on the Alabama Crimson Tide, enjoys the (mostly) sunny weather in Northwest Florida, and dotes on her dog, Penny.
Kylie can be reached atย kylie@roiinstitute.net.
Patti P. Phillips, Ph.D., is co-founder and CEO of ROI Institute. She is a driving force in the
global adoption of the ROI Methodologyยฎ and measurement and evaluation to influence organizational change. Her work as an educator, researcher, consultant, and coach supports leaders as they build the capacity to create and deliver value from investments in all types of endeavors. Her work spans private, public, nonprofit, and nongovernmental organizations.
Patti serves as a member of the Board of Trustees of the United Nations Institute for Training and Research and the UN System Staff College faculty in Turin, Italy. Additionally, she is chair of the Institute for Corporate Productivity People Analytics Board; Principal Research Fellow for The Conference Board; board member of the International Federation for Training and Development Organizations; and board chair for the Center for Talent Reporting. In 2015, she was honored by the Association for Talent Development as a Certified Professional in Talent Development Fellow for her contribution to the professionalism of the talent development field.
Patti contributes to various journals and is the author of books on measurement, evaluation, and ROI. CNBC, Euronews, and other news outlets feature her work.
Patti can be reached atย patti@roiinstitute.net
Ann Akins has been associated with ROI Institute for over 23 years.ย While primarily providing financial support for the company, in 2019, she also became the President of ROI expanding her role into the operational areas of the company.ย Prior to joining ROI, Ann was a regional controller for Waste Management with over $250 million in annual revenue.ย She has also led a customer experience center for FIS (a financial services company) with over 1,200 employees.ย This previous experience has helped Ann provide significant financial and operational insights for ROI operations.
Ann has a BS in Accounting from the University of West Alabama and a Masterโs in Business Administration from Mississippi State University.ย An avid University of Alabama sports fan, she enjoys attending UA football games and spending time with her children.
Ann can be reached at ann@roiinstitute.net.
Evan Seidner is a Senior Analyst for ROI Institute. In his role as an analyst, Evan does mixed-methods data analysis for clients. Evan also supports other team members in the evaluation process and implementation processes. In previous roles, he has worked with environmental NGOs in China as well as working for an agribusiness in Tokyo, Japan.
Evan holds a BA in Global Studies with minors in Political Science and East Asian Languages and Cultures from the University of Illinois at Urbana-Champaign and received his MA in International Relations from the American University in Washington, D.C.
In his free time, Evan enjoys learning photography and videography. He is a Japanese film buff, which he watches to help maintain his language skills.
Evan can be reached at evan@roiinstitute.net
Jack J. Phillips, Ph.D., chairman of ROI Institute, is a world-renowned expert on accountability, measurement, and evaluation. Jack provides consulting services for Fortune 500 companies and major global organizations, and regularly consults with clients in manufacturing, service, and government organizations in 70 countries. The author or editor of more than 100 books, he conducts workshops and presents at conferences throughout the world and has received several awards and honors for his work.
Jack has enjoyed almost 30 years of corporate experience in the aerospace, textile, metals, construction materials, and banking industries. He has served as training and development manager at two Fortune 500 firms, as senior human resource officer at two firms, and as president of a regional bank. Also, he served as management professor at a major state university.
Jack has undergraduate degrees in electrical engineering, physics, and mathematics; a masterโs degree in Decision Sciences from Georgia State University; and a Ph.D. in Human Resource Management from the University of Alabama. He has served on the boards of several private businesses, nonprofits, and associations, including the American Society for Training and Development, the National Management Association, and the International Society for Performance Improvement, where he served as president (2012-2013).
Jack can be reached at jack@roiinstitute.net.
Patti P. Phillips, Ph.D., is co-founder and CEO of ROI Institute. She is a driving force in the global adoption of the ROI Methodology and measurement and evaluation to influence organizational change. Her work as an educator, researcher, consultant, and coach supports leaders as they build the capacity to create and deliver value from investments in all types of endeavors. Her work spans private, public, nonprofit, and nongovernmental organizations.
Patti serves as a member of the Board of Trustees of the United Nations Institute for Training and Research (UNITAR) and the UN System Staff College faculty in Turin, Italy. Additionally, she is chair of the Institute for Corporate Productivity (i4cp) People Analytics Board; Principal Research Fellow for The Conference Board; board member of the International Federation for Training and Development Organizations (IFTDO); and board chair for the Center for Talent Reporting (CTR). In 2015, she was honored by the Association for Talent Development as a Certified Professional in Talent Development (CPTD) Fellow for her contribution to the professionalism of the talent development field.
Patti contributes to various journals and is the author of books on measurement, evaluation, and ROI. CNBC, Euronews, and other news outlets feature her work.
Patti can be reached at patti@roiinstitute.net
Hope Nicholas the Director of Publications for the ROI Institute.ย In this role she coordinates all publications, including books, articles, and case studies.ย Hope has significant experience with transcription, editing, proofreading, and project planning.ย Additionally, she contributes to multiple research and business development projects by facilitating communications and developing a wide variety of documents.
Hope is a graduate of the University of Akron and has over 15 years of experience working the field of healthcare administration and documentation. In her spare time, she enjoys painting, golfing, and playing tennis with friends and family.
Hope can be reached atย hope@roiinstitute.net.
Caroline Hubble, M.S.O.D., is Vice President of Consulting Services with ROI Institute, Caroline manages the implementation of complex program evaluation projects, including conducting research and evaluation studies in various areas such as leadership development, process improvement strategies, and program implementation. In previous roles, she managed evaluation, analytics, reporting, and operations for business line and enterprise-wide HR departments. Her operational, project, and relationship management expertise is noted for significantly contributing to improved business practices.
Caroline holds a BA in Psychology from Rollins College and received her Masters of Science in Organizational Development in May 2013. Her publications include: ASTD Handbook of Measuring and Evaluating Training, Chapter 4: Using Surveys and Questionnaires. ASTD 2010; Measuring ROI in Learning & Development: Case Studies from Global Organizations, Chapter 4 (co-authored). ASTD 2012โ Offering Money for School Pays Off, (co-authored) Chief Learning Officer Magazine, June 2012.
When not focused on evaluation efforts, she enjoys spending time on her small farm in the mountains of North Carolina, raising Savannah and Egyptian Mau cats, following NASCAR, and capturing memories through photography.
Caroline can be reached atย caroline@roiinstitute.net.
Jack J. Phillips, Ph.D., chairman of ROI Institute, is a world-renowned expert on accountability, measurement, and evaluation. Jack provides consulting services for Fortune 500 companies and major global organizations, and regularly consults with clients in manufacturing, service, and government organizations in 70 countries. The author or editor of more than 100 books, he conducts workshops and presents at conferences throughout the world and has received several awards and honors for his work.
Jack has enjoyed almost 30 years of corporate experience in the aerospace, textile, metals, construction materials, and banking industries. He has served as training and development manager at two Fortune 500 firms, as senior human resource officer at two firms, and as president of a regional bank. Also, he served as management professor at a major state university.
Jack has undergraduate degrees in electrical engineering, physics, and mathematics; a masterโs degree in Decision Sciences from Georgia State University; and a Ph.D. in Human Resource Management from the University of Alabama. He has served on the boards of several private businesses, nonprofits, and associations, including the American Society for Training and Development, the National Management Association, and the International Society for Performance Improvement, where he served as president (2012-2013).
Jack can be reached atย jack@roiinstitute.net.
Example
MYTH: The monetary value for a measure is difficult to locate or calculate.
REALITY: Most measures that matter have been converted to money.
The Kansas City SWAT team implemented the Outward Mindset program to reduce citizen complaints about the police, a meaยญsure of the quality of their work. The team leader, Chip Huth, and the team knew they could avoid the complaints by being nice guys, but they still had their jobs to do. Measures of productivity were equally important, includยญing confiscating drugs, cash, and guns; serving warrants; and making arrests. As a result of Outward Mindset, they were able to improve proยญductivity and quality. Reducing complaints was an important quality meaยญsure of success. However, according to Chip, โThe Outward Mindset approach did not merely help us reduce complaints, but also helped us account for an unquantifiable reduction in lawsuits, injuries, etc., and an increase in public trust, collaboration, and drug and gun seizures.โ
What else cause the complaints to decrease? The number of missions did not change, and the types of missions didnโt change. There were no policy changes and no high-profile cases in the news. In essence, the team could not think of any other factors and believed that their new Outward Mindset approach had made the difference. According to Chip, the complaints disappeared because of the behaviors from the Outward Mindยญ set, โThe way we responded to the reality of others while executing our work, including some changes in procedures that were born out of our commitment to seeing others, made the complaints disappear.โ
When comparing the monetary benefits of reducing complaints for one year to the cost of preparing this team to use the Outward Mindset, the ROI is 5,724 percent. This is an almost unbelievable ROI, but the process and the data prove it is credible.
Story from Show the Value of What You Do, Chapter 6, Page 97
Example
MYTH: The success of one project rarely influences investing in another project.
REALITY: The best results to continue to invest in are previous successes.
Jessica Kriegel, an organization and talent development consultant at Oracle with expertise in strategic planning, talent management, and leadership development, was particularly focused on intergeneraยญtional understanding. More specifically, she realized that generational labels do not work. The labels arenโt bad. Itโs the associations that society puts on those labels that are bad.
Although labels abound with each generational era, millennials (those born between 1980 and 2000) are the most misunderstood genยญeration. These labels keep individuals, managers, and even organizations from being as successful as they should be. Jessicaโs work at Oracle proยญ vided her with an opportunity to tackle this challenge directly. One projยญect clearly focused on this challenge, and she needed to show its value.1 A product ยญdevelopment team member reached out to the person in charge of the Oracle College Hire Program about their problem with the millennials. The team member suggested that the millennials didnโt unยญderstand the corporate culture, and as a result, there were behavioral and communication issues. This suggestion included a request for a training program to help the millennials acclimate to the company.
Jessica took this project on. She wanted to understand the issue and ensure that the correct program was implemented. She did not want to take on the project on the face value that millennials were the problem, and if we fix them, everything is okay.
Each year, this program develops approximately 100 to 200 new college hires, making this an important program. She examined some of the data that were surrounding the situation. A lack of productivity was reported, along with the excessive turnover of the millennials. Some were leaving soon after employment, which was very expenยญsive. Consequently, the business outcomes of interest were imยญ proved productivity and reduced turnover.
To understand the right solution, Jessica conducted a detailed surยญ vey with the managers and the employees. The surveys indicated that:
Next, Jessica conducted focus groups to examine the problem in more detail, and the focus groups identified areas where improveยญment was needed. The managers:
Meanwhile, the millennials felt they:
With these issues identified, the solution was designed to work with both groups to ensure that they worked productively and effiยญciently together, addressing their individual needs in two different programs (one for the managers and one for the employees). She set out to show the value of her program.
As Jessica conducted the programs, she captured reaction data. The employees felt the program was valuable, and it helped them understand more about their roles in the organization. She measured learning from both groups and found they were learning what they needed to do to be more successful. She measured application to see the extent to which they were working together because of the training. The reports were positive that they were working collaboraยญtively. With the increased collaboration came improvements in retenยญtion and productivity. With these improvements, she took a step to sort out the effects of the program from other influences, converted these measures to money, compared program benefits to the cost of the program, and calculated that the return on investment was 695 percent. The executives were impressed, and she communiยญcated the results to other groups and included a copy of her bestsellยญing book, Unfairly Labeled.
Leveraging the results of Jessicaโs project has made a differenceโ not only for the organization in which she was working at the time, but also for the program participants and the individuals she now reaches through her writing and speaking.
Story from Show the Value of What You Do, Chapter 7, Page 117
Example
MYTH: Data collection to show the success of a project is almost impossible.
REALITY: With planning and effort, data collection becomes feasible and successful.
Zoe Moore has been very determined. She helps organizations deยญvelop their Diversity, Equity & Inclusion (DE&I) strategies by ensuring there is diverse representation in leadership and recurring profesยญsional development focused on DE&I that improves workplace culture and supplier diversity programs to increase opportunities for underยญ resourced businesses. She has garnished attention around the subยญ jects, but still, many leaders ask the question, โSo what? What value do diversity, equity, and inclusion bring?โ
Of course, supplier diversity is important, but that is just the startยญing point. Equity and inclusivity are imperative across the global business of hospitality and are where the real value of Zoeโs work reยญ sides. But how can she demonstrate this value? She needs to collect data about:
This creates a new data collection stream for Zoe. While sheโs worked hard to improve awareness around supplier diversity through representation metrics, the challenge is to make sure that the teams integrate DE&I successfully, and she must collect data along the same five levels of success that form the value chain, with supplier diversity being the beginning point. This situation highlights that the five levels of success, first introduced in Chapter 1, can be examยญined at different times in projects and often from different perspecยญtives. This requires data collection along the value chain to make it work.
Story from Show the Value of What You Do, Chapter 5, Page 83
Example
MYTH: Objectives add little value to the outcomes.
REALITY: Objectives are powerful drivers of success.
Martin Burt, Ph.D., has a lifelong quest to understand and abolish poverty. His inspirational experience in public service includes servยญing as chief of staff for the president of Paraguay, the mayor of Asunยญciรณn (the capital city of Paraguay), and the vice minister of commerce. Martin is the founder and CEO of Fundaciรณn Paraguaya, a nonprofit devoted to promoting social entrepreneurship and economic selfยญ reliance to eliminate poverty worldwide.
Martin has challenged many assumptions about poverty by asking, โWhat if everything we knew about poverty was wrong? What if the legions of policy makers, social scientists, economists, aid workers, charities, and NGOs marching across the globe have been using the wrong strategy and tactics to wage the wrong war against poverty?โ
In his book, Who Owns Poverty?, Martin lays out the frameยญ work necessary to eliminate poverty. The answer to his question, โWho owns poverty?โ is clearโthe people in poverty. They donโt want to be in poverty. But people in poverty are not just persons with lowยญ income labels. They are individuals with joy, generosity, and creativity. They have problemยญ solving and entrepreneurial spirits, and they want to rise out of poverty. They just need help and support to get there, and they will get there if we give them an avenue to do so.
Martin created the Poverty Stoplight program that shows families what they must do to come out of poverty. Itโs not just increasing inยญ come, but itโs tackling a host of issues that altogether contribute to poverty. He developed 50 indicators of poverty, as shown in Table 9. Martin created the stoplight program with clear measures and three phases: red, yellow, and greenโred (a worst ยญcase scenario), yellow (making progress), and green (out of poverty). Along with each indicator, for each phase, the Poverty Stoplight team developed speยญcific measures of success so that all stakeholders, including the famiยญlies in poverty, can see where they are going, and the progress being made. Progress follows the five ยญlevel framework of reaction, learning, application, impact, and ROI.
Martin needs to show the value of the Poverty Stoplight program. He needs to push its evaluation to the ROI level for governments, as they calculate benefitยญ-cost analysis for helping to eliminate poverty. He also needs to calculate the ROI for the companies supporting the program, showing them that this is a good investment for the company and the community. The major foundations involved in the NGOs and the charities donโt need to see ROI, but they all need to see the impact. The impact is getting families out of poverty and reaching the green light on the Poverty Stoplight. Martin needs the Show the Value Process to show the value of what they do.
Story from Show the Value of What You Do, Chapter 4, Page 61
Example
MYTH: The solution to a problem is almost always obvious.
REALITY: The right solution is rarely obvious.
Ginger Luttrell spent 10 years as a software engineer, including conยญ figuration and programming at one of the worldโs largest business software companies. The intent of new systems implementations is to improve output, quality, cost, and time. Over the years of working with these systems implementations, she noticed a problem. When an implementation project would go live, endยญusers, those who worked in the business functional areas or departments, would ask quesยญtions such as:
Interestingly, end-ยญusers received comprehensive training on these systems during projects. Despite this fact, Ginger noticed that they exยญperienced problems and frustrations on the job. As a result, productivยญity, quality, and time savings sufferer rather than improved. The types of problems Ginger was addressing were universal. Additionally, some end-ยญusers would become so frustrated they would leave the company altogether. Ginger also saw that there was no channel for the voice of the end-users to get to the senior executives to address their issues and concerns.
Ginger began to realize a possible solution by addressing the preยญcise business needs identified (productivity, quality, time, cost, and retention). The endยญ-users needed a person who would support them on a routine basis. This person would serve as a mentor, coach, trainer, counselor, and trusted advisor to help them through complex impleยญmentation issues. This person is called a super user. At the same time, super users would help develop the end-ยญusers to become more valuable business team membersโmembers with a broader vision and more complex skill sets, who could offer greater support in their organization.
Ginger realized that the cost of providing super users was much less than the savings that would be generated to make the software system more effective for the organization. She tested her theory in different organizations and was convinced that she had the right solution for the perplexing problem of largeยญscale systems implementation. Eventuยญally, Ginger founded the Super User Network and now spends much of her time preparing and developing super users to perform this essenยญtial role in business departments worldwide.
Story from Show the Value of What You Do, Chapter 3, Page 45
Example
MYTH: Itโs almost impossible to connect most projects to business needs.
REALITY: Essentially, every project can be connected to a business need.
Chip Huth, the new leader of a SWAT team in the Kansas City, Missouri, Police Department, knew what he had to do. Citizen complaints against the team were excessive, averaging about 30โ40 per year. They were also expensive, costing the department an average of $70,000 per complaint just for the investigations. The deputy chief (DC) who appointed Chip was clearโChip needed to โclean upโ the mess. The DC was concerned about optics from the complaints and the many irritations they were causing.
Expectations were vague, and the accepted remedy was potenยญtially a โhouse cleaningโ and subsequent creation of a new team. Chip knew what he was facing. He accepted the assignment because he loved working in teams. Initially, Chip didnโt believe the team could be โfixed.โ Admittedly, Chip was completely self-centered in his thinking and simply loved the thrill of the work. He reassured the DC that he could whip the team into shape (not believing anyone could do so) and initially sought token improvements by addressing problem behaviors.
Unexpectedly, through a series of personal and professional encounยญters, Chip was exposed to a personal improvement process called Outยญ ward Mindset from the Arbinger Institute. Arbingerโs research indicates that people operate at any given time from one of two mindsets: an inยญ ward mindset or an outward mindset. When operating with an inward mindset, you focus only on your own goals and objectives without conยญsidering your impact on others. With an outward mindset, however, you see others as people who matter as much as you do. You consider their needs, challenges, and objectives. And you focus on collective results.
As Chip began applying the Outward Mindset framework to each area of his life, including his leadership responsibilities with the team, it seemed logical that this could impact the team. Although Chip was not certain that the Outward Mindset would deliver fewer complaints or build a better relationship with the community, he was willing to give it a try.
He introduced the team to this process. When they were exposed to it, they responded positively. The team asked Chip to lead them in applying the Outward Mindset skills and framework, in roleยญ modeling the behaviors, and showing them that it works. That was all Chip needed. He had the business measure, citizen complaints at roughly $70,000 each, the right solution, Outward Mindset, and the right people at the right time. He was ready to go.
Story from Show the Value of What You Do, Chapter 2, Page 29
How to Develop Turnover Costs
The first step in monitoring turnover costs is to define and discuss several issues about a cost control system. The key issues are presented here.
Some Costs Are Difficult to Determine
The huge difference that often exists between the cost estimates from HR and the actual turnover cost lies in the indirect or hidden costs category. Where direct costs are usually in the cost accounting system, the hidden costs are almost never considered. However, they can be developed using assumptions and estimation processes.
Fully-Loaded Costs
Using the approach to capturing the fully-loaded costs of turnover, each cost is identified and put into a specific category. Where an estimate is required, the entry is adjusted later. If possible, someone from finance and accounting should review and approve the data. The process should be able to withstand even the closest scrutiny, so organizations must ensure that all costs are included.
Reporting Total Cost
Costs are typically expressed as a percentage of the wages and salaries of the employees in a particular job group. This figure is usually determined after a detailed cost study is conducted. The percentage can be fixed for a group (for example, the sales force) or a specific job (for example, client relationship manager [CRM]).
For example, if a cost study in another industry has concluded that the cost to replace a CRM averages 150 percent of pay, this amount can be a beginning point. If there is some concern about the cost being too high, perhaps a lower number would be appropriate, such as 120 percent or 100 percent. After a figure has been determined, the turnover cost is then reported on statements along with the actual costs.
Morteza Zohrabi, MD, MBB, PMP, CRP, is the lead director in the implementation of countless mega-projects in both public and private sectors for large hospitals, government offices, manufacturers, and universities, to name a few. He is also a Certified Return on Investment Professional, publishing numerous articles and presentations on guaranteed value for money.
With over 20 years of expertise and hundreds of millions of dollars in savings, Morteza has done it all. He has delivered innovative solutions to old-time problems, with a track record of outstanding results, enhancing throughput, capacity, staff/client satisfaction, and shrinking turnaround times.
Through a combination of conventional and modern management techniques, Morteza and his team have revolutionized the management consulting industry by providing state-of-the-art business transformations and distinctively conforming as growth partners, encompassing guaranteed client satisfaction with each endeavor.
Morteza can be reached at morteza.zohrabi@yahoo.com.
Tom has a strong track record of enhancing business and individual performance through learning. He has conducted research and led teams to achieve award-winning results for more than 50 Fortune 500 companies. His strengths include assessing training needs, designing and facilitating learning solutions, and measuring results, including ROI.
At Atkinson Analytics, Tom helps clients achieve better results throughout the learning process, from defining needs to measuring the learning experience, on-job application, and business results. Tom enables individuals to demonstrate value to executives, sustain and enhance performance gains, and target opportunities for increasing return (ROI) on learning investments.
Tom previously worked as a learning director at Fresenius Medical Care and led a team of professionals in delivering an ambitious curriculum of programs and projects, including more than 1,000 sessions of the companyโs keystone leadership development initiative. At Deloitte Consulting, Tom was a specialist master, designing award-winning training and measurement systems for some of Deloitteโs largest clients.
Tom also helps parents of children on the Autism spectrum build meaningful relationships and a foundation for growth and development.
Tom can be reached at tatkinson@atkinsonanalytics.com .
Suzanne Schell, CRP, is the CEO of ROI Institute Canada. Suzanne teaches, consults, and conducts ROI studies primarily focussed on the Healthcare industry.
Suzanne acquired her Certified ROI Professional designation in 2010. She has brought the ROI Methodology to healthcare organizations, hospitals, and healthcare ministries and authorities across Canada, through workshops, consulting, and ROI Certification.
Her workshops and consulting give healthcare professionals, HR professionals, and executives the knowledge and skills to measure impact, value and ROI of initiatives, programs, and projects. Key learnings include developing data collection and analysis strategies to identifying change in behavior and impact, ROI. Answering the questions โ did the investment make a difference, did it deliver the impact expected, and is this the solution moving forward?
Suzanne can be reached at suzanne@roiinstitutecanada.com.
Rana Imran is an HR and OD professional and strategist with 15+ years of experience with multinational corporations in Pakistan and the Middle East region in diverse multicultural and international environments. Rana has worked with telecom, e-commerce, FMCG, education, healthcare, and hospitality industries.
Rana’s expertise is in organization design and development, business transformation strategy, organizational culture, employee engagement, performance management, talent management and development, capability building, leadership development, and measuring program impact.
Rana has delivered more than 36,000 hours of training on behavioral and business skills.
Rana is certified in Hogan Assessment, Global Diversity & Inclusion standards, is an IFC World Bank Certified Trainer, and has received a Measuring Return on Investment Certificate from ATD and ROI Institute. He is Certified Practitioner of Global Leadership Assessment 360 and Working Group Leader for the Global ISO HR Standard development committee TC260.
Rana can be reached at ranaimran.hr@gmail.com.
Burdette (Pete) Fullerton PhD is the founder of Dr. Peteโs Leadership Accelerator, Inc., an Executive Coaching and Return on Investment (ROI) assistance firm. In order to assist community economic development professionals, Pete started up Dr. Peteโs Leadership Accelerator. Pete is a passionate advocate for passing it on and leaving the campsite better than he found it. He values character, trust, and integrity and looks to assist like-minded economic development and chamber of commerce professionals in achieving personal and professional excellence. Pete has been a community economic development professional for over 30 years. In his career, Pete has participated in the completion of projects that have directly invested over $3.5 billion, which currently employ more than 17,000 in Kansas City.
In 2017, Pete earned his Doctorate at the University of Southern Mississippi in Human Capital Development. Pete attended Texas A&M University and received his Bachelor of Arts degree in Political Science and a Master of Public Administration from the University of Missouri. Pete holds certifications as a Certified Economic Developer (CEcD), Certified Return on Investment Professional (CRP), Certified Professional Coach (CPC), and Certified Executive Coach (CEC).
Pete can be reached at pf@drpeteleads.com.
Nader Bechini is passionate about assisting people and organizations improve and maximize their performance and bottom-line results. His life mission is to support others in strengthening their abilities to inspire, influence, and create an impact.
Nader is the director of ROI Institute in the MENA region. He is also a global facilitator with the Association of Talent Development (ATD).For more than 20 years, he led human performance improvement projects, programs, and initiatives, in more than 30 countries in North and South America, Europe, Africa, Australia, and Asia.
Nader has received several awards for his achievements. On three occasions, ROI Institute awarded him for the best international implementation of the ROI Methodology (2015, 2019, and 2020). The Moroccan Society for Human Resource presented him with an award for the best African HR expert in 2019. His articles in learning and development have been translated to Korean, Russian, Chinese, French, Arabic, and Spanish.
Nader facilitates workshops, consults with organizations, and speaks at conferences worldwide. He has been interviewed by several radio and TV programs in Saudi Arabia, Sudan, Tunisia, France, and Algeria.
Nader holds a masterโs degree in marketing from the University of Tunis El Manar, Tunisia.
Nader can be reached at nader@roiinstitute.net.
Matic Kadliฤek is an organizational psychologist and CEO of Video Center, Ltd. He is a sought-after people development consultant to more than 100 companies, building and implementing internal learning centers, e-learning content and platforms, and tools for successful leadership development and skill building, with psychological testing, coaching, succession planning, work environment diagnostics, mentoring, on-boarding, and providing frameworks for credibly measuring the impact of people development programs using the ROI Methodology (ROI Institute, USA). He is one of the first EuroPSY certificate holders for Work & Organizational Psychology in Slovenia. Matic is a dedicated mentor to young psychologists and takes humor seriously.
Matic can be reached at matic.kadlicek@videocenter.si.
Hormazd Mistry, ATD Master Trainerโข, is the founder and CEO of Sarosh Consulting and ROI Institute India. He is the India partner for ROI Institute.
Hormazd is a learning & development professional who has twice received the Most Fabulous Training & Development Leaders (2020 & 2021) award by the World HRD Congress. He has been a part of the L&D community for over 18 years and has worked for multinational organizations like Wipro Ltd., HERE Technologies, IKS Health, leading various L&D teams across geographies.
With a passion for L&D, Hormazd has experience in different roles in various L&D verticals. Over the years, he has delivered over 25,000 hours of training on various topics including but not limited to technical, TTT, leadership, soft skills, customer service, behavioral training, etc., with high business impact.
Hormazd now works with L&D/HR professionals and teams to enable them to deliver high business impact with their solutions. He also helps them in measuring and showcasing the impact & ROI of their programs.
Hormazd can be reached at hormazd@roiinstitute-india.com.
Hashem Othman Hashem is a top-notch ROI specialist, trainer, and consultant with 13 years of experience in measuring the ROI of a wide variety of projects, including HR practices, training, health care, and KPIs. His expertise covers isolation techniques, forecasting ROI studies, training best practices, and designing and implementing training solutions based on their ROI. Hashem also has experience in overseeing career development, setting performance metrics, evaluating project productivity, and helping to create successful long-term initiatives within organizations. He serves as a point person for CEOs, CFOs, and project managers to ensure proper investing of project budgets. Hashem maintains excellent communication, change management, and leadership skills. He is distinguished for his problem resolution abilities and high level of confidentiality. Hashem is equally effective at conducting ROI studies, measuring project impact, and training supervision and delivery.
Hashem earned his Master of Strategic Human Resource Management (MSHR) degree at the University of Wollongong, Australia, and a Bachelor of Applied Physics degree at Muโtah University, Jordan. He achieved the designation of Certified ROI Professional with ROI Institute in July 2017.
Hashem can be reached at h.othman@criticalinnovation.ae.
Beryl Oldham has 30 yearsโ experience in organizational learning and development. Before moving fulltime into consulting in 2013, she held senior generalist HR, organizational development, and learning and development roles in a range of government, local body, and corporate organizations. Her experience spans workforce planning, technical/professional and leadership/management competency frameworks design, performance management systems design and implementation, learning design, training systems management and evaluation, and measuring ROI for human resources, training, and other organizational initiatives.
In addition to being a Certified ROI Professionalยฎ, Beryl also holds AQF certificates in Training Systems Management and Training Evaluation, and the NZQA Certificate in Adult Education and Training (Level Five). She was the 2014 recipient of the NZATD Education Trustโs Learning and Development Practitioner of the Year Award and the ROI Instituteโs 2014 Award of Excellence for Most Innovative Approach to ROI.
Beryl is passionate about measuring results and her company, Complete Learning Solutions, provides learning solutions that cover every aspect of L&D and make a difference by proving the value of capability development in people and organizations. Beryl has been a partner with ROI Institute for nearly 14 years.
Beryl can be reached at beryl.oldham@completelearning.co.nz.
Farzaneh Majed is an ROI Institute Partner and Balanced Scorecard Institute Strategic Partner and Advisory Board Member with over 20 years of successful business development, change management, and innovation experience who has a passion for excellence and thrives on driving change. Her extensive career at the global level has resulted in many successful initiatives being adopted lucratively. She believes in simplifying processes and delivering results through strategic planning and effective execution.
As CEO and Managing Partner of Transform Alliance, Farzaneh is committed to assisting organizations to achieve transformation through the engagement of their people with TA’s alliance partners. Farzaneh is an advocate of Inclusion and Collaboration and owes her success to putting herself in the other person’s shoes, seeing the world from their perspective, and seeking a win-win solution.
Farzaneh is a Key Performance Indicator Professional (KPIP), Balanced Scorecard Master Professional (BSMP), has a Master’s Certificate in Project Management from George Washington University, and an Honours degree in Combined Sciences from the University of Brighton, UK. She established the L&D Division of Informa, was selected as Centre of Excellence for Training among Informa Group, and achieved an innovation award for continuous improvement. Farzaneh is the author of Generation Inclusion.
Farzaneh can be reached at farzaneh-majed@transform-alliance.com.
Colin’s career spans more than 35 years, and he’s still smiling. He began as an information technology (IT) mainframe programmer, growing to run complex international IT projects for corporations in many business areas requiring advanced leadership, cross-cultural stakeholder management skills, and the delivery of quantifiable business value.
Originating from the United Kingdom, he has lived in the United States, the Netherlands, and Singapore at various times and operated in many other countries.
Since 2007, Colin has concentrated on consulting and training, developing several thousand project managers and leaders for corporate clients and educational institutions. In 2014, he completed his ROI impact study to become a Certified ROI Professional (CRP) and partnered his company with ROI Institute through various locations in South-East Asia.
Supported by his dedicated team, Colin conducts ROI consulting projects and courses as well as project management and related subjects. He is a Project Management Professional (PMP), Certified Scrum Master, Certified Product Owner, and holds a master’s degree from Arcadia University, PA. Colin also specializes in dispute resolution and sits on the Principle Panel of the Singapore Mediation Centre where he mediates large quantum business disputes. And, when he has five minutes, Colin organizes the Singapore Filmmakers Network.
Colin can be reached at enquiries@knightgriffin.com.