ROI in Agile Transformation: Using Design Thinking to Deliver and Measure Results

Klaas Toes MSc, is President Global Business Development and Director, ROI Institute-Europe, and boasts over 30 years of hands-on experience in shaping human capital strategies, leadership, change, and sales for diverse businesses and governmental agencies. A prominent learning economist, Klaas emphasizes the importance of concrete data when determining business impact and ROI.

Klaas founded ROI Institute-Europe in 1998 enabling him to support the measurement and evaluation of tangible and financial values of learning for individuals and organizations. This venture was instrumental in laying the groundwork for people analytics in Europe.

By 2012, Klaas published De Waarde van Leren. As a well-respected voice in his domain, Klaas often shares insights via lectures and keynotes on various subjects including people analytics, business alignment, and ROI. In 2024 Klaas was co author of The Return On Investment for Training and Performance Improvement programs.

In 2016, Klaas's expertise earned him a pivotal role as the European Council Director for People Analytics at The Conference Board, a position he held till 2021. He's not just a thought leader but a successful entrepreneur in the fields of HR and Learning & Development, with his ventures impacting organizations on a global scale.

Jennifer Wingate is the Communication and Research Manager at ROI Institute. Jennifer is skilled in managing all marketing and public relations campaigns and brand messaging. Before joining ROI Institute, she was a marketing professional working for companies like Encompass Health and Bradley Arant Boult Cummings for over fifteen years. She assisted Encompass Health with the national re-brand in 2019.

Jennifer is a University of Alabama in Birmingham graduate with a degree in communications management and mass communications. She enjoys spending time with her husband and puppies and hunting for antiques in her spare time.

Jennifer can be reached at jennifer@roiinstitute.net

Tim Brock

Tim Brock, Ph.D., a senior consultant at ROI Institute, facilitates ROI Certification and ROI workshops and provides ROI consulting services. Tim leads ROI Institute’s weekly live virtual group coaching sessions that support individuals working toward earning the Certified ROI Professional® (CRP) credential. He is also an invited presenter at multiple international conferences and has also authored chapters on the ROI Methodology in several books.

Tim is a retired Air Force field grade officer, having served in both the enlisted and officer ranks, serving as a nuclear weapons maintenance crew chief and a missile combat crew commander for both the Titan II and Minuteman II intercontinental ballistic missile weapon systems. During his later career at Lockheed Martin, Tim led their science of learning and performance improvement team as a human performance system engineer and architect. He has experience in the education profession as a K-12 private school principal, a college director of education, and associate professor. Tim earned his Ph.D. from Capella University in education with a specialization in training and performance improvement.

Tim enjoys spending time with his three grown children and three young grandchildren. He and his wife enjoy taking extended cruises to relax, read, and see the world.

Hope Nicholas

Hope Nicholas the Director of Publications for the ROI Institute. In this role she coordinates all publications, including books, articles, and case studies. Hope has significant experience with transcription, editing, proofreading, and project planning. Additionally, she contributes to multiple research and business development projects by facilitating communications and developing a wide variety of documents.

Hope is a graduate of the University of Akron and has over 15 years of experience working the field of healthcare administration and documentation. In her spare time, she enjoys painting, golfing, and playing tennis with friends and family.

Hope can be reached at hope@roiinstitute.net.

Kylie McLeod

Kylie McLeod manages research, assessment, and communication at ROI Institute. She manages proposals, develops data collection instruments, collects, and analyzes data, writes final evaluation reports, and manages research projects. She has previously edited ROI Institute books and articles and developed promotional and marketing materials. She served as editor, production manager, and coordinator of the book, Value for Money: How to Show the Value for Money for All Types of Projects and Programs In Governments, Nongovernmental Organizations, Nonprofits, and Businesses (Wiley, 2020).

Kylie graduated with a Bachelor’s in Communication and Information Sciences from The University of Alabama in 2015 and a Master of Arts in Journalism from the same institution in 2017. Kylie holds CoreXM and CoreXM Expert certifications from Qualtrics and earned the Certified ROI Professional credential in November 2019.

In her free time, Kylie cheers on the Alabama Crimson Tide, enjoys the (mostly) sunny weather in Northwest Florida, and dotes on her dog, Penny.

Kylie can be reached at kylie@roiinstitute.net.

Caroline Hubble

Caroline Hubble, M.S.O.D., is Vice President of Consulting Services with ROI Institute, Caroline manages the implementation of complex program evaluation projects, including conducting research and evaluation studies in various areas such as leadership development, process improvement strategies, and program implementation. In previous roles, she managed evaluation, analytics, reporting, and operations for business line and enterprise-wide HR departments. Her operational, project, and relationship management expertise is noted for significantly contributing to improved business practices.

Caroline holds a BA in Psychology from Rollins College and received her Masters of Science in Organizational Development in May 2013. Her publications include: ASTD Handbook of Measuring and Evaluating Training, Chapter 4: Using Surveys and Questionnaires. ASTD 2010; Measuring ROI in Learning & Development: Case Studies from Global Organizations, Chapter 4 (co-authored). ASTD 2012’ Offering Money for School Pays Off, (co-authored) Chief Learning Officer Magazine, June 2012.

When not focused on evaluation efforts, she enjoys spending time on her small farm in the mountains of North Carolina, raising Savannah and Egyptian Mau cats, following NASCAR, and capturing memories through photography.

Caroline can be reached at caroline@roiinstitute.net.

Jack. J Phillips, Ph.D.

Jack J. Phillips, Ph.D., chairman of ROI Institute, is a world-renowned expert on accountability, measurement, and evaluation. Jack provides consulting services for Fortune 500 companies and major global organizations, and regularly consults with clients in manufacturing, service, and government organizations in 70 countries. The author or editor of more than 100 books, he conducts workshops and presents at conferences throughout the world and has received several awards and honors for his work.

Jack has enjoyed almost 30 years of corporate experience in the aerospace, textile, metals, construction materials, and banking industries. He has served as training and development manager at two Fortune 500 firms, as senior human resource officer at two firms, and as president of a regional bank. Also, he served as management professor at a major state university.

Jack has undergraduate degrees in electrical engineering, physics, and mathematics; a master’s degree in Decision Sciences from Georgia State University; and a Ph.D. in Human Resource Management from the University of Alabama. He has served on the boards of several private businesses, nonprofits, and associations, including the American Society for Training and Development, the National Management Association, and the International Society for Performance Improvement, where he served as president (2012-2013).

Jack can be reached at jack@roiinstitute.net.

Patti P. Phillips, Ph.D.

Patti P. Phillips, Ph.D., is co-founder and CEO of ROI Institute. She is a driving force in the
global adoption of the ROI Methodology® and measurement and evaluation to influence organizational change. Her work as an educator, researcher, consultant, and coach supports leaders as they build the capacity to create and deliver value from investments in all types of endeavors. Her work spans private, public, nonprofit, and nongovernmental organizations.

Patti serves as a member of the Board of Trustees of the United Nations Institute for Training and Research and the UN System Staff College faculty in Turin, Italy. Additionally, she is chair of the Institute for Corporate Productivity People Analytics Board; Principal Research Fellow for The Conference Board; board member of the International Federation for Training and Development Organizations; and board chair for the Center for Talent Reporting. In 2015, she was honored by the Association for Talent Development as a Certified Professional in Talent Development Fellow for her contribution to the professionalism of the talent development field.

Patti contributes to various journals and is the author of books on measurement, evaluation, and ROI. CNBC, Euronews, and other news outlets feature her work.

Patti can be reached at patti@roiinstitute.net

What's It Worth? Analyze the Data

Isolate Project Effects
Convert Data to Money
Determine the Intangibles
Tabulate Fully-Loaded Costs
Calculate the ROI

Example

MYTH: The monetary value for a measure is difficult to locate or calculate.

REALITY: Most measures that matter have been converted to money.

The Kansas City SWAT team implemented the Outward Mindset program to reduce citizen complaints about the police, a measure of the quality of their work. The team leader, Chip Huth, and the team knew they could avoid the complaints by being nice guys, but they still had their jobs to do. Measures of productivity were equally important, including confiscating drugs, cash, and guns; serving warrants; and making arrests. As a result of Outward Mindset, they were able to improve productivity and quality. Reducing complaints was an important quality measure of success. However, according to Chip, “The Outward Mindset approach did not merely help us reduce complaints, but also helped us account for an unquantifiable reduction in lawsuits, injuries, etc., and an increase in public trust, collaboration, and drug and gun seizures.”

What else cause the complaints to decrease? The number of missions did not change, and the types of missions didn’t change. There were no policy changes and no high-profile cases in the news. In essence, the team could not think of any other factors and believed that their new Outward Mindset approach had made the difference. According to Chip, the complaints disappeared because of the behaviors from the Outward Mind set, “The way we responded to the reality of others while executing our work, including some changes in procedures that were born out of our commitment to seeing others, made the complaints disappear.”

When comparing the monetary benefits of reducing complaints for one year to the cost of preparing this team to use the Outward Mindset, the ROI is 5,724 percent. This is an almost unbelievable ROI, but the process and the data prove it is credible.

Story from Show the Value of What You Do, Chapter 6, Page 97

So What? Leverage the Results

Communicate Results
Optimize Results
Leverage Results

Example

MYTH: The success of one project rarely influences investing in another project.

REALITY: The best results to continue to invest in are previous successes.

Jessica Kriegel, an organization and talent development consultant at Oracle with expertise in strategic planning, talent management, and leadership development, was particularly focused on intergenerational understanding. More specifically, she realized that generational labels do not work. The labels aren’t bad. It’s the associations that society puts on those labels that are bad.

Although labels abound with each generational era, millennials (those born between 1980 and 2000) are the most misunderstood generation. These labels keep individuals, managers, and even organizations from being as successful as they should be. Jessica’s work at Oracle pro vided her with an opportunity to tackle this challenge directly. One project clearly focused on this challenge, and she needed to show its value.1 A product development team member reached out to the person in charge of the Oracle College Hire Program about their problem with the millennials. The team member suggested that the millennials didn’t understand the corporate culture, and as a result, there were behavioral and communication issues. This suggestion included a request for a training program to help the millennials acclimate to the company.

Jessica took this project on. She wanted to understand the issue and ensure that the correct program was implemented. She did not want to take on the project on the face value that millennials were the problem, and if we fix them, everything is okay.

Each year, this program develops approximately 100 to 200 new college hires, making this an important program. She examined some of the data that were surrounding the situation. A lack of productivity was reported, along with the excessive turnover of the millennials. Some were leaving soon after employment, which was very expensive. Consequently, the business outcomes of interest were im proved productivity and reduced turnover.

To understand the right solution, Jessica conducted a detailed sur vey with the managers and the employees. The surveys indicated that:

New hires sometimes struggle with acclimating to the company,
There was almost no intergenerational conflict,
A few differences were perceived across generations, and
Both the new college hires and the managers seemed to enjoy their work and working together.

Next, Jessica conducted focus groups to examine the problem in more detail, and the focus groups identified areas where improvement was needed. The managers:

Felt that the generational cohorts were different,
Claimed that the millennials did not understand the difference between social and professional networking, and
Thought that millennials had low emotional

Meanwhile, the millennials felt they:

Needed more information about why their jobs mattered,
Wanted more communication from their manager, and
Were struggling to find a work life

With these issues identified, the solution was designed to work with both groups to ensure that they worked productively and efficiently together, addressing their individual needs in two different programs (one for the managers and one for the employees). She set out to show the value of her program.

As Jessica conducted the programs, she captured reaction data. The employees felt the program was valuable, and it helped them understand more about their roles in the organization. She measured learning from both groups and found they were learning what they needed to do to be more successful. She measured application to see the extent to which they were working together because of the training. The reports were positive that they were working collaboratively. With the increased collaboration came improvements in retention and productivity. With these improvements, she took a step to sort out the effects of the program from other influences, converted these measures to money, compared program benefits to the cost of the program, and calculated that the return on investment was 695 percent. The executives were impressed, and she communicated the results to other groups and included a copy of her bestselling book, Unfairly Labeled.

Leveraging the results of Jessica’s project has made a difference— not only for the organization in which she was working at the time, but also for the program participants and the individuals she now reaches through her writing and speaking.

Story from Show the Value of What You Do, Chapter 7, Page 117

How Much? Collect Data Along the Way

Data Collection
Data Collection Methods
Data Sources
Minimize Bias
Timing

Example

MYTH: Data collection to show the success of a project is almost impossible.

REALITY: With planning and effort, data collection becomes feasible and successful.

Zoe Moore has been very determined. She helps organizations develop their Diversity, Equity & Inclusion (DE&I) strategies by ensuring there is diverse representation in leadership and recurring professional development focused on DE&I that improves workplace culture and supplier diversity programs to increase opportunities for under resourced businesses. She has garnished attention around the sub jects, but still, many leaders ask the question, “So what? What value do diversity, equity, and inclusion bring?”

Of course, supplier diversity is important, but that is just the starting point. Equity and inclusivity are imperative across the global business of hospitality and are where the real value of Zoe’s work re sides. But how can she demonstrate this value? She needs to collect data about:

How the group reacts to diversity, equity, and inclusion (reaction),
What they are learning about the importance of DE&I and their role in making it work (learning),
What actions they are taking and the behaviors they are changing to make this successful (application), and finally
What impact is occurring with the team’s communication that is influencing workplace culture (impact).

This creates a new data collection stream for Zoe. While she’s worked hard to improve awareness around supplier diversity through representation metrics, the challenge is to make sure that the teams integrate DE&I successfully, and she must collect data along the same five levels of success that form the value chain, with supplier diversity being the beginning point. This situation highlights that the five levels of success, first introduced in Chapter 1, can be examined at different times in projects and often from different perspectives. This requires data collection along the value chain to make it work.

Story from Show the Value of What You Do, Chapter 5, Page 83

How? Select the Right Solution

How to Achieve Success
The Power of Objectives
Specificity Drives Results
Communicate, Communicate, Communicate
Plan the Remainder of the Journey

Example

MYTH: Objectives add little value to the outcomes.

REALITY: Objectives are powerful drivers of success.

Martin Burt, Ph.D., has a lifelong quest to understand and abolish poverty. His inspirational experience in public service includes serving as chief of staff for the president of Paraguay, the mayor of Asunción (the capital city of Paraguay), and the vice minister of commerce. Martin is the founder and CEO of Fundación Paraguaya, a nonprofit devoted to promoting social entrepreneurship and economic self reliance to eliminate poverty worldwide.

Martin has challenged many assumptions about poverty by asking, “What if everything we knew about poverty was wrong? What if the legions of policy makers, social scientists, economists, aid workers, charities, and NGOs marching across the globe have been using the wrong strategy and tactics to wage the wrong war against poverty?”

In his book, Who Owns Poverty?, Martin lays out the frame work necessary to eliminate poverty. The answer to his question, “Who owns poverty?” is clear—the people in poverty. They don’t want to be in poverty. But people in poverty are not just persons with low income labels. They are individuals with joy, generosity, and creativity. They have problem solving and entrepreneurial spirits, and they want to rise out of poverty. They just need help and support to get there, and they will get there if we give them an avenue to do so.

Martin created the Poverty Stoplight program that shows families what they must do to come out of poverty. It’s not just increasing in come, but it’s tackling a host of issues that altogether contribute to poverty. He developed 50 indicators of poverty, as shown in Table 9. Martin created the stoplight program with clear measures and three phases: red, yellow, and green—red (a worst case scenario), yellow (making progress), and green (out of poverty). Along with each indicator, for each phase, the Poverty Stoplight team developed specific measures of success so that all stakeholders, including the families in poverty, can see where they are going, and the progress being made. Progress follows the five level framework of reaction, learning, application, impact, and ROI.

Martin needs to show the value of the Poverty Stoplight program. He needs to push its evaluation to the ROI level for governments, as they calculate benefit-cost analysis for helping to eliminate poverty. He also needs to calculate the ROI for the companies supporting the program, showing them that this is a good investment for the company and the community. The major foundations involved in the NGOs and the charities don’t need to see ROI, but they all need to see the impact. The impact is getting families out of poverty and reaching the green light on the Poverty Stoplight. Martin needs the Show the Value Process to show the value of what they do.

Story from Show the Value of What You Do, Chapter 4, Page 61

How? Select the Right Solution

Find a Solution
Performance that Needs to Change
Learning that Supports the Change
Preferences that Need to be Met
Connect the Dots

Example

MYTH: The solution to a problem is almost always obvious.

REALITY: The right solution is rarely obvious.

Ginger Luttrell spent 10 years as a software engineer, including con figuration and programming at one of the world’s largest business software companies. The intent of new systems implementations is to improve output, quality, cost, and time. Over the years of working with these systems implementations, she noticed a problem. When an implementation project would go live, endusers, those who worked in the business functional areas or departments, would ask questions such as:

Why did we spend money on software and implementation projects yet spend little on training and support for those who use the software?
Am I the only one who thinks the help desk isn’t helpful?
Am I the only one who thinks that better system use will natu rally lead to better business results?
How can we organize training and support without asking for a burdensome budget?
What do I need to learn to be a better user and to better help my colleagues?

Interestingly, end-users received comprehensive training on these systems during projects. Despite this fact, Ginger noticed that they experienced problems and frustrations on the job. As a result, productivity, quality, and time savings sufferer rather than improved. The types of problems Ginger was addressing were universal. Additionally, some end-users would become so frustrated they would leave the company altogether. Ginger also saw that there was no channel for the voice of the end-users to get to the senior executives to address their issues and concerns.

Ginger began to realize a possible solution by addressing the precise business needs identified (productivity, quality, time, cost, and retention). The end-users needed a person who would support them on a routine basis. This person would serve as a mentor, coach, trainer, counselor, and trusted advisor to help them through complex implementation issues. This person is called a super user. At the same time, super users would help develop the end-users to become more valuable business team members—members with a broader vision and more complex skill sets, who could offer greater support in their organization.

Ginger realized that the cost of providing super users was much less than the savings that would be generated to make the software system more effective for the organization. She tested her theory in different organizations and was convinced that she had the right solution for the perplexing problem of largescale systems implementation. Eventually, Ginger founded the Super User Network and now spends much of her time preparing and developing super users to perform this essential role in business departments worldwide.

Story from Show the Value of What You Do, Chapter 3, Page 45

Why? Start with Impact

Begin With the End in Mind
Define Impact
Capture Value Before Being Asked

Example

MYTH: It’s almost impossible to connect most projects to business needs.

REALITY: Essentially, every project can be connected to a business need.

Chip Huth, the new leader of a SWAT team in the Kansas City, Missouri, Police Department, knew what he had to do. Citizen complaints against the team were excessive, averaging about 30–40 per year. They were also expensive, costing the department an average of $70,000 per complaint just for the investigations. The deputy chief (DC) who appointed Chip was clear—Chip needed to “clean up” the mess. The DC was concerned about optics from the complaints and the many irritations they were causing.

Expectations were vague, and the accepted remedy was potentially a “house cleaning” and subsequent creation of a new team. Chip knew what he was facing. He accepted the assignment because he loved working in teams. Initially, Chip didn’t believe the team could be “fixed.” Admittedly, Chip was completely self-centered in his thinking and simply loved the thrill of the work. He reassured the DC that he could whip the team into shape (not believing anyone could do so) and initially sought token improvements by addressing problem behaviors.

Unexpectedly, through a series of personal and professional encounters, Chip was exposed to a personal improvement process called Out ward Mindset from the Arbinger Institute. Arbinger’s research indicates that people operate at any given time from one of two mindsets: an in ward mindset or an outward mindset. When operating with an inward mindset, you focus only on your own goals and objectives without considering your impact on others. With an outward mindset, however, you see others as people who matter as much as you do. You consider their needs, challenges, and objectives. And you focus on collective results.

As Chip began applying the Outward Mindset framework to each area of his life, including his leadership responsibilities with the team, it seemed logical that this could impact the team. Although Chip was not certain that the Outward Mindset would deliver fewer complaints or build a better relationship with the community, he was willing to give it a try.

He introduced the team to this process. When they were exposed to it, they responded positively. The team asked Chip to lead them in applying the Outward Mindset skills and framework, in role modeling the behaviors, and showing them that it works. That was all Chip needed. He had the business measure, citizen complaints at roughly $70,000 each, the right solution, Outward Mindset, and the right people at the right time. He was ready to go.

Story from Show the Value of What You Do, Chapter 2, Page 29

Evan Seidner is a Senior Analyst for ROI Institute. In his role as an analyst, Evan does mixed-methods data analysis for clients. Evan also supports other team members in the evaluation process and implementation processes. In previous roles, he has worked with environmental NGOs in China as well as working for an agribusiness in Tokyo, Japan.

Evan holds a BA in Global Studies with minors in Political Science and East Asian Languages and Cultures from the University of Illinois at Urbana-Champaign and received his MA in International Relations from the American University in Washington, D.C.

In his free time, Evan enjoys learning photography and videography. He is a Japanese film buff, which he watches to help maintain his language skills.

Evan can be reached at evan@roiinstitute.net

How to Develop Turnover Costs

The first step in monitoring turnover costs is to define and discuss several issues about a cost control system. The key issues are presented here.

Some Costs Are Difficult to Determine

The huge difference that often exists between the cost estimates from HR and the actual turnover cost lies in the indirect or hidden costs category. Where direct costs are usually in the cost accounting system, the hidden costs are almost never considered. However, they can be developed using assumptions and estimation processes.

Fully-Loaded Costs

Using the approach to capturing the fully-loaded costs of turnover, each cost is identified and put into a specific category. Where an estimate is required, the entry is adjusted later. If possible, someone from finance and accounting should review and approve the data. The process should be able to withstand even the closest scrutiny, so organizations must ensure that all costs are included.

Reporting Total Cost

Costs are typically expressed as a percentage of the wages and salaries of the employees in a particular job group. This figure is usually determined after a detailed cost study is conducted. The percentage can be fixed for a group (for example, the sales force) or a specific job (for example, client relationship manager [CRM]).

For example, if a cost study in another industry has concluded that the cost to replace a CRM averages 150 percent of pay, this amount can be a beginning point. If there is some concern about the cost being too high, perhaps a lower number would be appropriate, such as 120 percent or 100 percent. After a figure has been determined, the turnover cost is then reported on statements along with the actual costs.

Morteza Zohrabi, MD, MBB, PMP, CRP, is the lead director in the implementation of countless mega-projects in both public and private sectors for large hospitals, government offices, manufacturers, and universities, to name a few. He is also a Certified Return on Investment Professional, publishing numerous articles and presentations on guaranteed value for money.

With over 20 years of expertise and hundreds of millions of dollars in savings, Morteza has done it all. He has delivered innovative solutions to old-time problems, with a track record of outstanding results, enhancing throughput, capacity, staff/client satisfaction, and shrinking turnaround times.

Through a combination of conventional and modern management techniques, Morteza and his team have revolutionized the management consulting industry by providing state-of-the-art business transformations and distinctively conforming as growth partners, encompassing guaranteed client satisfaction with each endeavor.

Morteza can be reached at morteza.zohrabi@yahoo.com.

Tom has a strong track record of enhancing business and individual performance through learning. He has conducted research and led teams to achieve award-winning results for more than 50 Fortune 500 companies. His strengths include assessing training needs, designing and facilitating learning solutions, and measuring results, including ROI.

At Atkinson Analytics, Tom helps clients achieve better results throughout the learning process, from defining needs to measuring the learning experience, on-job application, and business results. Tom enables individuals to demonstrate value to executives, sustain and enhance performance gains, and target opportunities for increasing return (ROI) on learning investments.

Tom previously worked as a learning director at Fresenius Medical Care and led a team of professionals in delivering an ambitious curriculum of programs and projects, including more than 1,000 sessions of the company’s keystone leadership development initiative. At Deloitte Consulting, Tom was a specialist master, designing award-winning training and measurement systems for some of Deloitte’s largest clients.

Tom also helps parents of children on the Autism spectrum build meaningful relationships and a foundation for growth and development.

Tom can be reached at tatkinson@atkinsonanalytics.com .

Suzanne Schell, CRP, is the CEO of ROI Institute Canada. Suzanne teaches, consults, and conducts ROI studies primarily focussed on the Healthcare industry.

Suzanne acquired her Certified ROI Professional designation in 2010. She has brought the ROI Methodology to healthcare organizations, hospitals, and healthcare ministries and authorities across Canada, through workshops, consulting, and ROI Certification.

Her workshops and consulting give healthcare professionals, HR professionals, and executives the knowledge and skills to measure impact, value and ROI of initiatives, programs, and projects. Key learnings include developing data collection and analysis strategies to identifying change in behavior and impact, ROI. Answering the questions – did the investment make a difference, did it deliver the impact expected, and is this the solution moving forward?

Suzanne can be reached at suzanne@roiinstitutecanada.com.

Rana Imran is an HR and OD professional and strategist with 15+ years of experience with multinational corporations in Pakistan and the Middle East region in diverse multicultural and international environments. Rana has worked with telecom, e-commerce, FMCG, education, healthcare, and hospitality industries.

Rana’s expertise is in organization design and development, business transformation strategy, organizational culture, employee engagement, performance management, talent management and development, capability building, leadership development, and measuring program impact.

Rana has delivered more than 36,000 hours of training on behavioral and business skills.

Rana is certified in Hogan Assessment, Global Diversity & Inclusion standards, is an IFC World Bank Certified Trainer, and has received a Measuring Return on Investment Certificate from ATD and ROI Institute. He is Certified Practitioner of Global Leadership Assessment 360 and Working Group Leader for the Global ISO HR Standard development committee TC260.

Rana can be reached at ranaimran.hr@gmail.com.

Burdette (Pete) Fullerton PhD is the founder of Dr. Pete’s Leadership Accelerator, Inc., an Executive Coaching and Return on Investment (ROI) assistance firm. In order to assist community economic development professionals, Pete started up Dr. Pete’s Leadership Accelerator. Pete is a passionate advocate for passing it on and leaving the campsite better than he found it. He values character, trust, and integrity and looks to assist like-minded economic development and chamber of commerce professionals in achieving personal and professional excellence. Pete has been a community economic development professional for over 30 years. In his career, Pete has participated in the completion of projects that have directly invested over $3.5 billion, which currently employ more than 17,000 in Kansas City.

In 2017, Pete earned his Doctorate at the University of Southern Mississippi in Human Capital Development. Pete attended Texas A&M University and received his Bachelor of Arts degree in Political Science and a Master of Public Administration from the University of Missouri. Pete holds certifications as a Certified Economic Developer (CEcD), Certified Return on Investment Professional (CRP), Certified Professional Coach (CPC), and Certified Executive Coach (CEC).

Pete can be reached at pf@drpeteleads.com .

Nader Bechini is passionate about assisting people and organizations improve and maximize their performance and bottom-line results. His life mission is to support others in strengthening their abilities to inspire, influence, and create an impact.

Nader is the director of ROI Institute in the MENA region. He is also a global facilitator with the Association of Talent Development (ATD).For more than 20 years, he led human performance improvement projects, programs, and initiatives, in more than 30 countries in North and South America, Europe, Africa, Australia, and Asia.

Nader has received several awards for his achievements. On three occasions, ROI Institute awarded him for the best international implementation of the ROI Methodology (2015, 2019, and 2020). The Moroccan Society for Human Resource presented him with an award for the best African HR expert in 2019. His articles in learning and development have been translated to Korean, Russian, Chinese, French, Arabic, and Spanish.

Nader facilitates workshops, consults with organizations, and speaks at conferences worldwide. He has been interviewed by several radio and TV programs in Saudi Arabia, Sudan, Tunisia, France, and Algeria.

Nader holds a master’s degree in marketing from the University of Tunis El Manar, Tunisia.

Nader can be reached at nader@roiinstitute.net.

Matic Kadliček is an organizational psychologist and CEO of Video Center, Ltd. He is a sought-after people development consultant to more than 100 companies, building and implementing internal learning centers, e-learning content and platforms, and tools for successful leadership development and skill building, with psychological testing, coaching, succession planning, work environment diagnostics, mentoring, on-boarding, and providing frameworks for credibly measuring the impact of people development programs using the ROI Methodology (ROI Institute, USA). He is one of the first EuroPSY certificate holders for Work & Organizational Psychology in Slovenia. Matic is a dedicated mentor to young psychologists and takes humor seriously.

Matic can be reached at matic.kadlicek@videocenter.si.

Hormazd Mistry, ATD Master Trainer™, is the founder and CEO of Sarosh Consulting and ROI Institute India. He is the India partner for ROI Institute.

Hormazd is a learning & development professional who has twice received the Most Fabulous Training & Development Leaders (2020 & 2021) award by the World HRD Congress. He has been a part of the L&D community for over 18 years and has worked for multinational organizations like Wipro Ltd., HERE Technologies, IKS Health, leading various L&D teams across geographies.

With a passion for L&D, Hormazd has experience in different roles in various L&D verticals. Over the years, he has delivered over 25,000 hours of training on various topics including but not limited to technical, TTT, leadership, soft skills, customer service, behavioral training, etc., with high business impact.

Hormazd now works with L&D/HR professionals and teams to enable them to deliver high business impact with their solutions. He also helps them in measuring and showcasing the impact & ROI of their programs.

Hormazd can be reached at hormazd@roiinstitute-india.com.

Hashem Othman Hashem is a top-notch ROI specialist, trainer, and consultant with 13 years of experience in measuring the ROI of a wide variety of projects, including HR practices, training, health care, and KPIs. His expertise covers isolation techniques, forecasting ROI studies, training best practices, and designing and implementing training solutions based on their ROI. Hashem also has experience in overseeing career development, setting performance metrics, evaluating project productivity, and helping to create successful long-term initiatives within organizations. He serves as a point person for CEOs, CFOs, and project managers to ensure proper investing of project budgets. Hashem maintains excellent communication, change management, and leadership skills. He is distinguished for his problem resolution abilities and high level of confidentiality. Hashem is equally effective at conducting ROI studies, measuring project impact, and training supervision and delivery.

Hashem earned his Master of Strategic Human Resource Management (MSHR) degree at the University of Wollongong, Australia, and a Bachelor of Applied Physics degree at Mu’tah University, Jordan. He achieved the designation of Certified ROI Professional with ROI Institute in July 2017.

Hashem can be reached at h.othman@criticalinnovation.ae.

Beryl Oldham has 30 years’ experience in organizational learning and development. Before moving fulltime into consulting in 2013, she held senior generalist HR, organizational development, and learning and development roles in a range of government, local body, and corporate organizations. Her experience spans workforce planning, technical/professional and leadership/management competency frameworks design, performance management systems design and implementation, learning design, training systems management and evaluation, and measuring ROI for human resources, training, and other organizational initiatives.

In addition to being a Certified ROI Professional®, Beryl also holds AQF certificates in Training Systems Management and Training Evaluation, and the NZQA Certificate in Adult Education and Training (Level Five). She was the 2014 recipient of the NZATD Education Trust’s Learning and Development Practitioner of the Year Award and the ROI Institute’s 2014 Award of Excellence for Most Innovative Approach to ROI.

Beryl is passionate about measuring results and her company, Complete Learning Solutions, provides learning solutions that cover every aspect of L&D and make a difference by proving the value of capability development in people and organizations. Beryl has been a partner with ROI Institute for nearly 14 years.

Beryl can be reached at beryl.oldham@completelearning.co.nz .

Farzaneh Majed is an ROI Institute Partner and Balanced Scorecard Institute Strategic Partner and Advisory Board Member with over 20 years of successful business development, change management, and innovation experience who has a passion for excellence and thrives on driving change. Her extensive career at the global level has resulted in many successful initiatives being adopted lucratively. She believes in simplifying processes and delivering results through strategic planning and effective execution.

As CEO and Managing Partner of Transform Alliance, Farzaneh is committed to assisting organizations to achieve transformation through the engagement of their people with TA’s alliance partners. Farzaneh is an advocate of Inclusion and Collaboration and owes her success to putting herself in the other person’s shoes, seeing the world from their perspective, and seeking a win-win solution.

Farzaneh is a Key Performance Indicator Professional (KPIP), Balanced Scorecard Master Professional (BSMP), has a Master’s Certificate in Project Management from George Washington University, and an Honours degree in Combined Sciences from the University of Brighton, UK. She established the L&D Division of Informa, was selected as Centre of Excellence for Training among Informa Group, and achieved an innovation award for continuous improvement. Farzaneh is the author of Generation Inclusion.

Farzaneh can be reached at farzaneh-majed@transform-alliance.com.

Colin’s career spans more than 35 years, and he’s still smiling. He began as an information technology (IT) mainframe programmer, growing to run complex international IT projects for corporations in many business areas requiring advanced leadership, cross-cultural stakeholder management skills, and the delivery of quantifiable business value.

Originating from the United Kingdom, he has lived in the United States, the Netherlands, and Singapore at various times and operated in many other countries.

Since 2007, Colin has concentrated on consulting and training, developing several thousand project managers and leaders for corporate clients and educational institutions. In 2014, he completed his ROI impact study to become a Certified ROI Professional (CRP) and partnered his company with ROI Institute through various locations in South-East Asia.

Supported by his dedicated team, Colin conducts ROI consulting projects and courses as well as project management and related subjects. He is a Project Management Professional (PMP), Certified Scrum Master, Certified Product Owner, and holds a master’s degree from Arcadia University, PA. Colin also specializes in dispute resolution and sits on the Principle Panel of the Singapore Mediation Centre where he mediates large quantum business disputes. And, when he has five minutes, Colin organizes the Singapore Filmmakers Network.

Colin can be reached at enquiries@knightgriffin.com.

How to Become ROI Certified Book Giveaway

Name(Required)

First Last

Phone(Required)

Email(Required)

Organization(Required)

Job Title(Required)

Address(Required)

Street Address Address Line 2 City State / Province / Region ZIP / Postal Code Country

CAPTCHA

HRDQ Book Giveaway

Name(Required)

First Last

Phone(Required)

Email(Required)

Organization(Required)

Job Title(Required)

Address(Required)

Street Address Address Line 2 City State / Province / Region ZIP / Postal Code Country

CAPTCHA

Patti P. Phillips, Ph.D., is co-founder and CEO of ROI Institute. She is a driving force in the global adoption of the ROI Methodology and measurement and evaluation to influence organizational change. Her work as an educator, researcher, consultant, and coach supports leaders as they build the capacity to create and deliver value from investments in all types of endeavors. Her work spans private, public, nonprofit, and nongovernmental organizations.

Patti serves as a member of the Board of Trustees of the United Nations Institute for Training and Research (UNITAR) and the UN System Staff College faculty in Turin, Italy. Additionally, she is chair of the Institute for Corporate Productivity (i4cp) People Analytics Board; Principal Research Fellow for The Conference Board; board member of the International Federation for Training and Development Organizations (IFTDO); and board chair for the Center for Talent Reporting (CTR). In 2015, she was honored by the Association for Talent Development as a Certified Professional in Talent Development (CPTD) Fellow for her contribution to the professionalism of the talent development field.

Patti contributes to various journals and is the author of books on measurement, evaluation, and ROI. CNBC, Euronews, and other news outlets feature her work.

Patti can be reached at patti@roiinstitute.net

Kylie McLeod manages research, assessment, and communication at ROI Institute. She develops data collection instruments, collects, and analyzes data, writes final evaluation reports, manages research projects, coordinates the communication process, and directs the ROI Institute Awards Program. She also edits ROI Institute books and articles, develops promotional and marketing materials, and assists with the proposal creation process. She served as editor, production manager, and coordinator of the book, Value for Money: How to Show the Value for Money for All Types of Projects and Programs In Governments, Nongovernmental Organizations, Nonprofits, and Businesses (Wiley, 2020).

Kylie graduated with a Bachelor's in Communication and Information Sciences from The University of Alabama in 2015 and a Master of Arts in Journalism from the same institution in 2017. Kylie holds CoreXM and CoreXM Expert certifications from Qualtrics and earned the Certified ROI Practitioner credential in November 2019.

In her free time, Kylie cheers on the Alabama Crimson Tide, enjoys the (mostly) sunny weather in Northwest Florida, and dotes on her dog, Penny.

Kylie can be reached at kylie@roiinstitute.net.

Andy Vance is Director of Client Relationships and IT Manager for ROI Institute. In his role as Director of Client Relationships, Andy generates sales, implements marketing strategies, and maintains connections with clients and participants. In his IT Manager role, Andy maintains ROI Institute’s technology infrastructure, produces virtual events, designing and updating webpages, and ensures online security.

Andy is a graduate of Phoenix Online University and has a Practical Project Manager certification. Andy also is a Glenwood Junior Board member a non profit dedicated to the support of adults and children with autism.

In his spare time, he enjoys hiking, camping, swimming, and building models.

Andy can be reached at andy@roiinstitute.net.

Jack J. Phillips, Ph.D., chairman of ROI Institute, is a world-renowned expert on accountability, measurement, and evaluation. Jack provides consulting services for Fortune 500 companies and major global organizations, and regularly consults with clients in manufacturing, service, and government organizations in 70 countries. The author or editor of more than 100 books, he conducts workshops and presents at conferences throughout the world and has received several awards and honors for his work.

Jack has enjoyed almost 30 years of corporate experience in the aerospace, textile, metals, construction materials, and banking industries. He has served as training and development manager at two Fortune 500 firms, as senior human resource officer at two firms, and as president of a regional bank. Also, he served as management professor at a major state university.

Jack has undergraduate degrees in electrical engineering, physics, and mathematics; a master’s degree in Decision Sciences from Georgia State University; and a Ph.D. in Human Resource Management from the University of Alabama. He has served on the boards of several private businesses, nonprofits, and associations, including the American Society for Training and Development, the National Management Association, and the International Society for Performance Improvement, where he served as president (2012-2013).

Jack can be reached at jack@roiinstitute.net.

Hope Nicholas is the Director of Publications for ROI Institute. In this role she coordinates all publications, including books, articles, and case studies. Hope has significant experience with transcription, editing, proofreading, and project planning. Additionally, she contributes to multiple research and business development projects by facilitating communications and developing a wide variety of documents.

Hope is a graduate of the University of Akron and has over 15 years of experience working the field of healthcare administration and documentation. In her spare time, she enjoys painting, golfing, and playing tennis with friends and family.

Hope can be reached at hope@roiinstitute.net.

Caroline Hubble is Vice President of Consulting Services with ROI Institute. She manages the implementation of complex program evaluation projects, including conducting research and evaluation studies in various areas such as leadership development, process improvement strategies, and program implementation. In previous roles, she managed evaluation, analytics, reporting, and operations for business line and enterprise-wide HR departments. Her operational, project, and relationship management expertise is noted for significantly contributing to improved business practices.

Caroline holds a BA in Psychology from Rollins College and received her Masters of Science in Organizational Development in May 2013. Her publications include: ASTD Handbook of Measuring and Evaluating Training, Chapter 4: Using Surveys and Questionnaires. ASTD 2010; Measuring ROI in Learning & Development: Case Studies from Global Organizations, Chapter 4 (co-authored). ASTD 2012’ Offering Money for School Pays Off, (co-authored) Chief Learning Officer Magazine, June 2012.

When not focused on evaluation efforts, she enjoys spending time on her small farm in the mountains of North Carolina, raising Savannah and Egyptian Mau cats, following NASCAR, and capturing memories through photography.

Caroline can be reached at caroline@roiinstitute.net.

Ann Akins has been associated with ROI Institute for over 23 years. While primarily providing financial support for the company, in 2019, she also became the President of ROI expanding her role into the operational areas of the company. Prior to joining ROI, Ann was a regional controller for Waste Management with over $250 million in annual revenue. She has also led a customer experience center for FIS (a financial services company) with over 1,200 employees. This previous experience has helped Ann provide significant financial and operational insights for ROI operations.

Ann has a BS in Accounting from the University of West Alabama and a Master’s in Business Administration from Mississippi State University. An avid University of Alabama sports fan, she enjoys attending UA football games and spending time with her children.

Ann can be reached at ann@roiinstitute.net.

Melissa Brown is the Director of Partnerships at ROI Institute. She manages ROI Institute’s network of more than 70 international partners and associates, as they implement the ROI Methodology® globally. Melissa also oversees marketing by coordinating and designing marketing campaigns, the layout and design of course materials, research projects, promotional collateral, and other communication resources. Melissa works alongside clients, partners, and associates to ensure quality results. Melissa is a Certified Digital Marketing Professional and American Marketing Association Professional Certified Marketer in Digital Marketing, and she also holds a Qualtrics Platform Essentials Certification.

Melissa earned a bachelor’s degree in Communication and Information Sciences from The University of Alabama and a master’s degree in Communication from Walden University. She also holds a teaching license in secondary language arts. Melissa and her family enjoy traveling, watching and playing sports, and spending time outdoors.

Melissa can be reached at melissa@roiinstitute.net.

Download the Slide Deck