We all profess the virtues of “measuring results” from our strategic investments and initiatives. It appears to make perfect business sense to calculate the ROI after a completed project to evaluate both its success and also learn from potential mistakes in order to improve.
Easier said than done. Success measures are almost certainly defined in the formulation of a business case but then often neglected particularly as prolonged initiatives wrap up. Why? There are many reasons, including changes in scope and direction of the project or a queue of other equally critical projects waiting to be tackled.
But perhaps the most common reason that post project ROI calculations fail to get done is how difficult they really are to calculate. Pure measurement can only happen if time were to stand still - no other change projects, no market and economy swings, no employee turnover, etc. - from the time the business case is committed through deployment on day one and until results are measured one, two, three or even more quarters later. In other words, all other factors would need to stay the same in order to purely measure the ROI impact of the initiative in question. This is never going to happen, so what are the alternatives?
ROI Projections Determine Priority
ROI analysis is how you determine priorities amongst ideas and make decisions about which ones to move forward. The benefits projected should be a numerical estimate of positive impacts over a timeline, and the costs should similarly project the outlays over the same timeline. Many unknown values will undoubtedly be factored into estimates of outcomes. Great leaders seek a high degree of accuracy in their assumptions, but they also accept the degree of guesswork involved and refuse to confuse lack of precise projections with lack of merit in their ideas. In other words, a leader may need to guesstimate a great number of assumptions, but his or her idea might still become the best change the company has ever implemented.
Results are Measured in Improvements
After the initiative is completed, the goal is to measure performance change and improvement, but not necessarily to replicate the original ROI calculation. A leader must isolate certain factors of measurement that are going to change as a direct function of the initiative’s impact. In an ideal situation, the leader measures select indicators on Day 1 of a release or “go live” date and takes the same snapshot over planned increments in the days, weeks, months, or quarters to follow. Whereas a new product release provides simple measurement of impact (i.e. # of products sold at average margin of x%), most operational improvement initiatives are much more difficult to measure in isolation.
One method commonly used is to begin at the top node of revenue and/or cost “tree”, and work down the nodes until a measure can be defined as a direct impact of the change delivered.
For instance, assume that a company is striving to reduce customer churn rates and chooses to implement new diagnostic tools for their call center. Reduction in churn leads to increased revenue and decreased costs to serve. Moving down the tree, one might choose to measure the percent of customer issues resolved on first call and mean time to resolve an issue (MTTR). The ROI projected in the business case speaks to revenue increase and cost reduction, but revenue and cost measures are too high up the tree to be useful in determining performance of the newly implemented diagnostic tools. And since churn is a burning issue, the company is likely implementing several parallel changes (such as pricing and customer loyalty programs), making it impossible to know how much revenue or cost improvement is attributed to the new diagnostic tools. Instead, a measure of change in first-call resolution percentage and MTTR will indicate improvements as a direct result of new diagnostic tools. Then, a correlation view between these measures and reduction in churn can be used to determine financial effectiveness of diagnostic tools and provide clues to further investment in this solution or similar ideas under consideration. Continuous fact-based learning leads to smarter decision-making.
Judging Performance of Leaders in the Aggregate
The management team of any business is selected to be innovative, resourceful, and intelligent about ideas considered and investments committed. The measured outcomes of any single project provide clues for further decision-making. However, the success of the team is ultimately measured at both the top and bottom lines, which reflect an aggregate view of outcomes when all implemented changes meet the surrounding environment of market forces, competitor actions, and economic conditions.
Great leaders will continue to evaluate upcoming change initiatives based on ROI projections, but often assess performance results through a different set of measures most useful to refining effectiveness of the solution delivered. In the end, the aggregate selection of change initiatives delivered will result in a P&L outcome by which the management team is ultimately measured. The best management teams don’t get bogged down in useless mental gymnastics and extrapolations for the sake of producing lengthy dashboards. Instead, these leaders use streamlined, targeted measures to make smarter decisions about new ideas and gain more insightful feedback about ideas they have recently implemented. You can’t always predict the future but you can prepare for it. And that’s why measuring results, while less of a science and more of an art, is so important.