This metric is appropriate for any city, state, county, or federal government organization that is tasked with providing some sort of outcome for taxpayers or other stakeholders. Some government organizations are easier to measure than others. The IRS has outcomes that are measured in dollars every month. An agency like the Department of Energy may have a tougher challenge coming up with outcomes, but this department was originally formed to eliminate the United States’ dependence on external sources of energy. I don’t think that outcome has been achieved and I think the trend is going in the wrong direction. The Energy Department’s current mission statement is much vaguer and harder to tell whether it has been achieved:
The mission of the Energy Department is to ensure America’s security and prosperity by addressing its energy, environmental and nuclear challenges through transformative science and technology solutions.
It would be tough to come up with some good outcome metrics for prosperity and security, but one of them might be how much of income the average American spends on gasoline, electricity, and natural gas.
Not only do government organizations need to measure outcomes, but so do organizations that sell to government, like defense companies, or are paid by the government, such as health care providers, as well as schools and government programs. Even profit-making manufacturers and service companies might measure outcomes if the product is designed to save people time or money, or somehow improve the quality of their lives.
What is scary is that a lack of demonstrable outcomes sometimes does not impact performance in government organizations. If an organization fails to achieve its mission, it continues to exist, get funded, and maybe even get more money. Luckily, there are some rewards for organizations that do achieve positive outcomes. U.S. News and World Report and other organizations rank hospitals based on key outcome metrics like mortality, and this seems to be having an impact. More people check into the good hospitals, so their financial health improves, they can hire better staff and buy better equipment, and health care improves even more. Two recent examples are Henry Ford Hospital in Michigan, which recently won the Baldrige Award, and Southcentral Foundation, a community health care provider in Anchorage, Alaska, that also won a Baldrige Award. Both organizations not only demonstrated improved health outcomes, but also lower costs. Demonstrating improved outcomes might also help ensure that you get your share of the budget when things get tight and make you much less of a target when leaders are looking for programs and places to cut. Most leaders are loath to cut programs that are clearly producing measurable outcomes, particularly those that are important to politicians and taxpayers. So one of the reasons to have a good outcome index is that you demonstrate a genuine desire to improve your performance. A secondary reason may be side benefits like recognition and rankings, awards, ability to hire better staff, avoidance of budget cuts, and promotions for managers and others.
Another important way this affects performance is that the data allows you to detect processes, programs, and initiatives that do not produce outcomes or cost more than the value they produce. Outcomes always need to be compared with costs. Some countries have much better services for their citizens than we have in the United States. People also pay much more in taxes. If you think it is bad here, go to Holland or some other European countries like Denmark. Being able to demonstrate improved outcomes at lower costs is really what gets attention and recognition.
The cost to develop and implement a system for tracking outcomes ranges from low to high, depending on the type of organization you’re in. If you are a health care provider, I am sure you already track key outcomes like successful surgeries, mortality, patients discharged, infections, and other factors. If you are in education, you have a lot of work ahead of you. School systems track test scores of academic achievement, but this is not really an outcome metric. An outcome metric is something like the percentage of graduates who get accepted to college and what colleges they get accepted to, or the percentage of graduates who are able to find a decent job. Our local Manhattan Beach school system is one of the best in the state and tracks many of these key outcome metrics. Harvard Business School tracks what percentage of its MBAs are running big corporations or holding high-level government jobs.
Where the cost will come is in collecting the data rather than designing the metrics. Getting access to data that may be in other databases is sometimes a challenge. If the Just Say No to Drugs program wanted to really track their effectiveness they could keep track of how many people are arrested for possession of drugs, how many are in rehab, how many are in jail, how many dealers are arrested, and how much pot is being dispensed in states like Colorado and Oregon and California. In other words, getting data on drug use would require accessing many different databases, which would take lots of effort and probably cost a bit of money.
An example of an outcome index for a community development department in county or city government might be:
Jobs | 40% |
Number of new jobs |
10% |
Number of people employed |
20% |
Promotions |
5% |
Turnover |
5% |
Economic development | 40% |
Number of new businesses started |
15% |
Slums and blight eliminated |
15% (square footage) |
Turnover of new businesses |
10% |
Youth | 20% |
Percent graduating high school |
10% |
Percent completing vocational training |
10% |
The exact formula that you use will be up to you, and there is no right answer. However, what you don’t want to do is take 50 to 100 metrics and roll them all into an index. Data should be stacked in a pyramid with the outcome index being the peak of the pyramid that breaks down into three to six dimensions, which break down into four to six metrics, which might break down into four to six submetrics.
One variation is just to have a couple of outputs and not create an index or analytic metric comprising lots of different measures. This might work if you are a simple organization with a single product or service. An airline, for example, is a pretty simple business, and it might just measure the percentage of passengers who are transported on time to their desired destination. One outcome metric certainly makes it simple. However, an airline has other outputs like bags and cargo delivered on schedule. Cargo and bags undamaged is another important outcome. Most organizations are sufficiently complex that they have multiple outcomes for multiple services and products, and each needs to be measured and combined so that management does not have to review 50 charts to see how the organization is performing on its core mission.
Another variation is to combine process, output, and outcome metrics into a single index. For example, a hospital or medical clinic might take all of the 75 or so HEDIS metrics and combine them into a single index. Most of the HEDIS metrics are input or process metrics. I think this is indeed possible, but it is not recommended. It is better to keep process and outcome metrics separate, even though there should be a link. Washing hands after each patient is probably a good process measure for preventing infections, but infections are an outcome that should be tracked separately. It is possible to be diligent about hand washing and still spread infections by other means, like those ties doctors still wear.
Outcome metrics are probably the easiest to get averages, benchmarks, and all sorts of comparative data. Regulators often track performance of various organizations, as do professional associations and trade groups. Each individual outcome metric needs to have a unique target set based upon customer and stakeholder requirements, industry averages, your own capacity and history, resource constraints, and benchmarks. As with most composite indices, “green” is usually defined as 80–100 percent compliant or greater. This may vary greatly depending on the industry and metric. A rating of 80 percent safe landings or 80 percent successful surgeries would probably not be good standards. With many outcome metrics we are looking for Six Sigma (less than three defects per million), or very close to perfection. On the other hand, a school system that had 80 percent of its graduates go on to college would probably be doing a great job.
There are huge benefits to having one aggregate index that can be viewed every day that tells leaders how the organization is performing on its key outcomes besides the financial ones. Having one overall metric also minimizes the chances of leaders micromanaging or obsessing about minor outcomes rather than focusing on the big ones. A CEO I worked with obsessed about getting ranked in the bottom 25 percent of a survey of customers, when those customers only represented about 10 percent of his business. Staking the outcome data in layers by importance and dimension allows leaders to drill down as necessary into the detail when there are problems but most of the time stay focused on the 30,000-foot view of the organization. Having good outcome data like this can also be helpful when it comes time to present to your board, donors, finders, the community, and other key stakeholders. Being able to demonstrate the value you provide to society, the taxpayer, or whoever your customers and stakeholders are is a huge advantage that many of your peers may not have.