3
Metrics Are the Means, Not the Ends
Numerical Targets Are Measure-mental

Everything gets measured all the time

Because of the obsession with financial measures that arose in the 1980s, someone was bound to note that it takes more than just managing the financials to run a business. Those someones were Robert Kaplan and David Norton, who published a paper on the balanced scorecard (BSC) in the Harvard Business Review (HBR) in 1992. The premise of the BSC is that four categories of measures are needed to manage a business successfully—financial, customer satisfaction, internal processes, and innovation and learning. Ideally, the measures in these four categories would be determined by the business’s strategic goals, and the scorecard could be used to operationalize a strategic plan. The BSC would include both internal and external objectives for implementing the strategy with numerical targets that would let you know your progress. See figure 2 for an example.

This BSC aligns an organization around creating premium products with the purpose of improving profits, revenues, and market share with those as the designated financial measures with end-of-year targets. The other categories help operationalize the strategy: process goals include the number of premium products in development, percentage of budget devoted to premium products, and defect rate; customer goals include satisfaction rate, retention, and percentage who purchase the premium lines; and learning goals include customer service training hours and R&D training budget. (There are no innovation goals in this example.) All these measures contribute to achieving the premium provider market leader status as the strategy.

However, if the BSC was going to help implement a new strategy, it didn’t go far enough. High-level objectives are fine for the C suite, but how do the workers contribute to market share or training budget goals? The next logical step was called “key performance indicators” (KPIs). Each of the balanced scorecard measures could be broken down into its component measures—for instance, revenue consists of sales volume and price. Each of those measures could be further broken down and then the resulting measures also broken down. For example, total sales is broken down into country sales and then into regional sales; price is broken down into unit cost and profit margin (see figure 3). This process of breaking down metrics into a hierarchy of component measures was sometimes called “cascading” key performance indicators. This way, every level in the organization could contribute to implementing the strategy and achieving the strategic objectives. As a machine operator or customer service representative, I could see how my role contributed to the overall objectives through the use of this hierarchy of submetrics.

We consultants took to this measurement system like remoras to a shark. The problem with most consulting at the time was that the firms did the analysis, developed the strategy or new process, created a recommended implementation plan, and then left for another client. Executing the strategy was left to the client. The process work helped but often didn’t cover all the strategic objectives, and it was up to the client’s management team to enforce both the new processes and the new strategic plan. One doesn’t make much money in consulting by trusting managers to do their jobs well. If they did, they wouldn’t need us. Now we had tools to monitor the implementation and results of our efforts.

With the balanced scorecard, we could break the strategy down into strategic objectives in the four areas and add targets. Combined with new work processes, this system of measures was a surefire method of implementing the strategy. In this way, how each person in the organization contributed to the strategic objectives could be monitored and measured. It provided discipline to the messy business of organizational change. This structure was wonderfully appealing to both consultants and clients alike. The result would be an entire company aligned around specific goals and targets, with everyone able to measure how well she is doing her part—a complete command-and-control system for implementing strategic objectives. Even better, consultants could sell the balanced scorecard and cascading key performance indicators as a stand-alone service.

Pretty much every consulting project I’ve been involved with over the last two decades has had some metrics component. The use of metrics for any kind of project is so widespread that people rarely ever question the value of collecting and monitoring measurements. It’s a given that you need a system of measures to accomplish anything. The following two points are from a PowerPoint presentation I have often used in client engagements that seem to be a mainstay in pretty much every consultant’s repertoire:

You can’t manage what you can’t measure!
A metrics scorecard acts like a car’s dashboard.
Executives monitor progress with large meters and are
notified of problems when small warning lights turn red.

The rise of information technology during this same period made the data collection and reporting needed for this whole measurement and monitoring system much easier to implement. It started with executive dashboards that showed progress toward key metrics. Then IT worked its way into employee performance appraisals with the development of automated performance management systems driven by SMART (specific, measurable, actionable, results-oriented, and time-bound) goals and metrics that linked upward and downward. As executive dashboards became popular, businesses wondered why they should be limited to executives, and with web technologies, any employee or department could have its own dashboard to monitor measures. These web pages look a lot like the control panels used to monitor machines, with red alerts, yellow caution signals, and green icons meaning “on-track.” Now executives and managers could see exactly what was going on without ever having to leave their desks or talk with anyone. They could manage everything with a simple, color-coded web interface. Just like statistical process control, it was a perfect command-and-control system with warnings whenever any of the measures went “off-spec.” What could possibly be wrong with that?

It’s funny how the targets are always met

The problem is that the system is trying to command and control an organization composed of people. And the problem with people is that, well, we’re people. We don’t operate like machinery. In fact, we really don’t like being commanded or controlled, and our reactions to these measurement systems cause our behaviors to change in unpredictable ways. One thing I have learned from these measurement systems is that if you pick a specific goal and attach rewards and punishments to it, you can pretty much guarantee that somehow that goal will be met. Unfortunately, this often comes at the expense of other worthy, but nonmeasured, business goals. The most straightforward example, and the place where numerical targets got their start, is in sales organizations. Today, most companies have moved away from paying their sales representatives straight salaries to offering commission-based compensation, that is, the more a person sells, the more he makes. Typically, a salesperson has quarterly revenue targets to meet to get the incentive compensation. Anyone familiar with sales knows that sales usually pick up at the end of each quarter and then drop off at the beginning. This is because salespeople offer promotions and other incentives, like rebates, to their customers to order before the close of the quarter. Of course, promotions and rebates harm profitability, but most salespeople aren’t measured on profits, so they don’t care.

Sales representatives often game the system to their advantage. Probably the worst example I have ever seen was when a regional manager got fed up with the unachievable revenue targets he was given every year. Not only did he not get a bonus, but all the members of his team were penalized in their compensation. It’s one thing to suffer the penalty yourself, but having to inform the hardworking people on your staff year after year that they didn’t make the cut can be heart wrenching. One year, he persuaded his distributors to buy more product than they wanted so he and his team could meet their year-end sales goals. He promised that they could return what they didn’t sell. He and his team met their targets and got their bonuses, and the company was flooded with returns two quarters later. (He had already planned his resignation.) Besides having to write off most of this product, a huge cost, the company also had to pay for the extra handling and storage and the bad will that resulted from this scheme. Of course, in this manager’s defense, the sales targets he was asked to meet had no grounding in reality but were the result of an executive’s desire for double-digit growth in a stagnant market. Somewhere at the root of this thinking was a consultant’s advice that stretch goals would spark creativity in meeting them. (I used to say this. I’m sorry.) They certainly did!

The games people play in meeting their targets are widely documented in business literature. Here is just a small sampling of examples:

• Perhaps the most famous case is the Sears auto repair scandal. The state of California charged Sears auto centers with fraud after Sears instituted incentive compensation schemes based on sales targets for certain parts and services. The result was that customers had unnecessary repairs done on their cars without their consent or knowledge. Needless to say, the company’s business suffered as a result of the customer fraud.

• Reengineering the Corporation relates an anecdote about the IBM credit department that had reengineered its processes and set performance standards for each step. Although it was getting 100 percent compliance with the targets, the lead time to process a credit application had actually increased. When workers were in danger of missing their volume targets, they returned bids to the senders when they found typos or other errors instead of just correcting the errors themselves.

• Among the examples Jeffrey Pfeffer writes about in an HBR article on pay myths is Highland Superstores. After Highland instituted commission targets for the sales staff, the resulting behavior of the salespeople was so aggressive that it alienated customers.

• Gregg Stocker, in his book Avoiding the Corporate Death Spiral, has identified an obsession with numbers as one of the steps in the spiral, and among the examples he gives are a public transportation authority and a post office. The transportation authority decided to align the pay of bus drivers with their on-time arrival rate. As a result, drivers skipped stops if they were running late, leaving passengers stranded. A post office that instituted targets for mail processing times found numerous bags of undelivered mail hidden away. When workers couldn’t process all the mail within their targets, they just hid the mail.

• In “Paying People to Lie,” Michael Jensen writes about numerous cases of deception and fraud in meeting sales targets, including a software company that was cited by the Securities and Exchange Commission for backdating sales, booking maintenance agreements as software sales, and paying fictitious consulting fees to customers in lieu of giving them product refunds.

• More recently, a federal investigation on fraud in home foreclosures cited that performance measures contributed to “robo-signing,” where foreclosures were processed without being read or having the proper documentation. How many people lost their homes as well as their life savings because bank staff were under pressure to process a target number of foreclosures?

These are the worst-case examples where people resort to deception and fraud to meet targets. Not everyone will stoop so low, but I have noticed that people will often manipulate the measures to meet them. For example, as a frequent flier for most of my life, I noticed that about a decade ago, stated flight durations lengthened. This coincided with industry-wide measures of on time arrival. When I first started flying decades ago, a flight was never early. It was either on time or late. Now it is commonplace for flights to arrive thirty to forty-five minutes early, which initially seems like a good thing. However, if you’ve arranged for a ride to pick you up at the arrival time, you could be waiting around the airport for a while.

In another example, I worked for a corporate department that conducted annual customer satisfaction surveys. For years, we had shown steady improvements. After a while, our service levels plateaued. Even though the satisfaction level was still high, the leadership team was under pressure to show continuous improvement. The answer was to count all the “neither approve nor disapprove” answers on a feedback survey as positive responses, showing a nonexistent improvement over the prior year.

The reason why businesses love measures is because they mistakenly believe that measures are real, hard data. Another misguided management mantra, “The numbers don’t lie,” forgets that people are the ones monitoring, collecting, configuring, and reporting the measures. Measures are not objective. Measures lie all the time, even financial ones. Finance is not a science but a matter of opinion; the rules are only generally accepted accounting principles (the United States uses GAAP versus international financial reporting standards [IFRS]) and can differ from country to country. What goes into a unit cost, or a capital expenditure versus an expense, or what qualifies as an asset can vary from company to company and can be manipulated to paint the desired picture. An extreme example is the financial games Enron played.

For one client, I was asked to improve the company’s manufacturing costs, which had recently ballooned. Nothing at the plant had changed except for the way overhead costs were allocated. Bundling up overhead costs and divvying them up by space occupied or by number of employees is not a true representation of what something costs. Yet most companies use some kind of overhead allocation formula because it is convenient. It takes too much time to figure out which products use the most electricity or what department consumes the most network bandwidth. What something costs is a matter of convention rather than a “hard” number. Revenues are harder to fudge, but when a sale is booked is a matter of convention. This can make a difference in meeting those quarterly targets.

You can play lots of games in accounting and financial reporting, and when you get to the squishier measures, there is even more room for creativity. When does the clock start on new product development cycle time—when the product is an idea or when it has a budget or when a project manager gets assigned? Are classroom supplies included in the training dollars per employee? Does defective product quantity include the pieces that were reworked to meet the specifications? See how much fun you can have? Even better, you can change the definitions every year to show improvement. Include the rework one year and exclude it the next, and, voila, defective parts show a dramatic decline! (But make sure you include a tiny footnote on the chart to document the change in measurement. You don’t want to be accused of fraud!)

In reaction to all this game playing, companies often implement countermeasures. These are additional measurements that act as a balance to the original metric designed to achieve the strategic goal. Let’s revisit the sales function where sales commissions are based on target revenue volumes at the expense of profit margins. The next iteration of the game includes a profitability measure. Now we have salespeople pushing the high-margin products, which seems like a good thing, except they are pushing them instead of other, cheaper products. The result is angry customers who have been bullied into buying pricier items. Annoyed, they start to buy from competitors. Or in reverse, let’s add a customer satisfaction index to the mix instead of profitability. Now the salespeople have no incentive to sell products at a reasonable margin. Selling at or near cost improves both sales volumes and customer satisfaction even though it hurts the company. So the only thing that makes sense is to measure revenues, profitability, and customer service. Now we have increased returns! Customers are motivated to buy with a no-questions-asked-free-returns policy. So add customer returns to the mix. After a few rounds of this game, the result is a laundry list of measures and targets. The focus on and priority of the strategic objectives are gone, and the only work that gets accomplished is the measurements. Worse, long-term goals or work that isn’t covered by an annual measure falls by the wayside. Any incentive to invest in the long-term future of the company is gone.

Measures create conflict where there normally is none

These examples show the game playing and willful manipulation that occur in these measurement systems—a worst-case scenario. Yet even when no game playing or manipulation is involved and the people being measured are compliant—a best-case scenario—these trickle-down measurement systems do the opposite of what is intended. I learned a very interesting lesson about conflicting metrics from a big supply-chain process reengineering project. The client had typical supply-chain problems—lots of inventory, long lead times, many unfulfilled orders, and unhappy customers. A few weeks into the engagement, the flagship plant burned down, severely diminishing the amount of product, throwing everyone into a panic. Rather than continue with the project, some of the consultants were asked to help manage the crisis. During the ensuing weeks, the consultants established a process for contacting customers, negotiating orders and due dates, working with contract suppliers, and shipping to meet delivery dates. As expected, the surplus inventory was eliminated, but additionally, the lead times decreased, orders arrived as planned, customers paid earlier, and both customer satisfaction and profitability improved, albeit at reduced revenues.

The bottom line improved because the margins on the smaller sales revenues increased and the costs of carrying inventory and accounts receivables decreased. These results blew me away—all the improvements without the process reengineering. At the time, I wasn’t sure why, but over the next few years, I continued to work in supply chain and eventually figured out why those large, segmented projects rarely showed the improvements we expected. Here is a typical supply chain, the typical measures, and the consequences of rewarding those measures. I am assuming no game playing occurs.

Sales—Salespeople are responsible for selling the goods and obtaining the customer orders. As I’ve already discussed, most sales functions have quarterly sales quotas, resulting in a big push to meet the quota at the end of quarter when it looks like it won’t be met. Customers who are planning to place orders in the beginning of the next quarter are given rebates or other promotions to place them before quarter end. Besides the peak at the end of the quarter, this causes a trough at the beginning of the next quarter.

Result: Sales representatives are creating artificial spikes and troughs in demand and may be eating into profit margins with promotions. The dreaded supply-chain bullwhip effect of spikes in demand actually starts in-house.

Order entry/customer service—Each order gets processed either by a computer or by a human. Either way, the purpose of this function is to ensure the order information is correct and determine when the order can be filled. Although some companies use processing times as their metrics, most companies have learned the dangers of quantity measures (filling lots of orders quickly but incorrectly) and prefer quality metrics like order accuracy. Accuracy is important because it can be costly to fill the wrong order, process a return, and then fill it again correctly. Plus, incorrect fulfillment tends to annoy customers.

Result: Customer service representatives are incentivized to verify that all the order information is correct, erring on the side of caution and possibly double-checking with the customer or sales rep, which is at odds with minimizing the overall order lead time.

Warehouse—If product inventory is on hand, the warehouse picks the product and stages it for shipping. Inventory management is a delicate balance. Too much inventory means that lots of money is tied up in unsold product that incurs warehousing costs and risks obsolescence from sitting too long on the shelves. Too little inventory could mean stockouts, where customer orders can’t be filled until more product is made, risking the loss of sales. Inventory managers typically determine how much stock to keep on hand by calculating how long it takes to make the product plus how long it takes to process an order (order lead time) and using a fudge factor based on the variability in demand. If product demand does not fluctuate much, they don’t need much safety stock to cover the variation. However, if demand is irregular, they need to store enough product to cover the peaks in demand. If it takes a long time to make the product, they again need to store a lot of inventory. However, more inventory means more carrying costs and more risk of obsolescence—hence, the delicate balance. Their ideal world would be to have stable demand and short order lead times.

Result: Inventory managers are usually penalized more heavily for stockouts than for having too much on hand. They usually err on the safe side and order more inventory to cover the peaks in demand. This means warehouses typically have more product than is actually needed, prompting sales promotions later on.

Manufacturing—Factories are primarily concerned with pumping out enough product to meet demand, maintaining the quality of that product, and keeping their costs low. This means that ideally, plants want to run at or near capacity. Typically, all the plant overhead costs, like utilities and salaries, are factored into the cost of products with the result that the more you make, the lower the costs.

Result: Plant managers are measured on their machine uptime, volume of output, product costs, and quality. They want to make as much product as possible but not necessarily the right products to fill orders.

Distribution—Finally, completed orders are sent to Distribution, where they are loaded on trucks and shipped to their destinations. This function is usually measured on its freight costs. Because of this, distribution managers aim to ship full truckloads, meaning some orders may sit at the dock waiting for other orders going to the same destination. Trying to ship full loads to a variety of destinations in a timely fashion is like a puzzle, weighing the need to be on time with the need to keep costs low.

Result: To keep freight costs low, shipments wait for other shipments to the same destination. This adds to the overall lead time.

This is how the individual functions are typically measured. However, for the entire order fulfillment process, the goals and associated measurements are the number of orders that are delivered on time, the total order fulfillment lead time, and customer satisfaction. All the functional areas should be focused on filling orders as quickly and accurately as possible, yet few are measured on this. Instead, many of these functional areas are working at cross-purposes to each other. To summarize:

• Salespeople are creating artificial spikes and troughs in demand that lead to the need for more inventory to cover the variability. Salespeople hate out-of-stock situations, which means they can’t make that sale so they want lots of stock on hand.

• Plant managers want to produce as much product as possible to keep the cost per unit low. This is at odds with keeping inventory levels low. Their desire to maximize machine uptime is at odds with filling orders as quickly as possible. They also don’t want to bring a machine down to satisfy an order.

• Customer service wants to verify orders to ensure accuracy, but this adds to the order lead time, which means more inventory.

• Distribution wants to ship full truckloads, which also adds to the order lead time and means more inventory.

• The warehouse and anyone responsible for managing costs, like the financial group, don’t want to pay the costs of storing large amounts of inventory that may become obsolete in the future.

By segmenting each function and setting up separate measures, we are motivating people to work at cross-purposes. The workers in each function are trying to maximize their own performance metric at the expense of the others. In the case of the plant fire, everyone in the supply chain became aligned around the goal of filling orders in a timely fashion. Some of the individual metrics, like distribution costs, had to be sacrificed to meet the overall goal of filling customers’ orders. The crisis had inadvertently forced the company to choose which goal was most important and aligned the supply-chain organization around it.

Why are companies instituting contradictory measurements? Aren’t the measurement systems supposed to be built around strategic goals? The first problem is that if everyone was measured only on meeting the delivery commitments, the other nonmeasured parameters wouldn’t be managed. Manufacturing, warehousing, and distribution costs would increase, possibly beyond reason, to meet delivery and customer satisfaction goals. We would end up creating our game-playing scenario. Secondly, it’s not really fair to measure and compensate people on variables beyond their control. Although customer service representatives play a key role in order fulfillment and customer satisfaction, they don’t control how much inventory to store or when to make a product. Therefore, they could be doing a great job, but stockouts could prevent them from meeting their targets. If you are going to tie compensation to the achievement of measures, you have to create measures and targets that are within the functional area’s control, even though they may be at odds with the company’s goals.

But look at what you have ended up creating—measures that conflict with the strategic objectives and the complete loss of priorities. All the measures become equally important; therefore, none are more important than the others—the antithesis of aligning around strategic objectives. Aside from all the game playing and people working at cross-purposes, you’ve also created a data collection, verification, and reporting morass of measurements and targets, all of which is subject to human interpretation and manipulation. None of this work actually adds any value. It doesn’t contribute to new product creation or better sales or improved operations. The net result is the same unprioritized, misaligned environment as before the metrics were instituted, plus lots of non-value-adding work administering the measurement system and minus the ability of the employees to use their own judgment to mitigate conflicting priorities.

Take a goal you want and turn it into something you don’t

Where my consulting colleagues and I got this whole system of cascading measures wrong is in the assumptions about how and why people work. As graduates of the best schools and general overachievers, we know that we are motivated by a job well done. Other people, though? We just assumed that the average worker needs either a carrot or a stick or a combination of the two to work in the company’s interests. The idea that other people could be motivated by achieving a goal or creating value or contributing to a team never occurred to us. Economic theories at the time stated that people behave only in their own self-interests, usually for a monetary gain. However, this can’t be true. Otherwise, charities, Wikipedia, open-source software, community boards, and countless other altruistic endeavors couldn’t exist. What we didn’t realize was that creating the rewards and punishments based on target measures made employees self-interested at the expense of the company’s interests. We motivated employees to meet the targets and nothing but the targets. The mere act of defining individual numerical targets creates a conflict with the greater organizational goals.

Our second flawed assumption, and this likely has its roots in arrogance, too, was that people would be mindfully compliant with their goals and take their punishments without fighting against the system. The expectation was that if an individual goal conflicted with a company goal, the employee would act in the greater good and use good judgment to avoid fraud or other destructive behavior. Executives were shocked when salespeople bullied their customers or bus drivers skipped stops. What happened to good judgment? Being controlled like a piece of machinery doesn’t leave much room for judgment. With specific directives and targets, employees are instructed to achieve their targets without question and without exception, that is, mindlessly. The whole system is designed to remove human judgment. Without the measures, management would have to rely on employees’ judgment to make the right decisions and perform the right work. Inventory managers and shipment clerks would have to balance the costs and benefits of storing more products or delaying shipments to fill a truck.

The times when I’ve been successful at improving supply-chain operations are the times when I got all the people together to negotiate priorities and determine what trade-offs to make. Of course, it helps if they know whether the company’s overarching goal is low cost or high customer service. But given the direction—emphasis is on direction and not directive—humans are usually able to judge what to do. Funny thing, when you remove human judgment from decision making, you get decisions that are not judicious. The point of improving operations is not to remove human judgment from the operations but to improve the human judgment that runs the operations. (Yes, sometimes that judgment needs a lot of improving.)

The important thing to realize about metrics is that they are a means, not an end. Numerical targets have been a disaster because they have supplanted the objectives the company really wants. Measures were supposed to help you manage, not become the way you manage. Tying them into incentive systems with punitive ramifications means the metrics have become the ends. The easiest way to illustrate this point is by using a weight-loss example. Most people can relate to the need to lose weight. Let’s compare the goal “Lose twenty-five pounds in six months” to “Improve my overall health and fitness.” Businesses would choose the time-bound, measurable goal. Yet this goal can lead to all kinds of health issues. To achieve this goal, you could diet, but afterward, you’d likely gain the weight back. If you choose to exercise, then you run the risk of gaining weight by building muscle because muscle weighs more than fat. If you aren’t anywhere near your target weight at five months, you may become tempted to starve yourself. This has the harmful effect of ruining your metabolism, making you more prone to weight gain. Or you may try a more extreme form of exercising, which makes you more prone to injury.

The second goal allows you to use whatever metrics you choose—weight loss, clothing size, body mass index, miles run, weights lifted—to monitor your progress. It doesn’t allow you to cheat your way to the goal by sacrificing your health, which is essentially what many short-term corporate goals do. The second goal is all about long-term lifestyle changes. And the absolutely best part about the goal is that it is not time bound nor achievable. You have to constantly work at it! You will never get there! That’s what continuous improvement is all about.

The biggest irony of this whole measurement madness is that by insisting a lofty, intangible, continuous goal is unattainable and replacing it with a measurable, time-bound goal, you are actually ensuring that you will never achieve the first goal because you have replaced it with a different goal. The balanced scorecard and accompanying array of metrics don’t help achieve an overarching goal; they replace the goal you want with ones that you don’t. A company that wants to create new and innovative consumer electronics products will be advised to state this in measurable terms, like “create x many new and innovative products by year-end.” This scenario is comparable to our weight-loss-versus-healthy-lifestyle goals—these are two completely different ends! In the second case, the most important parts of the goal are the how many and by when. The whole new and innovative part takes a backseat. The likely result will be a slew of new but not really innovative products—exactly the opposite of what the company wants. What bothers me most are all the corporate mission statements that seek to achieve measurable gains in market share or revenues or some other financial goal. Is that really what the management team and shareholders want? Or do they want a vital, healthy company that’s going to be around for a while?

The simplest way to solve this measurement madness is to decouple the metrics from incentive compensation and any other kind of rewards and punishments. This way the goals don’t have to be measurable, and the company can actually pursue the goals it really wants, not the substitute, short-term goals that meet the measurement criteria. Measures can be used to provide insight and improve knowledge, but they shouldn’t be the goals themselves nor should they become the management system. Measures can’t make good decisions. Only people can make good decisions. And the way to help people make good decisions is to ensure that they understand the company’s overall goals and priorities and that they have the tools and knowledge to help them improve their judgment. Measurements can probably help with that, but replacing management with measurement is nothing short of measure-mental.

I have updated the PowerPoint slides that I use when discussing metrics:

People manage to the measures!
Sometimes they even manipulate the measures!

A metrics scorecard acts like a car’s dashboard.
If you watch it instead of the road, you will crash!