TWO WAYS TO USE MEASUREMENT

Some people use software measurement like they use a daily weather forecast. They want to know how to prepare for the day, so an overview is all they need. Knowing the estimated average temperature and whether it will be rainy or sunny, they can make decisions about how to dress for the day and whether they should try to run an errand over lunch. Similarly, from a few department-wide indicators, the Director of Software Development can tell whether her projects will complete successfully or whether she should contact a headhunter on her lunch break.

For this class of measurement users, the details aren’t particularly important. They know that it will be hotter on the streets downtown than it will be in the shaded streets of the suburbs. The temperature isn’t likely to fluctuate far from the forecast. Of course, there’s always the chance that even a sunny day will suddenly turn nasty if certain conditions develop. If the forecast doesn’t show that as a possibility, they’re content to leave the umbrella behind. Similarly, since the earned value across the projects is tracking close to expectations, our Director of Software Development can go out for lunch and not worry about whether the VP will be waiting in her office when she returns.

Others use software measurement as some kind of archeological dig. They examine papyri bearing curious line glyphs and tablets with weather-worn bar carvings, from which they draw lessons from the past. For these folks, measurements are useful because they reveal how people and processes and projects really work. Measures tell stories about how teams succeeded or why they failed. What was life like on the streets of Cubicle City when the earthquake struck, the build crumbled, and the Atlantis Project slid beneath the waves of the Sea of Red Ink?

For this class of measurement users, the details are extremely important. They know that software development projects succeed or fail requirement by requirement, code line by code line, defect report by defect report. Trend lines are all well and good so long as the environmental factors don’t change. If any of them do change, the forecast can become invalid, with a sunny morning quickly turning into a dreadful afternoon. This class of measurement users knows that if you have detailed measurement data that shows how different types of events affect people, products, and schedules, then you can improve the durability and reliability of your forecasts and plans.

These two perspectives are not incompatible. In fact, both are necessary for a successful measurement program. Unless senior management can derive some operational and strategic benefit from the indicators you put on their Quarterly Business Review Dashboard, they aren’t likely to give your efforts much financial or logistical support. On the other hand, unless you’ve demonstrated that you can manage the chaos of your day-to-day tasks, you won’t be around to see them write the check.

Nevertheless, that’s no excuse to start counting everything in sight. Improperly conceived, a measurement effort can turn into an expensive exercise in arithmetic that causes more problems than it solves. Development teams wouldn’t think of starting to code without first doing some kind of design work. A measurement effort deserves the same care and preparation.