MEASURING PERFORMANCE WITHIN A PROJECT

There’s a big difference between measuring the performance of a software development project and measuring performance within a software development project. Software project performance is typically measured in high-maturity organizations. In these organizations, the business processes are documented and audited for efficiency. The development organization has corporate support and funding for formal software quality assurance and process improvement. In high-maturity organizations, the software project as a whole is measured as if it were a complete business process in itself: it’s effective (delivers on time), it’s efficient (meets budget), and it’s profitable (results in margin or profit).

In such organizations, metrics programs are a tool for measuring project capability and compliance. The paradigms for such software metrics programs are the well-established CMMI® level 4 organizations with fully functional organizational process performance and quantitative project management in place. With their substantial historical data and standardized and audited processes, these organizations can make statistically valid inferences from their measures about the project as a whole, identifying deviations and diagnosing process failures.

Much has been written about these formal software metrics programs. Any organization that wants to start an organization-wide, executive-sponsored software metrics program can follow the IEEE standards or the CMMI® model. When it comes to implementation details, dozens of books explain how to integrate measurement across the entire product and project lifecycle or how to use the data to improve your organization. When you need more extensive advice, read the classic texts by Grady, Fenton, Myers, Kan, and others (see Further Reading). They’ll tell you what worked for Hewlett-Packard, the National Aeronautics and Space Administration, and a host of other companies with ample revenue streams and a senior management staff committed to product quality.

The problem is that many software practitioners don’t work for mature, or even maturing, software companies. Some work in situations where the software development organization is one of those “challenged cost centers”—costs too much and needs attention—but none of the executives knows what to do about it. There’s no enlightened senior management to endorse software process improvement initiatives, nor any highly respected and well compensated consultants to guide the program when senior management interest falters because the results are too slow in coming. There’s no Software Quality Assurance or Software Engineering Process Group to bear the brunt of the work of process improvement.

Some software practitioners work in dot-coms or MIS/IT shops, suffocating under the pressure to deliver on impossible schedules without requirements, designs, or even adequate resources. There’s simply no room for documented processes when there are four developers in a one-person cubicle. There’s no time for reviews or audits when just getting the code done will take 12 hours a day, every day, for the next 18 weeks.

You can’t worry about maturity levels when you’re always living on the edge. Project managers, development and test managers, and team leads in situations like these have to find a way around the crisis of the hour. They need software measurement data that they can use in the day-to-day decisions, not in the next quarterly business review. They need to measure their performance against the project schedule, not the project’s compliance with historical norms for Estimate at Completion.

Measuring progress entails far more than checking off a line item in the work breakdown structure. Projects need to be viewed as organisms rather than task lists. In the human body, different types of cells perform different tasks, in different locations and at different times. In projects, different people in different roles work individually to complete tasks that in turn trigger other people to begin or complete their tasks. You can’t diagnose and treat a disease in an organism unless you know all the symptoms and how they interact. Similarly, you can’t diagnose and treat performance problems in a project unless you know what symptoms to look for, where to look for them, and what to do about them.