A STRATEGY FOR ADDRESSING PRODUCT QUALITY

Suppose you decide to start your measurement effort by measuring defects because there are a lot of them and they take time that’s not included in the project plan. This is really two issues: (1) defect rates, and (2) estimating the effort required to correct defects. Better information related to the first issue will help resolve the second issue, so focus first on defects rather than estimates of time to fix.

Before you can glean any useful information from defect rates, you have to know what kinds of defects affect your projects. Ask yourself the following questions:

Do you know where defects are introduced in the development process?

Do you know where most of your defects are first discovered?

What kinds of defects occur (e.g., functional, usability, data integrity)?

What kind of defect appears most often?

How do you decide whether a defect has to be fixed right away or not?

If your bug tracking system categorizes defects by severity and priority, are those categories really helpful in determining the nature of the defect and its impact both on the release and on the customer?

What kind of testing generates the most defects?

Is this testing related to validating customer requirements or to proving the product works on specific environments, or is it another kind of testing?

You probably won’t have answers to all these questions immediately. In fact, you may have to use the measurement effort to provide you with some of the answers. The point to remember is that defects don’t just happen; they are caused. Once you know the conditions under which they appear, you can start managing the project environment to limit their effect on the project.

Using your answers to the questions above, set up a defect profile:

1. Identify the characteristics of the defects.

2. Group the defects according to some classification scheme.

3. Identify the root cause(s) for each class of defect.

4. Determine if different classes of defect typically have different impacts on the project and the customer.

Now that you have the profile, you can look for appropriate measurement points:

5. Identify the typical project milestones or phases of development and test efforts.

6. Make a list of the questions you would like answered at these measurement points. Don’t worry about the data itself yet. The goal of this step is to identify as best you can the information you want to extract from the data.

Once you have set up your problem tracking system (discussed in detail in Chapter 4), this approach is fairly easy to implement. It will give you some insight into where your own activities may be producing defects or not catching them. It will also give you visibility into where other organizations may be contributing to the defect rate. Only indirectly will it improve your productivity or streamline your workflow, because it is focused on obstacles rather than opportunities for efficiency. For that information, you’ll need to implement another kind of strategy.