In her paper, "Developing an Effective Metrics Program," Rosenberg described a group of "sample goals, questions and metrics. The goals are general and could be adapted with minor modifications to any project development. Questions are derived to quantify the goals. The metrics needed to provide the answers to the questions are then chosen and shown in italics."
Here is a goal from her paper, with associated questions and metrics:
GOAL: The system must release on time with at least 90% of the errors located and removed
QUESTION: When will 90% of the errors be found?
METRIC: Effort (such as hours worked)
METRIC: Errors (count errors detected)
QUESTION: What is the discrepancy rate of closure?
METRIC: Errors (count errors detected)
METRIC: Closure status of the errors
Use your knowledge of measurement dysfunction to critique this set of questions and metrics. In particular
(a) If you collected these metrics, would they provide you with answers to the questions? Why or why not? What other information, if any, would you need?
(b) If you could answer these questions, could you know whether at least 90% of the errors had been located and removed? Why or why not? What other information, if any, would you need?
(c) If you relied on these metrics, would any aspect of the project be systematically under-managed or mismanaged? Explain your thinking.