More white papers:

• Going Beyond CM - A CM+ Overview
• Integrated Problem Tracking
• Change Package-based versus Task-based Change Control
• Taking the Complexity Out of Release Management (.pdf)

Find out more ...

• CM+ provides an integrated suite of Enterprise SCM applications.

Neuma White Paper:

Integrated Problem Tracking

1. Introduction

The CM+ product provides a full management environment for designers, managers, and other staff involved in a development project. Problem Tracking is one of the applications within the CM+. The CM+ product also includes tools for Project Management, Activity Tracking, Document Management, Requirements Traceability, Change Control, Build Management and Configuration Management. CM+ provides a major step forward in running the processes required for any project attempting to meet today's demanding quality standards.

2. Why Track Problems?

Problem tracking is the process of handling problems reported against your product hardware, software, processes or documentation from first identification through to the problem closure. CM+ automates this process so that:

• You won't miss fixing a problem because all problem reports are permanently recorded in the STS™ repository.
• You won't waste time trying to solve problems that someone else has already solved.

You can record workarounds for problems that won't be fixed right away, so you can still alleviate your customers' difficulties and so that support staff are aware of workarounds.

• You always know the current status of each problem and who is responsible for moving the problem to the next state.
• You can measure the quality of your products and processes objectively, so that the program manager will know when the product is ready to release.
• You can identify and categorize the root causes of problems so that you can improve the development process to remove the causes.
• From the volume of problems and the average time taken to fix problems in your shop, you can get accurate resource requirements for planning future releases and products.

Problem tracking is an important communication mechanism which can replace much of the electronic mail and the many phone calls and meetings generally required for problem status reporting. Through problem tracking and reporting, both internal and external customers can be kept aware of the status of problems they have originated.

In addition to the benefits of recording, tracking and assigning problems, CM+ has comprehensive statistics reporting functions.

• Tables: You can generate tables of problem statistics, in text or GUI format. As an example, you may want to look at a table of problem status vs. problem priority. When you use a GUI table, you can zoom in to show the problem records underlying the summary number (record count, effort total or otherwise) in each table cell.
• Graphs: All tabular summaries are also available as graphs. Two graphs commonly accessed through buttons in the STS™ GUI are the arrival and fix rate graphs.
• Gantt: You can use gantt charts to graphically display problem history. A common operation is to display, using the gantt charts, the life-cycle of emergency problems.
• Ad-hoc reports and queries: Create browse panels and/or listings of problems meeting criteria you specify in varying levels of detail. Or integrate with a word processor

Neuma's Fundamental Rules of Problem Tracking

Neuma recommends that the following rules be enforced to help ensure successful problem management.

• All problems must be tracked in the system. "A problem is not a problem unless it has been entered into the problem tracking system." There must be a single, reliable source for all problem data.
• Universal access to problem data is just as important as universal access to problem origination. The entire project team must have access to all problem information.
• Screening of problems will always be necessary to ensure proper routing of the problem report and proper interpretation of the data fields. It is recommended that a Problem Review Board be established to help review problem data. This may be a single person in a small project, or may involve representatives from several areas of a large project. The role of a Problem Review Board becomes most critical when new releases are in field trial or are being shipped to the initial set of customers.
• Problem tracking must be integrated with other parts of the development process to maximize its benefits and to help ensure the timeliness and accuracy of the data. A clear flow of responsibility must exist for each problem from the point of its origination until it is closed, that is, accepted by the originator as being resolved.
• The user interface must be configured so that designers, testers and managers alike may look at standard report summaries with the push of a button.

3. The Problem Tracking Process

The key to successful problem tracking is a well-defined process. Problem tracking involves the following major areas for which detailed process documentation is highly recommended. These are covered briefly in the following pages:

• Problem reports collection
• Problem classification
• Problem awareness
• Problem resolution
• Problem feedback
• Quality assessment

4. Problem Reports Collection

Problem reports will come from many sources. Some will come from the product and process development teams. Some will come from internal or external verification teams. Some will come from customers. It is essential that everyone who encounters problems with your products or processes have adequate means of reporting problems. The problem collection mechanism must be such that problems do not fall through the cracks. It must also allow for special procedures for identifying urgent problems.

5. Problem Classification

Problems need to be classified in several ways and perhaps in several stages. Classification is generally done by assigning values to various attributes of a problem report. Typical classifications would include:

• Identification of product versus customer (e.g. configuration) problems.
• Identification of area of responsibility, by product, owner or department.
• Prioritization of the problem.
• Identification of platforms, product versions, etc.

6. Problem Awareness

It is essential that the development team is aware of outstanding problems and their urgency so that it can address problems in proper order. This involves ensuring there are proper resources to handle all problem reports. The reporting feature of problem tracking allows both high-level and detailed reports, selected and sorted based on the attributes of the problems. Reports may appear as text listings, numeric summaries, graphical summaries or interactive query displays (using the graphical user interface).

The key properties of problem reporting are that it be fast, flexible, easy and interactive. Outside of these boundaries, managers tend to delegate report production with the result that it has none of these qualities. Neuma's philosophy ensures that managers can have ready access to exactly the data they need, when they need it, even during a meeting. It is recommended that each manager and designer have a single button that may be selected to identify unresolved problems for which the person has the responsibility.

7. Problem Resolution

Problems must be resolved as they arise. Often this will mean a simple customer configuration change. Sometimes, the problem will be one in which the product needs, for example, a software or documentation change. At other times, the resolution will take the form of a workaround.

The keys to problem resolution are:

• assigning responsibility for the problem
• ensuring an easy way to reassign "responsibility" and to track such changes
• establishing an accurate priority
• having an effective problem resolution team

Typically, design team members will only look at problems assigned to them. Then they will address the problems, in order of priority.

When a designer fixes a problem, CM+ will automatically mark the problem fixed, supplying a timestamp and traceability to the update by which it has been resolved.

8. Problem Feedback

An important part of problem tracking is ensuring that those who originate problems receive notification when the problems are fixed. Typically, this is done as part of a release and is included with the release notes. With CM+, the production of these release notes can be automated. Problem feedback is also required in other forms. For example, you can configure CM+ so that high or emergency problem reports are sent by email to the originator (including customers) when the fix is ready to be installed or even when the problem cause and/or workaround has been identified. Internal verification teams require timely information so that the flawed functionality can be re-tested before the product is released.

CM+ serves as a communications medium for problem reports for all users. Report generation and electronic mail may be used to facilitate the feedback process for staff who, for any reason, are not CM+ users.

9. Quality Assessment

Problem tracking allows a team to measure the quality of its product and processes. This is often in the form of "number of defects per interval of time". CM+ can produce such reports as problem arrival and fix rates for a product or product area. CM+ reporting summaries may be used to identify the number of problems against various other attributes, and the identification of quality for the various parts of the architecture or for the various releases.

Typically, information which helps to identify the root cause of a problem is associated with each problem so that the development process itself may be improved by analysis of the problems, with a result of fewer problems occurring. For example, analysis may reveal that more problems occur when certain areas of the software are modified or when software is modified using certain tools. Correction of the design or of the tool set could help reduce such problems.

10. Defining the Problem Tracking Process in Neuma PT

Neuma knows that every company has its own terminology and its own problem handling process, so CM+ has been designed to be completely configurable, from the set of attributes that need to be tracked, right down to the names of the states and the priorities. You can also tailor the input screens to include the fields necessary for your project or for particular users.

You can also customize the graphical user interface to include the menus and buttons you need to automate your problem tracking process. You can even capture and enforce your process, allowing you to work productively with your QA department. You can:

• choose your state names and allowable transitions
• display your process flow chart on your screen or printer
• decide whether or not to enforce the process

The key to a successful problem tracking application is a well-defined process. The problem tracking process indicates how a problem is to move through the development team. It also identifies the attributes of a problem report in a clear, unambiguous manner. For example, is a problem's priority an indication of a customer's view or of a project view of the urgency.

The overall process is indicated by a status flow diagram which indicates the various states of a problem report, the responsibility at each state, and the actions which cause transitions from one state to another. The process defined in this section reflect Neuma's recommendations for a typical software development project. This may be modified as necessary for the particular needs of your own project.

11. Problem Status

The accompanying status flow diagram, generated by Neuma PT based on the process configuration, indicates a typical flow for problem reports in a software project. The diagram reflects the following problem states:

orig: The problem has been originated. Either the problem is automatically assigned to an owner on origination, or this assignment is done separately by a problem control team.

open: Work has begun on fixing the problem. This transition often occurs automatically when the problem number is referenced by a software update which is to fix the problem.

reopen: A problem which was previously thought to have been successfully addressed (fixed) was found to be persistent or incorrectly resolved. Often, there is no distinction made between "open" and "reopen" in the subsequent handling of the problem.

answer: An answer has been provided which is intended as the final response by the owner of the problem.

submit: The resolution was provided through an update to the product. The update has been submitted to the system. This state typically occurs when a software update is submitted. The update identifier is usually appended to the problem report description.

sitest: The problem has been integrated into a system build. Depending on specific process methodology, this status may or may not indicate that the integration team has verified the problem fix. It generally should. In any event, the problem is ready for verification testing.

svtest: The verification testing has passed the problem successfully. There should be a specific test case written to address each problem and the test case identifier should be specified when the tester marks the problem as verified.

closed: The originator has verified that the problem has been addressed as initially expected. Often this is done by an agent of the originator, such as the customer representative.

Additional states often included in a problem flow process include:

accept: This state indicates that the owner has accepted the problem as a valid problem and accepts responsibility to fix it. If a problem is not accepted, a reply should be given and the problem status moved to the "answer" state.

defer: If a problem will remain unresolved for an indefinite period of time, it should be moved to the "defer" state. In this state, the owner indicates that no action is planned.

Within CM+ you may configure your problem tracking process to include any of these state or your own project specific states.

12. Problem Priority

You can also configure CM+ to reflect your company's problem prioritization scheme. You may have two or more separate processes to handle problems of different priority. For example, an "emergency" problem may need a streamlined, efficient process, and a problem which is really a "feature request" may be better handled by the activity planning process.

Problem priority is often used to reflect a combination of factors including the customer's view, the potential damage that may result, and the required response time. Neuma recommends that priority be tracked in one of two ways.

• The priority simply reflects the time by which a response is required.
• There is a second priority field which captures the original (originator's) priority so that this information is never lost.

Note that the priority assigned when the problem is found and entered is often very subjective and inaccurate. Neuma recommends a symbolic assignment of priority with the following meanings:

inv: The priority is not yet known. The problem must be investigated to determine if there are easy workarounds.

req: The "problem" is a feature enhancement request. It will continue to be tracked until the request is tracked as part of a requirements specification.

low: The problem is of low priority. There will be no specific planning as to when it will be resolved until the design team is ready to address the problem. This typically identifies a problem which rarely occurs, has minimal impact, or has a straightforward, unobtrusive workaround.

med: A medium priority problem is one requiring resolution by the next major release of the product. Typically, some important functionality is affected.

hi: The problem must be resolved by the next release of the product. On occasion, specific sites may need some help in dealing with the problem during the interim period.

emer: The problem needs immediate attention. Typically, a one to three day response is required, with the response time dependent upon the severity of the problem. An emergency problem will cause some vital processes to come to a stand-still or to be severely limited.

13. Feature and Test Case References

Each problem report is a description of a defect in a product or process. A defect is a deviation from the specification for the product or process. Ideally, each problem report should refer to the part of the specification which has not been met. In practice, specifications are often loose or don't address many of the potential problem areas. In a well integrated environment, problem reports can be used to help improve the specification. The specification would normally be a decomposition of high level specifications into a detailed set of features. These should be formally tracked with test cases written for each feature.

14. Communication and Configurability

In a traditional development environment, a great deal of emphasis is placed on the production and distribution of problem reports and on the notification (e.g. by email) of the origination of new problems. In CM+, the system itself becomes the communication medium and obviates much of the need for reporting production and notification processes. At the same time, rule and trigger design may be used to configure just the right level of notification for your project.

Typically, problem reporting is set up so that the user can access the reports most often required by pushing a button or by filling in a dialog panel. The command structure underlying the GUI allows complete tailoring of the GUI menu system to allow very specific or very general reports at the click of a button or through customized dialog panels. It also allows for a dynamic report production. Understanding the basics of the "show", "table", "graph" and "gantt" commands and of the "sort" and selection operators allows the user to produce reports using a single command.

15. Customer Tracking

You may need to supplement your problem tracking functionality with customer tracking. CM+ may be easily extended to include a customer and customer request tracking application.

16. A Client/Server Transaction-based system

CM+ is a client/server transaction-based system. This means that it runs as a client/server system in which the client sends transactions to the server. The server then updates the central repository and the changes or additions become available to all other CM+ clients.

A transaction is a package of data update requests resulting from command or GUI operation. When you "commit" a transaction, you send this package to the server. If the server is temporarily unavailable, your transactions are queued and processed when the server becomes available again.

Before you commit your transaction, you can cancel it at any time, with no effect on the data at the server. Until you commit your transaction, the server has no knowledge of it. CM+ will not allow you to exit without warning you if you have a transaction in progress.

With the CM+ transaction-based client/server system:

• all changes to the system are registered as transactions, for full traceability and automatic recovery in case of a system failure;
• the client session normally operates on a workstation different than the one on which the repository and CM+ server reside; and
• there may be several clients concurrently updating the repository through the server.

Information is exchanged between the client and the server in order to complete a transaction. Information does not have to be exchanged between the client and the server for query operations. In this respect, the client is considered a "smart" client which may operate even while the CM+ server is temporarily disabled. This has the effect of distributing the query load so that hundreds of clients may operate off a single server.

17. Summary

CM+ provides effective and affordable problem tracking support to improve your processes and your communications. It plays a key role in meeting all of your problem tracking needs. In addition, the problem tracking application is supported over the internet using any Web browser, improving access and availability.