White Paper: CM: THE NEXT GENERATION of Change Control

CM is about managing configurations. This means identifying the configuration and managing changes to it. Managing changes to the configuration is the domain of Change Control. A good change control process and toolset will help ensure that only the appropriate changes are being made to a product, will allow the team to focus on change management rather than configuration management, and will give the team easy access to all configuration and change data, including full traceability information.

Tool support for change control has evolved quite a bit over the decades. Initially it started as the ability to have an exclusive lock on a file when you checked it out - this was the "state-of-the-art" as the '60s moved into the '70s. The concept of change packages, or updates, introduced progressively through the '70s, '80s and '90s, were instrumental in moving change control along. Of course, we've come a long way since then. Change control must begin far earlier in the cycle than code changes.

We start with managing product requirements, from initial creation through, in some cases, nearly constant, evolution. At some point a baseline is sent of to the development team, where the requirements are allocated into design requirements, or design activities/tasks. Again we have a need for change control as the design evolves. This moves us into implementation where the control of code is also important. We use change control for documents, for test cases, and more.

Change control deals with two main areas: change authorization, and change tracking.

Change Authorization
Given one state of a product, typically identified by a baseline, the goal is to evolve the product to its next logical state. At a macro level, we're dealing with moving a product from one release to another. Sometimes we're dealing with just getting it from one milestone to the next, as is often the case in more agile shops.

The question is, what changes are going to go into the next iteration of the product? At a product requirements level, there is the customer, on the one hand, and the product manager, on the other. They negotiate a contract for what is going into the next release. In the case of a mass markert product, the "Customer" is really the market itself. The "Contract", in an agile environment, is more of a "next iteration" agreement based on customer feedback and the overall product release goals. In a more traditional setting, the contract identifies all of the changes going into the product, along with the budget and timelines. Inevitably this contract is essentially a "Release Contents" document along with a releasistic time line and cost estimate. But the Release Contents are surely to change, for most complex products, before the release timeframe is reached. This may be due to Agile feedback, it may be due to changing technology, or it may be due to competitive pressures.

Both the original Contract, and changes to the Contract, need to be authorized. Authorization means that the Customer has agreed to them and the Contractor, or Business developing the product, has agreed to them. It is here that change control is critical. Changes must be organized and recorded in such a way that they can be individually, or collectively, authorized. Individual authorization allows for a more agile approach. It also makes it easier to target specific expertise to the change authorization process.

Authorization between the Customer and Contractor is done in many different ways. Traditionally, it is a physical document being signed. But newer tools allow the customer to raise requests and/or requirements directly into a contractor repository and to track these through to electronic authorization.

Once the requirement is authorized, or more typically, when an entire requirements baseline is authorized, the requirements are allocated to design requirements/tasks. Additional design tasks result from non-conformances (i.e. bugs or deficiencies) oustanding within the product. These tasks then have to be approved by the design managers and/or design/authorities. Normally this process is accomplished through a process of prioritization and assignment. In a traditional environment, this results in a project schedule. In an agile environment, priorities are used to drive the development, and possibly assignments as well.

Assignment is typically the trigger that indicates approval of the design task. In some cases, assignment is simply to the design manager, rather than to an specific individual, and a second level assignment by the design manager is made to the specific developer. Within an organization, it is good practice to use "to-do" lists to indicate approval. A good CM/ALM tool will show a to-do list of assigned tasks, both for the manager and the developer. And this list should be used to launch the creation of source code changes, to ensure traceability is captured.

When design tasks are assigned, there is often another level of authorization, especially in shops that have tight control over file ownership. This authorization gives the right to change particular files, especially when the file is outside of the ownership group.

In at least a couple of tools, I've seen the checkout operation performed by the source file owner into the change package or update that is the container for the change. This checkout operation is the one that is controlled by ownership, while the checkin operation does not need subsequent approval by the owner, although the owner is typically involved in the code review process.

The next level of approval for a change comes with its acceptance into a build. Often this is a blanket level approval, implicit in the fact that the change received approval for implementation. However, in many cases, especially when build stability is an issue, or when approaching the release date, the system build and integration team want to be much more cautious in accepting changes into the build. In effect, this is nothing more than change promotion, once the change definition has been completed. However, the promotion is still on a change by change basis. Where possible, it is a good practice to allow changes to flow through the system in the same order as they are generated, dealing with exceptions on an as-needed basis.

Although approvals will continue up the chain as builds move through to, eventually, production, beyond this point, it is the build increment, rather than changes, that are being approved.

So in summary, change approval deals with:

Change Tracking
At each of the approval stages it is important that tools are available to register approvals and to gate the approvals to the proper player for the next step in the process. A tool supporting inboxes, or to-do lists, will help the process flow more smoothly and reliably.

Once changes are approved, there is still another side of change control to be done - that of tracking of the changes. Changes are made using various tools, resulting in modified artifacts which are tracked in the CM/ALM tool. It is critical that the modified artifacts are linked directly to a change package (i.e. an update) through which they can be tracked. This applies to source code changes as well as changes to requirements.

There is not a one-to-one correspondence between a modified artifact and a change approval. In fact, there can be one-to-many, many-to-one or many-to-many relationships between authorized changes and the modified artifacts. In many shops, most authorized changes involve changing multiple files. So at a minimum, a one-to-many relationship must be supported. Although the flexibility to address multiple approvals in a single update (i.e. change package) may be desireable (e.g. fix two problems with one checkout), a more granular level of traceability results if an update addresses no more than one approval.

In some tools and processes, the modified artifact is tied to the approval object (e.g. approved design task or problem report). Although this provides some level of traceability, it does not do an effective job of supporting change control. Quite often it takes multiple updates to address an issue. For example, the first attempt at a fix may resolve 80% of the problem with near-zero risk, while the remaining 20% can be addressed in a subsequent, higher-risk update. It is often desireable to split the risk across builds, or even across releases. This will depend on the delivery pressures and risk margins. It is a much better practice to trace (i.e. link) each update to the approval object, and allow each update to track the changes to one or more artifact.

In other words, the update/change package itself should be a first order object separate from the modified artifacts and separate from approval object. In many cases it's worthwhile implementing an approved feature over a series of updates so that compatibility and contention issues can be worked out early, while the bulk of the feature can subsequently be completed in one or multiple updates.

It is also good practice to separate work performed by different developers for a single feature into separate updates. This allows for better traceability of problems to a developer, allows for easier peer reviews of the code, and help force good software engineering discipline into an environment.

Because updates link artifacts to approvals, they are a central component of change tracking. They form the hub through which traceability queries must pass. Furthermore, as updates are the atomic component for incrementing a build, the CM tool must enforce that only entire and completed updates flow through the system. The CM tool must also track dependencies between updates. These can be explicitly expressed through a dependencies attribute, but the CM tool must also infer dependencies based on the order of updates made to a file. For example, an update which creates revision 1.5 of a file, is naturally dependant on the updates which created revisions 1.1 through 1.4. Any attempt to promote the 1.5 update must ensure that the dependant updates have also been promoted. Of course, because each update may result in multiple files being modified, there is a resulting network of dependencies among updates. If the CM tool tracks such dependencies but does not actively detect and notify users or them, the change control process will prove to be less effective, and will have to rely on continual verification by an audit team.

Once we have a first order change packaging mechanism in place, change control can proceed on a change-by-change basis, rather than on a file-by-file basis. What this means is that the following operations, for example, may be performed as a single Change-based operation, rather than as a series of File-based operations with the risk of one being omitted:

Tool Considerations
Third generation CM/ALM tools minimize the amount of tedious configuration management that is performed, automating most of this. Instead, the tools focus on support of change management. An effective process integrated with an advanced toolset will permit the entire team to focus on creating and managing changes, rather than the configurations.

The CM/ALM tool will be most effective if the processes are based on the state of the updates rather than on the state of file revisions associated with them. That is, promotion of an update, or roll-back of an update, should normally involve only a change in the state of the update. The need for branching and merging to support promotion and roll-backs will depend both on branching strategies and on the ability of the process and tools to support flow through of updates, for the most part, in the same order as they are checked in.

It is recommended that each update be specific to a single developer, to a single product, and to a single development stream (i.e. target release). Tools should allow updates for a particular product stream to be considered as a collection, separate from other streams or other products. This simplifies the task of the CM team, and helps to automate functions such as a nightly build, automated baseline generation, and context view specification.

Tools which have grown up as file-based tools will necessarily be more clunky in managing change-based operations. In any event, it is ideal if change management and artifact management are both done using a single repository, and using a single integrated user interface. This is especially the case when we consider that there are requirements, requests, tasks, problem reports, updates/change packages, files, builds and several other objects of concern. To spread these out over multiple repositories or user interfaces necessarily reduces the level of integration, and the ability to provide useful traceability navigation. You will notice a stark difference between a tool which was designed with both change management and artifact management in mind versus one that manages artifacts with a separate tool for change management.

Effective change tracking will normally result if there are the following key components:

1. A good change control process
2. An effective way to capture traceability data based on user actions
3. An effective way to query and navigate traceability data

Ideally, the tools will allow change review boards, and other important meetings to use live data for the most timely information. They will also supply the guidance necessary to execute the process.

Where Do You Stand
How good is your CM process from a change control perspective? Is it clear, intuitive and easy to follow? If you're having difficulties with your current CM process here's a few things to check:

Change is the heart of software development. The "soft" in software comes from the fact that software is easy to change. But if your processes or tools are burying this ease, it's time to change!

Joe Farah is the President and CEO of Neuma Technology . Prior to co-founding Neuma in 1990 and directing the development of CM+, Joe was Director of Software Architecture and Technology at Mitel, and in the 1970s a Development Manager at Nortel (Bell-Northern Research) where he developed the Program Library System (PLS) still heavily in use by Nortel's largest projects. A software developer since the late 1960s, Joe holds a B.A.Sc. degree in Engineering Science from the University of Toronto. You can contact Joe by email at farah@neuma.com

More white papers:

Find out more ...

Neuma White Paper:

CM: THE NEXT GENERATION of Change Control