More white papers:
Find out more ...
Neuma White Paper:
CM: THE NEXT GENERATION of Agile CM
The two biggest gains of an
Agile development process, in my opinion, are the continuous
integration, which when properly administered leads to greater product
quality, and the interactive customer feedback loop, which helps to get
the requirements right before development is over. These are huge
advantages. There are many agile design processes. But how do you
extend the agile advantages to the CM function?
view of Agile CM will not lead to Agile CM. Why? Because, for the most
part, with noteable exceptions, tool makers worry about their own
little world, and when all of the little worlds that make up the ALM
function are put together, they fall short. If you want Agile CM,
start off with a good, agile process. If your CM tools can't support
the process, look around for more flexible ones.
Achieving an Agile CM Process
is an Agile CM process? If you said it's one that doesn't get in the
way, you're only partially correct. CM adds a critical capability to a
project. This capability is even more critical in an Agile
environment. CM helps the agile process keep rolling. But it also
provides the information required to make decisions with good judgement
and timing. An Agile process is about good communication. Your CM
tools should be your Central Communication Point. All project-specific
information should reside there. I should have to go to people for
judgement and for skills, but not for information.
An Agile CM
Process is one that supports all team members, making them more
productive and simplifying their management tasks. If your developers
need to perform tedious labelling, merging or check-in tasks, odds are
that the quality of these tasks will degrade over time. If on the
other hand your developers add information to the CM vault as a side
effect of their work or in order to realize productivity improvements,
the opposite with hold true. Automation is a key for Agile CM, but
automation that pervades the process, not just one that does a build.
Tell me when I violate a dependency. Let me know when a new branch is
required. Let me see my prioritized to-do list. Give me a single
click to review my change, or any other change for that matter.
Eliminate the need for me to label things. Make my life easier while
helping me to make everyone else's job easier.
If you're looking
for a quick fix to your current CM process to make it Agile, forget
it. If you want an Agile process, you'll have to remove the road
blocks all the way through. Just as the weakest link in a chain
determines the strength of the chain, all parts of the process have to
be agile to attain an Agile process. If you can't easily roll back a
change, you'll hit road blocks. If you can't automate your build
procedure, you'll introduce costly, time-consuming problems.
you want to make your CM process Agile, go through it and identify how
you can reduce end-user complexity, tool misfits, areas of risk, and so
forth. If a developer asks "Why can't I just...?", don't answer right
away. Instead, ask yourself "Should the developer be able to
just...?" If the answer is yes, then look at what's preventing him/her
from doing so, and fix the process.
Agile CM Checklist
Below is a checklist of process items you should consider to make your process Agile.
- Easy Traceability
- High Quality Data
- Labels Can Mean Trouble
- Reduce Dependence on Version Codes
- Easy Rollback of Changes
- Change-based CM
- Code in Repository
- Object-oriented Queries
- Interactive Meeting Support
- Priority-based Feature/Problem Management
- Instant Queries/Reports
- Easy Re-base/Workspace Synchronization
- Easily Identify Changes for Each Iteration
- Branchless Promotions
- Automated Build for SI and Designers
- Branch per Stream
Traceability is much more important for agile processes than for
traditional ones. Agile decision makers meet often and need to have
the necessary information at their finger tips to make good decisions.
If you can't trace easily from changes to features and from features to
testing, this will not only slow down your process, but will
necessarily make the traceability data less useful, less used, and
hence lower quality. Your processes simply cannot be sufficiently
agile if your data is difficult to get to or to relate to other data.
High Quality Data - Picking up data from context.
High quality data is a result of having good forms with well-defined
set of options. But more than that, data has to be populated from
context to minimize user data errors and to simplify and speed up user
interaction. If a user is working on a particular development stream
of a particular product, that information should automatically be
populated in any changes, problems, features, etc. that the user adds.
If a new change is started by clicking on a problem to fix it, the
problem should automatically be traced to the change.
Labels Can Mean Trouble
- If you need to manually label things, you've introduced unnecessary
administration into your process. You should be able to completely
define and organize changes, baselines, build records, branches, etc.
without having to create labels. Creating a label is a short cut for
an inadequate process/tool, or a workaround for an overloaded
capability. For example, if I check out a file against a change, the
branching (if necessary) should look after itself based on the change.
No need to apply labels. A common mistake of many CM tools is to
overuse branching - for promotion, for parallel development streams,
for release identification, etc. Because of this, a branch does not
imply what it could and so labels are introduced to discriminate the
overloading of the branching concept.
Reduce dependence on version codes at the user interface.
Developers, and even CM managers should rarely have to look at version
codes. If they do, then the tool is not doing a good enough job in
specifying the context and user interaction, or the tool makes
specifying the context too much of a chore. Version codes are fine for
looking back at what was done in history. But the user interface of a
modern CM tool should be change focused, not version focused.
Easy rollback of changes.
Any agile development shop is going to have to pull changes that upset
the stability of the product. This should be as easy as rolling back
the status of the change, if there are no higher promoted versions in
the development stream. If there are, the tool should let you know and
leave you with the decision as to whether to roll back later changes as
well, or to yank the change from the stream. Both of these options
must also be easy to do.
Change-based CM - One step check-in, one step promotion, one step delta, one-step propagate/yank.
If you have to type in the reasons for each file revision you check
out, or check in each file separately, do delta reports on a
file-by-file basis, etc., you're process is a lot more tedious than it
should be. This will increase the reluctance to make and deal with
changes and will subtlety affect your process and your product quality.
Code in Repository.
One of the keys to agile development is knowing what's happening. The
CM repository needs to be a communication tool. If you can't put
you're code change in the repository when you're done your "unit"
testing, only you know it's state, and others have to work from an
older state of the corresponding files. This complicates everyone's
work, forcing additional parallel changes and merge/test cycles. Make
sure your process will allow developers to put code into the repository
when it's ready, not when someone else is.
Object-oriented queries - Testcases missing, failed requirements.
Communication is as much about getting information as it is about
providing it. If you can't go to an object and find out the things you
need to about it, your interface needs improvement. There's no room
for poorly integrated management information in an agile environment.
If you need to know if a requirement passed it's tests in a build, you
need to go to the requirement and ask that question. Similarly if you
have a requirements sub-tree and you want to know if there are
requirements not covered by testcases, you need to go to the sub-tree
and ask that question. If you need to find the build in which a
problem was fixed, you need to go to the problem and ask that question.
If the tools don't let you do that, you'll have to learn some other
more awkward way of getting that information.
Interactive CRBs, weekly/daily meeting support.
Your CM/ALM tools need to be able to support interactive CRBs, and
weekly or daily team huddles. It must provide up to the minute
information that can be easily summarized, and zoomed into for detail.
If you need to have an expert in the room to do this, that's fine, as
long as the expert can do it in real time for you to run your meeting.
Better yet, your tools should be able to provide a list of objects that
form the focus of your meetings, especially for CRBs. And tracking
meetings and action items from the meeting would be an additional
Priority-based feature/problem management.
Agile development is primarily feature/problem driven, rather than
schedule driven. Perhaps you pick features/problems from a pool, or
perhaps they're assigned to you. Your tools must provide you with the
capability to easily prioritize features and problems and to look at
the features/problems which are candidates for or designated for the
next iteration. These must be priority sorted so that the view
reflects what's most important. Ideally, when your users log into the
CM/ALM environment, or even their IDE environments, they can see a list
of problems/features assigned to them or to their group pool, at a
click of a button. Similarly they should be able to see their changes
Instant queries/reports. This is important
for meetings, but it's also important for communication. If queries and
reports are less than instantaneous, they will fall into disuse to some
degree. The longer it takes, the less it will be used. The less it is
used, the poorer the communication of information. And this extends to
operations on the query results, such as navigating traceability links
and zooming in for more information. Metrics must be visible outside
the tool, but when viewed inside the tool, it must be possible to look
at them in more detail when necessary to help understand why a metric
has fallen out of bounds. If there are significant wait times to get
these answers, your process improvement efforts will suffer and as a
result, your agility.
Easy rebase/workspace synchronization.
If it's a chore to rebase your workspace to reflect the latest changes,
don't expect it to be done often. The result will be development done
against older environments, leading to less integration testing with
other new features. As well there will be additional workload on your
developers when the time comes to check in their new development.
Easily Identify Changes for Each Iteration.
An agile process is generally characterized by short iterative
development cycles, or at least "release" cycles, where we use the word
release loosely here. When testing identifies problems with a new
build, it's important to be able to rapidly compare it to the previous
one so that the list of changes that went into the new iterations can
be used as a starting point to identify the source of the problems.
It's not good enough to identify a list of files that have changed.
Often in an iteration, a dozen or two changes may involve several dozen
file changes. If you can easily scan a list of 15 changes and pick out
two or three potential problem sources, you'll streamline your process
of correcting the iteration.
Branchless Promotions. If
you are managing promotions using branches, a promotion will mean a
merge operation is necessary. This is a pessimistic algorithm that
assumes all changes are promoted in random order. For the most part,
changes will flow through the system in the same order as their
original creation (i.e. completion). It is the exception that requires
rearranging the order of changes. Yet promotion-based branching incurs
the merge penalty always. Time to rethink this strategy to eliminate
extra promotion branches so that merging occurs only when required.
Another bad side effect of a branch-based promotion model is that it is
a big deal to introduce a new promotion level, so there is a lot of
reluctance, even if it is necessary. In agile CM, this leads to not
improving the process in the name of keeping things agile. Promotion
should occur through promotion of changes, not through merging into a
new branch. The tool should be able to look at the change state and
determine the appropriate context view and/or baseline/configuration
based on these states.
Automated Build for SI and Designers
This is a big topic and is usually a central topic for Agile CM.
Designers need to be able to do their own builds and testing. And all
building must be automated. It's no good if the designer built it one
way and it worked, but the System Integration team can't get it to
build. If your nightly builds are not automated yet, make that a
priority task. Do what you need so that your builds will run
automatically every night. Perhaps one that has all the "checked in"
changes, and another more stable one that has all the
"selected/approved" changes. When this is done, make sure that the
developers can use the same build automation capabilities that the
nightly builds use. That will allow them to test what is going to be
built and integrated before your nightly build. That will mean fewer
errors in system builds.
Branch per stream. In an agile
environment, it is important to be able to rapidly iterate, yet to work
towards releases which are longer term than an iteration. The
traditional main trunk model works well for a single release, but when
subsequent releases appear, the support structure is compromised. As
well, the strategy for working on future releases is typically
complicated to the point that only developers themselves are aware of
future release work until the future release is promoted to the main
trunk. Instead, it is recommended that a main branch exist per release
stream. That way, work may proceed on any stream at any time, allowing
support, advanced work or normal development without having to change
the process or artificially schedule dates for the main branch
reflecting a new release. If you really need "leapfrog" check-in
capability, whereby parallel teams are working on the same release but
using overlapping iterations, consider two parallel streams per
release, which eventually will merge into one.
What Else Can You Do
There are many things you can do beyond the above checklist.
of all, look at the tools you are currently using. Make sure the
architecture of those tools will take you well into the next
generation. Odds are that if your company is successful with it's
current project, the results of that project will be expanded upon for
many years to come. You don't want your tools failing you along the
Look at how you handle parallel checkouts for short term
parallelism. Many of the successful projects, including our own,
rarely need to do parallel checkouts. Or if so, they are done in an
informal fashion (i.e. get the code make the changes, reconcile on
checkin). Why? Well, for us, several factors come into play.
- We check our code back in as soon as it's ready
- Our design architecture allows us to focus most of our overlap on very simple changes which can be turned around quickly.
File ownership allows small groups to take responsibility for specific
areas of the system. But we still do need parallel checkouts.
our tool allows short term parallel checkouts without the need for
branching. The change record contains the same information as provided
by a branch, and the CM+ tool allows us the option of recording or not
recording parallel checkouts. At check-in time, we are informed if
there have been any changes since the checkout operation and, if so, a
reconcile operation for the change being checked in is initiated.
no real value to having a couple of extra branches to show the history
of the past three days work. As long as the two parallel changes make
it into the build and the change deltas clearly show the changes
independently, I'm happy. In fact, often you'll find branching for
short term parallel changes only confuses the branch history. Some
tools even have ways of purging such branches to clean up the history.
All this adds to the complexity. So why is branching done for short
term parallel changes? To provide visibility of the changes. Your CM
tool should provide this visibility without forcing you to complicate
your branching architecture.
Tools must have a net payback for
all users so as to encourage their continued usage. If your users
currently have to collect all of the files and do a delta report on
them and then email that report around, that few extra minutes of work
will add up over the course of several hundred changes. Your tool
should allow collaborative reviews from a single change identifier
using a single click or two. If merging a change from one stream into
another (we call this Change Propagation) is difficult because the user
first has to go through a long process of figuring out which files need
to be merged and what the merge ancestors are for each merge, what will
happen? Any process requiring a merge will be strongly discouraged. If
users have to create configuration view specifications to ensure that
they can see the appropriate versions of their files, there will be
loss of productivity and manual errors causing frustration. It all
comes back to ensuring that the tools you select will support your
Agile CM Requires Good Software Architecture
you think that you can ignore your software architecture and still have
an Agile CM process, think again. I've seen countless architectures
where a handful of files are bottlenecks to the whole development
process. The same file needs to be changed by virtually everyone. A
single file is stretching the limits of your compiler or version
control tool. Introducing product variations is causing nightmares in
the software. Here are three key things that all projects can do to
simplify their lives.
Minimize contention on files.
Files have high levels of contention because they were designed to have
high levels of contention. If I have to put a change number in a
single file every time I make a change, I'll expect a lot of
contention. If instead, all I have to do is call a routine with my
change number, the contention disappears. Same functionality, no
contention. Most files which have a high contention on them can be
dealt with in this manner. Whether it's a change number, a message
code, or even a function. Replace the one-file policy with a
distributed, less contentious one. You'll also find that your version
numbers for the file stay down to earth rather than growing
astronomically, and your CM tool may like that.
File ownership for quality.
Did you ever work in a car wash? How much attention did you give to
each car? Now when you're cleaning you're 1972 Mustang do you give it
the same level of attention or more? Code is something created. The
creator has pride in his/her creation. Don't destroy that pride by
making it open season on everyone's code. File ownership does not mean
that the owner has to make all the changes - just that the owner has
the responsibility and authority for the design of that code on an
ongoing basis. The owner reviews all changes at a minimum. The CM
tool can help support and enforce file ownership, ensuring that the
otherwise "open season" Agile process does not degrade code quality.
In the haste to get the product out the door, data is hard-coded into
source code files. This may make changing the data more difficult, but
the real crunch comes when product variants are produced by having
variant files with different values of the hard-coded data. This can
be a CM nightmare if there are many variants along different axes.
Work from the outset at a strategy to make data run-time configurable.
This does not involve more code (usually less in fact), but does
require an architectural decision up front to allow data to be run-time
configurable. The result is that code is written with that
assumption. Allocations are done as required rather than up front,
thus making use of changes to the configuration parameters. Variables
that can be changed through the user interface are used rather than
hard-coded "#define" statements. Slightly different algorithms are
used to support some dynamically changing values at system start-up, or
even throughout the life of the application, rather than assuming
static values. The amount of flexibility not only reduces the CM load,
but speeds up the design as the decision on what the hard-coded values
should be can be deferred to the product team.
Agile is Just Good Design and Common Sense
go through all of these factors and tell me which ones you would not
want to take advantage of if you weren't looking at creating an Agile
CM Process. Are you surprised? Agile Design and Agile CM are just
good common sense practices with some terminology for wrappers. So if
you're not looking for an Agile CM Process, I might recommend that you
read this article and the other related articles anyway. It just makes
Joe Farah is the President and CEO of Neuma Technology . Prior to co-founding Neuma in 1990 and directing the development of CM+, Joe was Director of Software Architecture and Technology at Mitel, and in the 1970s a Development Manager at Nortel (Bell-Northern Research) where he developed the Program Library System (PLS) still heavily in use by Nortel's largest projects. A software developer since the late 1960s, Joe holds a B.A.Sc. degree in Engineering Science from the University of Toronto. You can contact Joe by email at firstname.lastname@example.org