More white papers:
Find out more ...
|
|
Neuma White Paper:
CM: THE NEXT GENERATION of Top 10 Best Practices
There are a lot of best practices in the CM trade. You might be
hard-pressed to pick the best ones out. But that's exactly what we'll
try to do in this month's article. You may see a few that you're
quite familiar with. There may be a few that are hotly contested
issues. I'll bet there's more than a few that you haven't fully
considered before. Of the dozens of CM/ALM best practices we use
ourselves, I've tried to pick the top ten. It was a difficult task, so
I'll start by mentioning the 10 runner-ups.
20. Organize Shared Code into Separate Products
When
you're producing software that is shared by different products, issues
such as development ownership, timing of releases, etc. are typically
difficult to manage because the different products utilizing the shared code have different sets of requirements, and possibly differing
processes. Often the shared code starts out as part of one product's
development but business logic dictates that it be shared in multiple
products.
The best practice in this situation is to partition
the shared code into one (or more) separate products. That product has
as its customers, the other product teams. Control of releases,
frequency, features, etc. really have to be managed as a full product is
managed. Different customers will want to move to new releases of the
shared product at different times. Some will want the new features,
some won't want the risk. The shared code may need to branch into
different release streams to support the various customers.
19. Use Change Promotion instead of Promotion Branches
Branches
are often used to track changes. This strategy works, but it is
generally used because of insufficient tool and process technology.
Promotion branches help ensure changes are promoted through the
promotion levels dictated by the process. However, they assume a
pessimistic strategy which caters to the worst case rather than the
most usual case. As a result, there is a lot of merging, labelling,
etc. that is unnecessary. Roll-backs are also difficult. As well,
it's quite difficult to adjust the change process to support additional
promotion levels, because of the cost of each promotion level in terms
of merging/labelling, and because of the administrative issues involved
with the insertion of new promotion branches among the existing ones.
Ideally,
you should use technology where promotion is simply a status change on
the change object (assuming there is one). With the appropriate
processes and technology, merge operations should only be required when
changes don't flow through the change promotion process in the same
sequence from level to level. So generally, the configuration management is
reduced to simply change management - promoting/rollback the change
status. The technology should allow you to view configurations at each
promotion level based on the change status alone. So although this is
a best practice, it is highly dependent upon the proper technology.
18a. Separate Customer Requests from Engineering Problems/Features
Customers
want to raise problems, ask for new features, or simply specify how
they believe the tool should operate, without reference to whether it
is a feature request or a problem report. Some customers will ask for
the same features as other customers, perhaps with a different twist or
priority. Customers want to know the status of their requests, many
of which can be dealt with directly by the technical support team
without involving the engineering team. Some are data configuration
issues, others non-problems. The customer just wants the majority of
requests addressed successfully.
The engineering team doesn't
care about how many customers report a problem or request a feature -
they just want their marching orders: a list of problems to be fixed
and a list of features to be implemented. Sure they're prioritized
based on customer input, but ultimately, this reflects the value of the
feature in the product. It's too hard for customers to classify issues
as features or problems, and it doesn't really matter anyway - it's
input to the product team. But it's critical to distinguish
engineering problems from features as they are completely different
processes. All this to say, track customer requests separately from
engineering problems/features. That is not to say that you should use
a separate repository. And of course the engineering records should be
traced back to the customer requests.
18b. Separate Problems/Issues/Defects from Activities/Features/Tasks
A
related best practice that we touched on in the last paragraph is that
problem fixing is a completely different process from feature
development. One is a fix to the product because it didn't meet the
requirements. No new requirement specification. And the specification
for the change is the problem report itself. The first step is to
ensure reproducibility. The problem has to be investigated for
potentially all supported development streams. And so forth.
Feature
development is completely different. The feature has to be designed
and fully specified. New requirements are needed to put the case
forward for the feature. User documentation has to be updated. A new
set of test cases has to be established. Problems and Features (or
more generally activities) are different beasts. Don't try to track
them both as "tasks". If you have separate ALM tools to deal with
those processes, that's perhaps a different matter. But in a fully
integrated system, keep them separate.
17. Use Tags and Separate Variant Code into Separate Files
Avoid
the temptation of using parallel branches to manage separate variants.
Consider that if you have 3 variant options for language, 3 more for
security level and 2 more for packaging, you'll have 18 branches
already. Variant management must be addressed in the software
engineering realm, not in the CM realm primarily.
The CM realm
should allow you to tag files with the appropriate variant tags, and to
select files based on one or more specified variant tags. In some
cases, the CM tool might go so far as to allow you to tag specific
changes as "variant" changes, that can be "automatically" applied to
your source code when that variant is selected.
But the
bigger task of the CM manager is to make sure that the development
organization understands how to deal with variants. The preference is
run-time configuration. Next to that is link/load time configuration
(by selecting the files to load or to link together). Then comes
compile-time variants, also known as conditional compilation. This is
less appealing because you then need different variants of the same
object file, making your build task more complex (often the entire
build is replicated for each variant to get around this problem). But
at all costs avoid the parallel branch approach.
16. A Problem Is Not a Problem Until It's In the CM Repository
This
best practice will seem obvious to those who have always relied on good
processes. A problem is not a problem unless it is defined in the CM
repository. That means that you don't have source code changes to fix
problems that have not been identified in your problem tracking system
(which should be part of your CM/ALM system). When you fail to abide
by this rule, you get source changes in that have not been authorized.
You get problems fixed that you don't know about and can't tell your
customers about. Your quality metrics get messed up. And so forth.
If
you've always relied on your developers to just identify and fix bugs
without reporting them, stop it. Force them to raise a problem report,
but make it easy to link that problem report to their change so that
they don't have duplicate data entry describing the change. If you
don't know how many problems you've fixed, you'll never have any idea
as to how many are still left to fix. Simply ignore problems (other
than true emergencies, and other than getting the source to do the data
entry) unless they are in the repository.
15. Use Live Data CRB/CIB Meetings
Use
live data at your CRB/CIB meetings and update the information at the
meeting. Today's meeting rooms make this easy to do. If your tools
are not up for the task, time to review and possibly upgrade them. But
the more modern change/problem/feature management tools will support
interactive query and update in the context of such meetings. The
result is more timely decision-making, quicker response times and less
administration with fewer errors.
14. Warm-standby Disaster Recovery
Whether
it's disk crashes, earthquakes, terrorism, floods, hurricanes, or
whatever, you need to be able to continue development and provide
ongoing support to your customers. The way to support this is to
ensure that your CM/ALM toolset has full redundancy at an off-site
location that can be quickly switched to in case of a disaster.
Whether it serves strictly as a backup site or as a fully redundant
node in a multiple site or remote site solution, you cannot afford to
put together a solution for your team that will leave them high and dry
in the case of a disaster. And it has to work for your entire ALM
solution. This may not sound like a CM best practice at first, but it
is and for some of you it may need to make the top ten list.
13. Continuous Automation
Ultimately,
the quality and responsiveness of your CM/ALM solution will be governed
by the level of automation. The more advanced the automation, the
higher your quality and the lower your costs. And your processes will
be affected significantly. When automation is in place, there's less
need for auditing, there's more time for improving processes, there's
more budget for improving your tools, and there's more focus on your
core business. Once you've automated, step back and look at your
gains, but also look at your remaining manual steps. Can they be
further automated?
12. Change Control of Requirements
Change
control is not a concept purely for source code. Requirements need
change control as well. This means that you must be able to group
changes to requirements into requirement change packages. You need
ownership of your requirements and this needs to be tied into who is
allowed to make changes to them. Version control is a necessity as is
the ability to create baselines. Promotion levels for your
requirements and the ability to view the various configurations. You
likely have fewer people involved in changing requirements, but that
doesn't mean you don't need change control.
11. Org Chart Integrated with CM Tool
Your
organization chart is a key component of your CM solution. If you need
to identify metrics for different parts of the organization, it sure
comes in handy. If you need to automatically reallocate permissions
because of vacations, again the org chart is helpful. If you want to
do reporting by department, or if you want to direct problems to the
correct area, the org chart. If you want to identify what your staff
has on their to-do lists, an org-chart based query should let you
roll-up a list against all the staff in a particular group.
The Top Ten
OK.
So we've gone through the ones that did not make the top 10. Not
because they're not important. I'm sure many of use see them as key
practices, and so you should. But let's continue the countdown and
identify the more critical, and more basic best practices.
10. Tailor your User Interface Closely to your Process
You've
got a CM tool, perhaps an in-house tool, maybe an open source tool, or
even a commercial tool. If you can't customize the user interface to
match your process you're in for a lot of headaches. A couple of
"what's this box for" fields on a form will slow down your development
team. Calling a file a module or a workspace a directory will add to
confusion. You'll either have to put more money into training or put
up with the data entry errors the foreign terminology leads to.
Your
process is well understood by your team. Make sure you take the time
to tailor the tool to match that process. And part of that includes
hiding any functionality that is not specific to the user's role. Some
tools have hundreds of buttons, menu items, etc. If all users have to
see all of them, they'll never find what they need. So hopefully you
can customize the tool so that only an hour or two of training is
required for each role. If not, take the time to write a compact role
guide which gives the specifics for how to use the tool for all of the
actions in a given role. Include a single page quick reference sheet
that can serve as a reminder until the team members are more familiar
with the tool. And a separate advanced guide/sheet might help those
who are more ambitious.
9. Automate Administration to Remove Human Error
Administration
can be a costly aspect of your CM solution. I've heard horror stories,
and I'll bet you have too. Excess administration is also a road block
to process improvement, primarily because of the impact on all of the
administration resources. But the worst part of admin-intensive
solutions is that human errors will cause both decreased quality and
lower productivity. If you can automate your administration, do so.
And keep going until there's no more to automate. There are solutions
out there which claim to require a fraction of a person for CM
administration. And I believe these claims in some cases. What are
they doing right that your solution can't do?
8. Enforce Change Traceability to Features/Problem Reports
If
your developers are creating changes without authorization, your
product quality will eventually suffer. Every change needs an
authorized directive, whether an approved/assigned problem report, an
approved development activity/feature, or a work authorization. Only
then can you direct what's going into your product releases.
But
that's only half of the picture. If your team's changes are not linked
back to the authorized record (i.e feature/activity or problem report),
they may be marching to their orders, but you won't know what state the
product is in. And if problems arise, you'll have a difficult time
tracing them back to the source. The best way to force traceability
between changes and features/problems, is to force the user to select
the problem/feature in order to create the change package (or checkout
a file). The user interface actions should trigger automatic
traceability information between the selected problem/feature and the
change package (or file revision in the worst case).
If your
tool can't enforce this traceability, your team will have to spend time
defining the traceability, either by entering information into
different areas of the solution at checkout time, or by audit (and
linking action) at the time of code review. Your code review should be
a time to do part of your configuration audit - that is ensuring that
the code changes match the request. But if the request is not traced
to the code changes, this will be difficult at best, and your process
will not be bullet proof.
7. Main branch per release vs Main Trunk
Do
you want to start a good debate in your organization? Bring up this
topic. Is it better to have a single Main trunk to which all changes
are eventually merged or to have one main branch per supported
release?
Well to start with, all changes are never merged to
a single Main trunk. With a Main trunk, the team has to decide which
release the trunk reflects. Changes not made to that release have to be
made somewhere else. If they're for future releases, they'll
eventually get merged to the Main. Otherwise, not. Decisions have to
be made as to when the Main switches from one release to another.
Directions need to be given for how to deal with changes which are for
other releases not on the main trunk, hopefully without delaying a
developer's check-in operation (which would in turn either delay other
development or incur unnecessary parallel development and the resulting
merging and re-testing).
A Main-branch-per-release strategy
is simple. You make the change in the branch that the request has
targeted. The way you do that is always the same, regardless of
release or timing. Some shops (and tools) support making changes
directly to the trunk, without any branching - these also support the
concept of a change package. In more advanced tools, changes to older
releases can be automatically inherited (optionally) into newer streams
(e.g. if the code has not changed between the older and new stream).
The process is simpler and supports stream-based branching strategies
(more on that later).
6. Dumb Numbering
Source
files have names individually chosen by each developer. They don't
even have to be unique (although you need a unique identifier for every
source object). Not so with problem numbers, requirements, activity
codes, test cases, etc. Naming these would be considered an onerous
task. In a relational world, this is not so much a problem. But in an
engineering database, everything needs a unique id so that it can be
referenced (for traceability purposes) and identified for configuration
identification purposes.
Too many want to make the "unique
identifier" a way to know the content: system+priority+3 digit number.
The problem happens when the priority changes or the system is split
into two.
All of your unique identifiers suddenly change!
Unique identifiers must be numbered using a dumb numbering system. For
the most part, this means a fixed prefix plus a number (e.g. a.1232 for
an activity code). Some might want to tack on a product prefix, only
to find that the same requirement or test case is going to be used in
the next product, invalidating the id. Unique ids need to be kept
dumb.
There is room for some prefacing when the preface
cannot change. This occurs when a "parent" record owns the record
being identified. For example, a change record might be numbered
sequentially as a sub-record of the developer making the change.
History cannot be changed so this would be an appropriate prefix.
Perhaps documents could be sequentially numbered within their document
category (tech note: TN.33, project status PS.412, etc.) - in this
case the document type record is the "owning parent".
But
apart from such ownership sub-record prefaces, putting data into a
unique id is just asking for trouble. In the old days this was a
common practice because not everyone had access to the data records
which described the object. That's different today, or at least if
it's not, you have other problems that must be remedied first.
The Top Five
We're
down to the final five. These are not just important, but critical for
moving your process forward. Perhaps you're familiar with all of these
already. If you're not, get familiar with them so that you understand
the impact clearly.
5. Continuous Integration with Automated Nightly Builds from the CM Repository
Your
development team is continuously submitting changes to your
repository. You can put code reviews in place (highly recommended - in
fact this would have made the top five if the topic were a bit
broader), have "unit testing" (i.e. change testing) rules, and perhaps
a number of other software engineering guidelines. But if you don't
build the system regularly and test it, you're risking some major
delays as unforeseen incompatibilities bring new "builds" to a halt.
If 1,000 software updates have been submitted, it may take a while to
narrow down the issues to their caused, or even to isolate the issues
in themselves as some may have overlapping symptoms.
Continuous
integration is a process of performing builds very regularly, at least
nightly, if not more frequently, and doing basic sanity testing on the
build, and a good dose of automated testing too, if you're so equipped
to do so. When problems arise, they can be readily narrowed down to a
few to a few dozen software updates. At this frequency of builds,
problems generally arise in one's and two's, rather than by the
bushel.
A second advantage is that the resulting build
environment can be used by your development team. This can be from a
simple confidence that they can use the latest built source code for
their own testing, to generating shared compile/build environments for
use in incremental builds or multiple subsystem testing. The more
timely feedback on the cause of the problem is also helpful as the
developer still has the changes clearly in his/her mind and has not
fully moved on to something else.
If you want to build an
Agile environment, continuous integration is a necessity. And the CM
tools should support it. Settle for nothing less than fully automated
builds without CM manager/Build manager intervention unless problems
occur. If you can't do that, your processes are incomplete or your CM
tools and procedures are too complex.
4. Data record Owner and Assignee
Ever
hear the expression garbage-in garbage-out? Why does this happen?
Developers take pride in their work. If they own a file and are
responsible for all changes to it, they will treat it with a certain
amount of respect and discipline. If anyone can change any piece of
code at any time, this care is lost. "So what" if Bob has taken care
to keep his code concise and well factored. I need to make a quick
change to fix a problem and I'm going to use a copy and paste method
because it's the fastest (really?!). Bob, on the other hand, would
have added a parameter to the existing routine(s) so that the same code
could be used for the purpose, without replicating functionality, bugs,
and future code diversion.
Source code ownership is
important. At the file level, at the branch level. But data ownership
goes beyond source code. Every data record needs someone who is
responsible for it - for it's accuracy, for ensuring it runs through
it's process, for assigning tasks related to the record and for
tracking such assignments.
The best practice of data
ownership is meant to help ensure data quality, and this helps with
process quality. My recommendation - an "owner" who is responsible for
the data and associated process/tasks involved, and an assignee who has
been assigned to the next step in moving the data through it's
process. For example, a development manager might have responsibility
for a problem report, ensuring it gets successfully resolved. But
(s)he may assign it to a developer, then to a tester along the path to
closing out the problem report. Every data record needs ownership in
the CM world so that there is both accountability and clear direction.
3. Status Flow for all Records with Clear In Box Assignments
A
close relative of data ownership, a clear state flow is required for
each type of object. Your tools should clearly show you the state flow
for each object, and a status should be tracked against every object.
Each state transition should clearly identify the applicable
roles/permissions for the transition. It should be possible to specify
additional rules, and triggers pertinent to the transition. When you
put all of this together, your data along with the processes should be
telling the team what needs to be done.
Based on your role,
you should have a number of in-boxes: documents to be reviewed,
problems assigned to you to be fixed, changes that you need to review,
other tasks assigned to you. These should appear in your in-box as a
result of status changes and ownership assignment. In some cases
assignment is by name. In other cases it's more generic, such as by
group function. In the latter case, the in-box is potentially shared
by others. Some people are better communicators than others. If you
depend on your staff to keep assignments clear, you'll have hits and
misses. If your process and data is doing the job for you, in concert
with a decent CM/ALM tool, you'll march along more efficiently, and
you'll be able to easily identify your bottlenecks and problem areas.
2. Stream-based Branching Strategy - do not overload branching
Branching
provides an effective means for supporting parallel releases. However,
most shops use it for various other purposes: change promotion, change
packaging, build tracking, variant management, short term parallel
changes, feature tracking. I'm sure you can add to the list. The
result is that what should be a straightforward two dimensional "tree"
structure of revisions (i.e. release stream and revision within a
release), turns into a spaghetti maze of branches. Elaborate branching
strategies are designed and reviewed intensively to ensure that the
team can clearly follow it. Branching, labelling and merging skills
are honed to ensure that these can be done effectively.
But
try to ask your CM tool to understand what all the branches mean - well
that's more of a problem. The result is that an inordinate amount of
effort goes into these mazes, unravelling them. And what's worse is
that the CM tool can't give the developers guidance. Instead, the onus
is on the developers to learn and follow branching strategies. So CM
becomes tedious rather than automated. The significant amount of
effort is directed to configuration management, rather than to change
management.
Stream-based branching is the only strategy I've
seen that eliminates almost all manual CM. It's the only one I've seen
that eliminates the need to label (manually). It's the only one I've
seen that allows developers to start using the system without training
on branching strategies. It's the only one I've seen that truly
minimizes branching. It's the only one that can automatically tell
developers that they need to branch.
It scales from very
small to very large projects extremely well. It does require tools
which deal with change packaging, change promotion, build tracking,
etc. without using branches. I'm sure having this as the #2 best
practice will raise a few eyebrows, especially among those who have
never tried it, but it so dramatically simplifies CM without
compromise, that it is a necessary path for any next generation CM.
The Best of the Best Practices
1. Use of Change Packages
This
one is obvious. Everyone will agree on it, I hope. It would really be
my hope that this is taken as much for granted as is storing file
revisions - it shouldn't have to be a best practice - it should be a
given.
But although there are several tools which support
change packages, and others which at least go half way and tie file
revisions to a common task, sadly, many, if not most, projects today do
not use change packages. They check in files one at a time. They
create delta reports by appending the delta reports of several
different files. They promote each file independently, hopefully with
some manner of determining which files all have to be promoted
together. They merge changes by merging each file separately, relying
again on some virtual container (usually in the developer's head) to
identify which files are involved.
Unfortunately, they are
supported by industry pseudo-APIs such as the SCC API for IDE
plug-ins. This is a file-based API, although at least one tool has
managed to work in a change-based operation around the API.
If
your developers complain that CM is too much of an overhead on them,
you've got to give them an incentive to make their life easier. Change
packages, when properly supported, in an integrated tool (with
integrated repository and user interface) will actually decrease the
developer's workload. Not just in the long run, but on everyday
operations such as delta reporting. The sales job (i.e. selling CM to
developers) becomes much easier.
So what is a change package?
It packages all changes, both files and directory structure, together
along with the reasons for the change (in the form of traceability
references), and the integration data, such as product, target
release/stream, change status, unit test comments, etc., into a single
record. Build managers only have to consider 20 changes instead of 75
files for their incremental build. Team members can stop looking at
revision codes and start looking at changes. Configuration
lineup/baseline algorithms can ensure that all file for a change are
included as part of the configuration. Some tools will even analyze a
workspace and build (and possibly submit) a change package based on the
differences with respect to the context view (e.g. CM+).
Change
packages make developer's work easy. It also makes it easier for them
to trace back through history looking for specific changes. Change
packages have been around in some tools for a couple of decades. But
in other tools, the "state-of-the-art" is the introduction of new
methods and tools which cope with the fact that they don't support
change packages. If you're not using change packages in your CM
environment, your CM will be viewed as overhead or heavy-handed
process. Your productivity and quality will both be limited.
Perhaps
you've had a bad experience using change packages. If so, the problem
was not change packages, it was the technology that was trying to
deliver this functionality to you. Find a vendor that will show you
how easy it should be. Then, use change packages!
Best Practices Summary
So there you have it - the top 10, in fact, the top 20 (or 21).
1. Use of Change Packages
2. Stream-based Branching Strategy - do not overload branching
3. Status flow for all records with Clear In Box Assignments
4. Data record Owner and Assignee
5. Continuous integration with automated nightly builds from the CM repository
6. Dumb numbering
7. Main branch per release vs Main Trunk
8. Enforce change traceability to Features/Problem Reports
9. Automate administration to remove human error
10.Tailor your user interface closely to your process
11. Org chart integrated with CM tool
12. Change control of requirements
13. Continuous Automation
14. Warm-standby disaster recovery
15. Use Live data CRB/CIB meetings
16. A Problem is not a problem until it's in the CM repository
17. Use tags and separate variant code into separate files
18a. Separate Problems/Issues/Defects from Activities/Features/Tasks
18b. Separate customer requests from Engineering problems/features
19. Change promotion vs Promotion Branches
20. Separate products for shared code
Perhaps
these best practice descriptions are more detailed than you expected.
Perhaps they don't quite fit into your picture, or you find them too
opinionated. Maybe you expected one on how best to do labelling or on
how to derive a branching strategy - these would, of course, be
incompatible with the best practices that I've mentioned here. Perhaps
you have some obvious ones that are missing here - Give me your
feedback. I'd love to hear your input. There are many more on the
fringes that fall into software engineering, or tool architecture,
rather than CM process. What about the order of importance in which
I've presented them? That's gotta ruffle some feathers. If not, I'd
love to hear that too.
Joe Farah is the President and CEO of Neuma Technology . Prior to co-founding Neuma in 1990 and directing the development of CM+, Joe was Director of Software Architecture and Technology at Mitel, and in the 1970s a Development Manager at Nortel (Bell-Northern Research) where he developed the Program Library System (PLS) still heavily in use by Nortel's largest projects. A software developer since the late 1960s, Joe holds a B.A.Sc. degree in Engineering Science from the University of Toronto. You can contact Joe by email at farah@neuma.com
|
