Configuration Management in Software Testing

What is configuration management?

Our systems are made up of a number of items (or things). Configuration Management is all about effective and efficient management and control of these items.

During the lifetime of the system many of the items will change. They will change for a number of reasons; new features, fault fixes, environment changes, etc. We might also have different items for different customers, such as version A contains modules 1,2,3,4 & 5 and version B contains modules 1,2,3,6 & 7. We may need different modules depending on the environments they run under (such as Windows NT and Windows 2000).

An indication of a good Configuration Management system is to ask ourselves whether we can go back two releases of our software and perform some specific tests with relative ease.

Problems resulting from poor configuration management

Often organisations do not appreciate the need for good configuration management until they experience one or more of the problems that can occur without it. Some problems that commonly occur as a result of poor configuration management systems include:

•  the inability to reproduce a fault reported by a customer;
• two programmers have the same module out for update and one overwrites the other’s change;
• unable to match object code with source code;
• do not know which fixes belong to which versions of the software;
• faults that have been fixed reappear in a later release;
• a fault fix to an old version needs testing urgently, but tests have been updated.

Definition of configuration management

A good definition of configuration management is given in the ANSI/IEEE Standard 729-1983, Software Engineering Terminology. This says that configuration management is:

• “the process of identifying and defining Configuration Items in a system,
• controlling the release and change of these items throughout the system life cycle,
• recording and reporting the status of configuration items and change requests, and
• verifying the completeness and correctness of configuration items.

This definition neatly breaks down configuration management into four key areas:

• configuration identification;
• configuration control;
• configuration status accounting; and
• configuration audit.

Configuration identification is the process of identifying and defining Configuration Items in a system. Configuration Items are those items that have their own version number such that when an item is changed, a new version is created with a different version number. So configuration identification is about identifying what are to be the configuration items in a system, how these will be structured (where they will be stored in relation to each other) the version numbering system, selection criteria, naming conventions, and baselines. A baseline is a set of different configuration items (one version of each) that has a version number itself. Thus, if program X comprises modules A and B, we could define a baseline for version 1.1 of program X that comprises version 1.1 of module A and version 1.1 of module B. If module B changes, a new version (say 1.2) of module B is created. We may then have a new version of program X, say baseline 2.0 that comprises version 1.1 of module A and version 1.2 of module B.

Configuration control is about the provision and management of a controlled library containing all the configuration items. This will govern how new and updated configuration items can be submitted into and copied out of the library. Configuration control is also determines how fault reporting and change control is handled (since fault fixes usually involve new versions of configuration items being created).

Status accounting enables traceability and impact analysis. A database holds all the information relating to the current and past states of all configuration items. For example, this would be able to tell us which configuration items are being updated, who has them and for what purpose.

Configuration auditing is the process of ensuring that all configuration management procedures have been followed and of verifying the current state of any and all configuration items is as it is supposed to be. We should be able to ensure that a delivered system is a complete system (i.e. all necessary configuration items have been included and extraneous items have not been included).

Configuration management in testing

Just about everything used in testing can reasonably be place under the control of a configuration management system. That is not to say that everything should. For example, actual test results may not be though in some industries (e.g. pharmaceutical) it can be a legal requirement to do so.