Timebox testing in DSDM developments

Timeboxing means that when a pre-determined date occurs, an item is delivered whether or not it is 'complete' as originally envisaged. This is a perfectly logical approach to development.

Any functionality which has not been developed in time is not delivered. Something useful is therefore delivered on the due date - as opposed to everything requested at some (often considerably) later date.

However, applying this principle to software testing is neither so simple nor so beneficial. The purpose of software testing is to measure the quality of a piece of software and to establish its fitness for the next stage of its life-cycle.

Obviously, if all intended testing (including fault fixes and re-testing) has been completed within the timebox, no problem arises. But if all intended testing has not been completed, what should happen? Should we:

• stop testing and deliver a partially untested system, almost certainly rich in undetected faults?
• stop testing, remove the untested or faulty functionality, and deliver what can then, in truth, only be regarded as an entirely untested system?
• carry on until the end of the current cycle (pass) of testing, and then deliver a tested system with known, and possibly 'unacceptable' faults?
• ignore the timebox completely, and carry out as many cycles of testing as are necessary to remove all known (or unacceptable) faults i.e. Revert to a ‘traditional’ testing life-cycle?

Whichever option is chosen, we will have been forced either to prejudice system quality or to abandon timeboxing. I tend to the view that it is so likely that testing will not be completed within the prescribed period that it is unwise to timebox testing in the first place. (My 'tester's pessimism' has developed as a counterpoint to the seemingly unquenchable, and often enforced, optimism of project managers!)

Abandoning the timeboxing of testing raises another question, which goes to the heart of the DSDM approach. If one timeboxes development but not testing, has one not driven a coach and horses through the logic of DSDM? In other words, does the decision to timebox development logically necessitate inadequate testing?

Does it make any difference whether we are talking about 'interim' stages of testing which deliver systems to other testers (e.g. Unit or System Testing) or about 'final' stages which deliver systems to users (eg. Acceptance Testing)? It might seem obvious that it is a more serious matter to deliver a fault-ridden system to its users than to a testing team.

I do not think this stands up to analysis. A poor quality system delivered for System or Acceptance Testing will cause considerable delay and inconvenience to the testers, to say nothing of the increase in the cost of fixing faults found later than they might have been.

(I have seen a plausible calculation that the true cost of a production fault which takes the programmer half-an-hour to debug and correct is £25,000!) If the System or Acceptance Testing has itself been timeboxed, the problems noted above will only be exacerbated.

Chris Allen, Prudential