Friday, 17 August 2007

In an Extreme Programming world don't forget the value of traditional QA testing

When Extreme Programming is embraced, it is tempting to get carried away with the power of unit-testing.

If you're unit-testing everything across the whole build then surely everything should work. It is important to remember that unit-testing, when done properly, is just that. Testing units of code for compliance against a known set of outcomes. It doesn't necessarily test system wide interactions.

The development team here have recently spent a lot of time migrating our existing transaction management from the EnterpriseComponent, COM+ based, system to the TransactionScope framework provided in .Net 2.0.

Nicholas, Development Manager, as described the detail of the issues we've faced in his post: Transaction Scope - Hopes dashed. It has been horribly dissappointing, the performance profiling we did showed a big improvement, the deployment process was simplified. Everything seemed perfect.

The key to writing effective unit-tests are focusing on the unit itself and the logic in that unit and nothing else. This makes the test very quick to run, meaning you can run a full build, with tests in a short enough time to make you prepared to do it regularly.

But proper Quality Assurance testing picked up these issues. It wasn't until we ran these system wide test that the issues materialised. Generally this leads to further unit-test to ensure the issues don't materialise. Although in this case we decided to back-out this set of changes.

So unit-tests are an integral part of what we do, but good old grunt work testing is still the last, and most important, line of defence.

No comments: