Saturday, December 6, 2008

Pesticide paradox

Do you have a lot of automatic tests? Do you continously add new test cases and improve existing ones? No? You shouldn't sleep well... ;-)

Please be aware that the value of the same tests executed over and over again (e.g. regression tests) decreases with time. This is known as "pesticide paradox".

Following ISTQB "Certified Tester Foundation Level Syllabus":

If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this “pesticide paradox”, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

The conclusion:

It's not enough to have automatic tests suite. It's essential to improve it over the time.