(Warning: This is an incoherent post – there has been no attempt to organize the paragraphs such that it flows in a thoughtful manner. These thoughts just poured out of my mind at that moment in time…)

Software Testing is important to maintain the quality of software produced. Good testing is difficult, hindered by time, management, and possibly testing techniques.

To me, good tests must be documented. This may come in the simple form of documented test cases, although I seriously prefer automated tests to perform those test cases. Automated tests are usually run more often, as compared to needing people to sit and key in pre-defined input and check outputs.

Using human as testers have an additional advantage, they help to introduce human error. Good software should be able to recover and remain stable under human error conditions (prefably all kinds of error conditions) To simulate this situation, automated tests should introduce elements of randomness. For example, free text input such as text boxes should be tested with all possible input characters from the keyboard, as well as random combination of input keys. The former ensures that the system is able to handle all characters, and the latter simulates “a monkey banging the keyboard”.

Applying randomness in testing catches more errors than if you always used fixed input, say “John Smith” for full names. Or even worse, “test” as a full name. Can your system accept “Thomas D’Cruz”? Or a 94-character long indian full name? Yet once you catch an error with random input, are you able to reproduce the same input to verify that your system has handled that case?

This is a common flaw when doing automated random testing. Test sequences MUST be logged (to files) so that the errorneous test can be repeated with the exact same input so that the bug can be verified (as a bug) and verified (as fixed). Although threaded programs may not return you the same output given those same input, the information will also serve as a debug log that will assist you in reproducing the error. If the error cannot be reproduced, it cannot be verified.

Using frameworks such as JUnit will assist you in combining several tests into a test suite so that automatic regression testing is simplified. Hopefully every refactor or change to the base source can be accompanied by a test run.

RSS feed for comments on this post · TrackBack URL

Leave a Comment