Effective bug smashing with test cases
There are a lot of opinions buzzing around about how testing and test driven development helps you writing better software. Reading Secrets of the rockstar programmers just now, I’ve stumbled over an intresting interview with Hani Suleiman, co-author of Next Generation Java Testing. As this is a book purely on testing, you could expect him to be a fervent advocat of TDD. Surprisingly, he takes a very pragmatic approach to testing and simply states, that it should be as easy as possible to be done while understanding that deadlines often prevent you from equipping your polished code with the last 20 percent of test coverage.
Nevertheless, the quote, that pinged my mind, was this one:
The user said, “This thing doesn’t work.” You’re gonna test something that potentially doesn’t work. And so, once you have that test, it’s very easy, because you have that verifiable bug.
(Hani Suleiman on testing and bugs)
Let me elaborate on that a little bit. Hani’s observations largely correspond to the way I usually get started swinging my swat. So suppose you have system in state A in your version control system and a bug report claiming that there is something wrong in this state. Starting to find the cause for the bug you typically accept the ticket, indicating that you are currently working on it. Assume you can reproduce the issue with a test case you fix the according code snippets, make sure you get that green bar and commit everything to the VCS again. Finally you mark the ticket as fixed and add “fixed with revision XYZ” as comment to guarantee tracability, that’s it.
That sound really proficient, doesn’t it? Where the hell is the problem? I have to admit, that this is the way I am working currently, and I really feel comfortable with it. So here is, what the quote made me think about:
Comparing the original state of the VCS (revision XYZ - 1) and the fixed state of the VCS (revision XYZ) you only see one more test case and some modified. Who really tells you that the test case did not work before? As you have committed the already working test case along with the changes, you don’t have a possibility to check, whether the test case is really tight enough to reproduce the problem correctly. So wouldn’t it be better to add the following step to the bug fixing heuristic described above:
- After you have completed the test case reproducing the issue you check that one in into the VCS (although this is not buildable, due to the failing testcase!)
- You add a comment to the ticket that the current revision lets you reproduce the bug
- Continue, like described above
Perhaps I am to addicted to traceability but I really like to be able to track what’s going on, what’s going wrong, what could be solved how and so on.
What do you think about this?
