I thought I'd share a positive experience I had today with unit testing.
If you don't know what unit testing is, essentially it's an automated process used to ensure that your code actually does what you think it does. It's a fundamental rule of extreme programming, has an active cult following, and has spawned test-driven development.
Without getting into an essay on the background of unit testing, how it's useful, or how to set it up with Flash or Flex, I just wanted to relay my experience this afternoon.
Someone had contacted me about a possible bug in a piece of software that I wrote. Naturally, I assumed this was nonsense (after all, who writes buggy code anyway?). It turns out there was an edge case that I never thought would occur and it was causing erroneous results. I was given the steps to consistently reproduce the problem, and immediately went to work.
The first thing I did was create a new test. I coded up the scenario that was causing trouble and added it to my test suite. After running the test suite, I could see that, indeed, the code was failing.
The next thing I did was fix the problem. In this case it actually turned out to be pretty minor and was fixed in no time.
After I thought it was fixed, I ran my test suite again. All of the tests passed this time. I committed the code to the repository, and let the person know that the bug was fixed.
Lather, rinse, repeat, as often as necessary.
How do I know the bug was really fixed? Because all of the tests passed.
How do I know I didn't introduce new bugs while fixing this one? Because all of the tests passed.
Can I prove it was fixed? Yes.
In hindsight, however, I probably should've thought of this particular test case on my own. After all, part of test-driven development is to create your tests before you create a single line of code. I had a similar test for the situation, but not one that recreated it perfectly. The lack of a test meant I let a bug slip through the cracks because I thought I had everything covered. Such is life.
So, here comes the moral of the story:
Come up with as many tests as you can on your own before you even start a project. The more meaningful tests the better - try to think of all of the edge cases you can and attempt to break your code any way you know how. Whenever a bug report comes in, try and turn it into a reproducible test case. This will give you confidence in fixing the bug (so you don't accidentally break anything in the process), and assure you that the bug is indeed fixed.
I'll leave you with this Monty Python quote. Pretend the French Soldier is the test suite, and King Arthur and his Knights are the bugs:
French Soldier: Ha Ha! Take that, you silly bed-wetting Englishpersons!*flings cow*
King Arthur and Knights: Run away! Run away!
6 Comments
yeah unit testing is good. in AS3 because you can have default arguments for a function, if you have a constructer that looks
like this
public function DebtStream(pName:String, pStartTime:int = 0, pStreamLength:int = 0,pAmountInDebt:Number = 0,pGrowthRate:Number = 0, pLimit:Number = 0) {
you have to write mutiple test files for it since you need to replicate the different setup routines, for each edge case. this caught me out today. granted multiple files is not necesarely the only sollution, but without changing the way the test case instance is constructed in your test case (meaning not using setup to create the instance) it is the best IMO.
Posted by: johannes | February 16, 2006 6:41 PM
Your Monty Python anology was what really brought things together for me. I think we all should strive to relate programming towards Monty Python movies and sketches whenever possible ...
Posted by: Wally Atkins | February 17, 2006 2:08 PM
If only we programmed in Python all day (the language named after Monty Python)!
I'm glad that all it took was a flung cow for you to understand. :-)
Posted by: darron | February 17, 2006 2:21 PM
"How do I know I didn't introduce new bugs while fixing this one? Because all of the tests passed."
Not that I'm trying to say unit testing is bad, but How do you know you have all the possible tests now? I assume you thought you had all the possible tests before this bug popped up.
Just wondering!
Disclaimer, I've never used unit testing and am interested in the subject :)
Posted by: viewer | February 22, 2006 1:14 PM
I don't know that I have all of the possible tests now. In fact, it's not really possible (unrealistic) to be able to come up with a 100% "complete" suite of tests, especially for large pieces of software - this is one of the down sides to unit testing.
However, I know that when I fixed the problem I didn't introduce a new bug because all of the pervious tests passed successfully. Everytime I find a bug I create a unit test for it, so I can say "bug free to the best of my knowledge" with total confidence. That's the best unit testing can do - unit testing can only show the presence of errors (when tests fail), not lack of them (because it's hard to create a test for every scenario under the sun).
Posted by: darron | February 22, 2006 7:59 PM
Hey Darron,
In terms of Unit Testing how does one get started in terms of ActionScript/Flex Bulder 2? I love using Eclipse for Java development so naturally Flex Builder kicks ass as well but I've not done any unit testing before. So what packages/plugins should I download? And which books or web tutorials do you recommend? Perhaps I should start by reading about Xtreme Programming....
Your site rocks! Cheers!
Posted by: Ralph Caraveo | February 23, 2006 12:25 AM