Test Driven Development is Cool
For those who are unaware test driven development (TDD for short) is a method for making software where you start by developing automated tests for your program then write your program to pass those tests. It's a process which I'm a big fan of. Since I've been using TDD for a while now I figured I could share a few comments from my experiences using TDD.Test Driven Development Makes you Think
More specifically, TDD forces you to think about how the code you're writing will be used. This is especially useful when designing an API. A typical process begins with you designing an interface for whatever you're doing, then writing tests that use that interface. Remember my earlier article on web services? For a short summary; I discussed how you should try automate common work within an API to minimize the code required to do useful stuff with it. I used Testlink's XML RPC API as an example of a suboptimal API. It's not awful but using it results in a lot of code bloat and, as a result, poor performance.
In order to develop a good unit test you need to first think of how your API is going to be used. Each potential use for your API becomes a single test. It's very easy to come up with a few dozen tests for a relatively simple API. Doing this you'll see what parts of the API get re-used very quickly, i.e. boilerplate. And if you're developing a lot of tests I think you'll begin to see why I refer to such code as menial bullshit. And when your API makes you angry you'll begin to think of ways of making it easier to use.
Testing is Faster with TDD
Let's imagine you're making a game. Games have a huge variety of data that needs to be stored. Sure, the absolute quantity of data isn't all that big (relatively speaking, financial and analytics software have games beat by lightyears) but there is a lot of very different data. Some things such as sound samples, music, 3D models, 2D images and such have established formats others such as game levels, entities, AI code, saved games, recorded gameplay footage (for games that still do that, I miss old FPS games...) tend to vary from game to game. And all of that data needs code to handle loading it into the game and saving it from creation tools.
Using a conventional dev/test cycle would mean creating the code, integrating it into the game and tools and finally creating content using the tools before being able to do any effective testing of the code. And testing the resulting data means opening up the game and tools and manually using them to see that the data is correct. In-game testing in particular tends to require going through a lot of extra effort (i.e. starting the game, getting to a testing level, and setting up your test.) That's slow.
Using Test Driven methods means deciding what sort of data your file format will initially contain producing some mock sample then writing unit tests that use your game's file loader to load the file then verify the data they produce matches with what you expect. Each unit test works with a small amount of data to verify a specific part of your file loader works correctly. You can also easily create additional tests to check that different combinations of features in your file loader work correctly. Obviously there's the risk of combinatorial explosion but that's easy enough to avoid by limiting unit tests to covering features in isolation and obvious feature combinations. This will give you the most bang for your buck. Other tests can be added if bugs are discovered.
The speed advantage comes from eliminated testing overhead. When you need to test your code you simply run your unit tests. No need to run your asset creation tools, or the game or anything else. The only limit to the speed of your testing is the speed of your code. The best part, if you're skeptical about TDD, is you don't need to fully buy into TDD to do this. Just supplement your manual testing effort with unit tests. Even better, if you automate your product's build process then you can have your tests run automatically too by making your build tool run the tests right after a successful build finishes. Your entire dev team can then be notified about failing tests just like they would be for build breaks.
Test Driven Development has Problems
So far I've painted an extremely rosy picture of TDD. However, like all processes, it's not perfect. Far from it in fact. There are a good number of caveats about using TDD.You Must Make Testable Code
Remember my comment on testing being made faster by TDD? This is only true if your code can easily be unit tested.
This is both good and bad. On one hand testable code tends to be very easy to work with since it doesn't have a lot of baggage that prevents it from being picked up and used. On the other hand, that baggage may be necessary for your code to perform optimally. Nine times out of ten however you're best off developing testable code. And if you think you think your case might just be that one you're probably wrong. Check first. This applies to all code regardless of whether you're making web applications or embedded software on a high performance real time OS
One tip: modularize. This means not only conventional modularization by keeping objects and procedures from being overly inter-dependent, but also moving parts of your project into libraries to make it easier to build tests without pulling in huge quantities of code. This pushes compile times downward for when you're testing and allows faster testing.
You Still need Conventional Manual Tests
Unit tests and other automated tests that are commonly produced by test driven development are great for verifying software interfaces and behavior. However, unless you're producing automation tools, you're not going to find anywhere near the full set of issues.
For example, you can't test that your software's workflow fits your users' expectations. Automated tests can't verify that a game is fun. Artistic merit is impossible to judge with software alone. The most bizarre interoperability issues typically only crop up during general use of your software and require manual testing. For a completely contrived example on a smartphone, when playing music on a media player, visiting a flash site on one browser tab and visiting an HTML5 site on another and using the GPS causes the phone to crash at random. But only for certain commuters in the big city during rush hour.
Unless your test developers are completely evil and have a lot of time and money you're not going to get these kinds of things happening in your automated test. Particularly if you follow conventional best practices for unit tests which tests stuff in isolation. The only way you're going to find these issues is if you have manual testing reflective of how your real users would use your product.
Test driven development works as an excellent complement to ad hoc manual testing. Without TDD you're unlikely to have the tester time available to do the kinds of heavy exploratory testing that would find issues like our phone crash during a morning commute and without skilled testers doing that sort of testing you're unlikely to get the critical information need to fix the issue.
