Related link: http://www.cafeaulait.org/oldnews/news2005February1.html
The only technical web site I regard as a daily must see is Elliotte Rusty Harold's twin Cafe con Leche (XML) and Cafe au Lait (Java). (I like
planet.xmlhack.com,
and the blogs of Tim Bray and Dare Obasanjo too.)
It has been interesting to see Harold move from being quite sceptical of automated bug-finding tools to becoming a big fan. Java has great bug-finding tools: in part this is because it needs them (for example because you might be writing multi-threaded code, and therefore have to deal with synchronization issues), and in part because Java provides enough information for programs to detect a whole classes of bugs. Some Python fans might also say that Java has too many lines of code, which cause extra bugs in the first place.
There was a lot of discussion a while ago on the virtues of XP (Extreme Programming) and more recently on the value of terser languages such as Python versus Java. One of Java's strengths is that it can be reasoned about by automated bug finding tools (largely because it does not have pointers and because it uses automatic memory management).
It strikes me that different projects, designs different application areas, different development environments, different test environments, different programmers, and different programming languages each will have unique characteristic errors. So it is unreasonable to generalize too far about development methodologies (strong typing is better than test based is better than automatic bug detection is better than rigorous design, etc). The first thing to do is to figure out which kind of errors your software is prone to, then to implement the appropriate prophylactic.
And there is the rub: if you are inexperienced, or the project is starting from scratch, you will have to rely on the expertise, intuition or superstitions of others. And because you use different programmers, etc., etc., their silver bullet may not be as effective for you. (It may be more effective, if you are lucky.) But you have moved out of the realm of software engineering (i.e. how to extract and apply objective measures to replicate successful projects) and into voodooland.
An approach I took a few years back, which I think is pretty workable when starting brand new development and team, is this: start off implementing a little of as many different strategies as possible (for Java, this would be Eclipse, JUnit tests, JLint/Findbugs or PMD, an IDE tool, a workable human communication system, a good version control system, periodic scheduled audits for i18n, accessibility, memory profiling, and usability, a skills-enhancement program, an Open Source library research effort, and so on.) Then cut back on the ones that are not providing value compared to their cost, and ramp up the ones that give bang per buck.
For example, we started off putting in assertions about method arguments. This was before Java provided assert, and it was not very workable nor did it find the kinds of errors we made. Presumably strong typing caught many of them already. So we cut back to just checking for null-valued arguments: that has been workable and effective.
There is an educational effect at work too. When a programmer makes certain errors, and then has to fix them, they often learn not to make those errors. It is not just moving from being a novice to being a guru; it is being habituated by the particular feedback against making certain errors. So the silver bullet can change: after using the automated testing tools for a year or more, they may yield smaller results, because the programmer is more aware of the kinds of errors they detect and because they have already been acted on. But the same is true of unit testing: putting a test in place detects initial errors fast, but after then only detects occassional-introduced errors. Unit testing trains the programmer too.
I rarely see this mentioned. Software development methodologies are usually couched in terms of improving software quality. But they often act in the medium term by improving programmer quality. To a certain extent, a prudent project leader should, rather than thinking in terms of processes to improve software quality, think in terms of processes to improve programmer quality.
Silver bullets don't kill errors, people do.
Categories
Web
Comments (2)
Read More Entries by Rick Jelliffe.
!(unit testing == silver bullet)
A probably well-known caveat about unit testing: It depends on what the test method body looks like to know whether it actually helps the programmer improve his code. If the programmer is the one writing the code *and* the test, he can easily be blind to a set of conditions that will turn into a bug later. Sometimes, consciously or not, people try not to find the bugs in their code.
If you want good testing to improve the programmers, a *different* programmer should write tests for the programmer's method. (Eek! As a programmer, I can't believe I said that; I wouldn't want to have to do that in practice myself.) Maybe it could be QA programmers who writes the test methods; maybe they would actually *like* writing them for others' code.
;-)
lone werewolf hunting
This is an excellent post (typos and all). The title is particularly apt and catchy.
As a slave to process, I've made a habit of creating documents as development deliverables. I initially used templates I pulled from a standards body, but over time I started re-arranging the document structures to better fit the way I used them. They come more naturally now. I think that sort of personalized adaptation can make us better.
As an independent I'm free to do what I feel is right, but I know not everybody is so fortunate. Hopefully this sort of wisdom can penetrate the purveyors of more rigid and dehumanizing methodologies, so that developers are slaves to their own habits more often than to somebody else's square filling exercises.