Digging yourself out of a debugging hole
Real-world suggestions for not losing your head.
I've had the pleasure of teaching both Plone and Java development (and in some cases, software development basics) "on-the-job" to several people I've worked with. Yesterday, I read this great article, and found the following advice particularly good:
2. Good programs do not contain spelling errors or have grammatical mistakes. I think this is probably a result of fractal attention to detail; in great programs things are correct at all levels, down to the periods at the ends of sentences in comments.
“Aw, c’mon! You’re kidding!” You might think that nit-picking like this is beneath you, whereupon I will start pointing out errors in your code. It was embarrassing the first couple times this happened to me.
This is very true. There simply is no excuse for leaving your code in a mess, and that includes comments, indentation, adherence to naming conventions, and so on. It may seem unimportant, or you may kid yourself into thinking "I'll fix that up later" (of course you won't), but trust me - if your code is a mess, making it work correctly is harder for you, and harder still for anyone else who may be working with your code.
I'm sometimes called over to help developers who simply "can't make it work". And usually, the code they're staring at is in a terrible state. It may have started out fine, but with repeated attempts and making it work, and as the level of frustration has grown, it all falls apart.
Some tell-tale signs include:
- Grossly confusing indentation
- Meaningless variable names (such single letters, placeholders like "foo" and "bar", or variables named after the programmer, like "mike1", "mike2"...)
- Variables that used to refer to one thing, but now is used for something completely different. The name is unchanged, of course.
- A mixture of capitalisation and naming conventions.
- Either a total lack of comments, or blocks of code commented out seemingly at random.
- Outright syntax errors. Sometimes on big projects, I've seen developers get build errors from the IDE, but because a subset of the code will compile. Hot-replace of code is particularly dangerous here.
The first thing I do in these situations is to take over the keyboard and clean the code up, while the developer is watching. I'll try to ask questions about what they intended, delete any unreachable or commented-out code, line up all the indentation, rename variables, and put in comments. At this point, the mistake is usually obvious.
Everyone is capable of doing this. Programming is a discipline of repeating existing patterns. If you're a developer on a team, or working with software you didn't write entirely yourself, there will be conventions and patterns to copy. Unless you understand those things, you don't stand a chance, and unless you keep your code in good order, you are just wasting time. The error that the interpreter/compiler sees is probably nowhere near the snippet you're staring at.
To finish off, here are some useful debugging tips that apply no matter what technology you're using. Invest the time in becoming good at debugging, and your future as a programmer will be much brighter.
- Make sure everything builds cleanly. If your IDE (or Pyflakes) is telling you that you have errors, fix them, even if they're not directly related to what you're doing. So long as your project is in an inconsistent state, all bets are off. If you don't have any way to find this out (e.g. you're writing Python code and you don't have an IDE or use something like Pyflakes to syntax check your code), get that set up right away. Putting it off does not save you time. Quite the opposite.
- Tidy up the code. Follow the prevailing conventions in your project, and make the code look pretty. Add comments for things that are non-obvious. Make sure variable names are sane. If you're going to ask someone for help, you'll be wasting their time unless you can clearly explain what you're trying to do and what your code is doing. It's quite likely that in trying to explain it to yourself, you'll find the mistake.
- Don't be too clever. If your code is difficult to understand because you've tried to accomplish ten things in one line of code, if you've got complex conditional expressions with lots of nested brackets, if you've called arcane APIs that you're not very comfortable with: get rid of them. Write predictable, easy-to-understand code. You can always optimise later if you need to. I see this in particular with relatively inexperienced programmers who've started to learn about optimisation, and start doing insane things like re-using the same variable for multiple things in order to save stack space, or try to avoid function calls because they read that function calls can be expensive. Stop it. It doesn't make any difference. Write code you actually understand, and everyone will think you a better programmer because they will understand the code too.
- Make sure the code is actually executes. One way to do that is to deliberately raise an exception or trigger an error, and make sure that this error is actually caught. This is a very useful technique for configuration files: if some configuration in an XML file, say (like a Zope ZCML file) does not appear to work, remove a closing tag or introduce some other kind of syntax error, restart your application, and check that you actually get an error. If your code or file isn't being used at all, you're probably looking in the wrong place.
- Learn to use the debugger. In Python land, that means the pdb module. If you're using an IDE like Eclipse, there's a built-in debugger - learn how it works. Sometimes if you're debugging an application that runs in an application server container, you'll need to learn about remote debugging. If you can't step through your code, you can't figure out what's going on. Putting off learning to use the debugger is another example of trying to save 15 minutes in the short run, costing you hours and hours in the long run. Yes, you may be able to get away with using "print" statements for tracing, but this is much slower and more error prone (e.g. if you need to restart your application each time you want to check a particular variable), and sometimes doesn't work (if the program isn't connected to a console).
- Write unit tests. For any module you write, you should have a unit test harness set up, so that you can quickly and easily add another test for the code you're trying to debug. It should take no more than 10 lines of code. If your application is slow to start up, the time you spend waiting for 10-15 restarts will probably be greater than the time you'd spend writing a test that executes in seconds. Test-first development is an ideal that not everyone manages to adopt, but having the testframework in place first is definitely worth it every time. Unit tests are also a good way to invoke a debugger in a predictable way. Finally, writing a unit test makes you think about the inputs and outputs to your function, and forces you to write code that can be run largely in isolation, without depending on too much other system state. That in turn reduces the chance of errors.
- As an option of last resort, make a copy of your code, delete it all, and write it again, without looking at the original. You may've just made a stupid mistake that you won't make the second time around. Or you may choose a slightly different approach that happens to work, maybe because you made an incorrect assumption previously.
- If that doesn't work, go home, have some time off, and try again in the morning.