malvasia bianca

Up until a few months ago, the way I would, say, add a new book to the list of books I’d read was to edit a file (WriteDbc.java), compile it, and run its main, which would write out HTML pages directly from the Java data structures that it built up.

Then, a few months ago, I stuck in an SQL database as an intermediary: the main routine would write out the data structures into an SQL database (first wiping out the previous contents of the database!), and a separate program would go from the SQL to HTML. Which is ridiculous in the long term – the point of a database is to store data, not be a transient repository – but a useful step nonetheless.

But now I’ve finished a CLI tool to let me edit and add to the SQL database directly. So yesterday, I did svn rm WriteDbc.java after writing the data out one last time, and today I used the CLI tool for the first time in earnest (to reflect the fact that I just finished this excellent book and am about to start this one). And it works just fine.

Things that I learned while writing the CLI tool:

• I might have fantasized that I had a traditional layered architecture for this, with at least hints of a persistence layer, an abstraction layer, a presentation layer. Well, I don’t. One manifestation of this is that the CLI tool and the SQL-to-HTML tool share essentially no code – my wrapper over java.lang.sql, a couple of simple data types (a date wrapper and a book title wrapper), and that’s it.

  The CLI tool is driven by a table mapping entity types (book, author, etc.) to SQL tables and, for each entity type, another table mapping keywords to columns; that’s it. Really simple to use, really simple to modify: after the first three entity types, adding each new one took less than five minutes. (With one special case exception.) And going from a CLI tool that could add a new entity (a book that I’d just bought, for example) to a CLI tool that could edit an existing entity (reflecting the fact that I’d finished the book, for example) took maybe 30 lines of code. It’s just not complicated enough to justify more architecture than that.

  One benefit of this is that, if I want to dabble in another language, I’ll be able to easily rewrite one part without having to touch the other. So if I want to, say, rewrite the CLI tool in Ruby, nothing’s stopping me (other than the fact that I don’t know Ruby yet) – I even have acceptance tests in place to back me up. Yay.

• The one acceptance test surprise was that the acceptance tests went from taking a minute to taking two minutes; what’s worse, they would hang occasionally. The reason for the time increase was that, if an acceptance test wanted to test a scenario with 15 entities, it would have to run the CLI tool 15 times, starting up the JVM and connecting to the database 15 times. And, it turns out, doing so can turn a 3-second test into a 15-second test. So I modified the CLI tool to accept multiple commands in a single invocation; now the acceptance tests only take about 45 seconds.

  The sporadic hanging still bothers me. I can’t see how it could be some sort of infinite loop in my code, though I’ve been wrong before. I could imagine that I’m not freeing SQL resources appropriately – I have tests verifying that I’m releasing them in many circumstances, but it could well be that there’s some place where I don’t know that I’m supposed to release resources. (I would have naively assumed that resources would be freed when the process exits, so bugs like this wouldn’t matter, but maybe not.) Or maybe there’s a problem somewhere else.

I am glad that this is over with (there are a few loose ends to tidy up, but nothing major or urgent): it will be very nice to shift back to taking an hour to implement a feature that will make an actual visible change on the output, instead of taking rather longer than that to change the guts in a non-user-visible fashion. Especially since I have some features that really need to get implemented: I can’t add one of the books that I’m currently reading to the database, for example! Next weekend I will be able to, though, unless I’m too busy to be able to program at all.