malvasia bianca

A week and a half ago, I noticed that the latch on my MacBook Pro had stopped reliably holding the lid shut. Which isn’t a huge deal—I mostly use it at home, and turn it off between uses—but at some point it would annoy me. My new organized self recognized that the most likely reason for me to not get it repaired immediately was plain old procrastination, so I looked at my calendar, decided now was as good a time as any to be without it for a week, loaded up a few dozen podcasts that I was curious about to tide me through my iTunes-less period, and prepared to have it repaired.

I wasn’t sure if the Apple store could fix it on the premises, but I figured I’d give it a try, so I made an appointment there. As always, they weren’t ready to help me when my appointment time arrived, but at least this time they were ready to help me only 20 minutes or so after the time instead of most of an hour later. (Though even 20 minutes late on a random Thursday morning suggests that their scheduling algorithm could be improved.) They said it had to be shipped away to be fixed; I figured I might as well let them take care of the shipping instead of calling to ask for a box, so I put it in their hands. (This was on a Thursday.)

I’d been under the impression that they used fast shipping, but when I checked Apple’s web site on the weekend, they said they still hadn’t received it. On Monday, same story. On Tuesday, a (quite pleasantly) different story: not only had they received it but they’d already fixed it and were preparing to have it shipped back. On Wednesday, it was being shipped back, there was a tracking number, the shipper claimed it was out for delivery, and late that morning it had been delivered.

So I figured I’d get a phone call from the store soon. By 3pm, though, no phone call; I called them, they confirmed they’d gotten it, but they wanted to run some tests on it to make sure it worked, and couldn’t tell me how long it would take. I questioned this; the person on the other end noted the annoyance in my voice, went away for a minute, and came back and told me that I could just pick it up without them testing it if I wanted. Which I planned to do; an hour later, I got a phone call saying that they’d finished testing it.

I showed up at the store at 5:20 or so; somebody came over to help me, typed on a computer, and told me to wait a minute. No problem, that’s why I downloaded all those podcasts. (In an amusing bit of synchronicity, I was listening to a portion of a Retronauts episode where they were talking about the recent XBLA port of Marathon 2. Sounds like they’ve fixed the nausea issues, maybe I should give it a try.) Five or ten minutes later, somebody else checked on me, but wasn’t concerned that I was still waiting. Another five minutes, and I talked to the original person, and he was more concerned. Eventually, twenty minutes or so after I’d entered the store, somebody emerged with my laptop. And, as it turned out, with one of its DIMMs removed and placed in a separate bag. Another three minutes or so and I was out, barely in time to pick up Miranda.

Thoughts:

• Ironically, some of the things that annoyed me were a result of systems helpfully providing information. If Apple hadn’t given me a mechanism for tracking where my laptop was, I wouldn’t have known that it apparently took four and a half days to get to the repair depot, or was sitting in their store for five hours before they let me know.
• Some of the employees did go out of their way to try to be helpful. Some didn’t, but none were actively annoying.
• The system they work within, however, is actively annoying. I’m tired of Genius Bar delays. (I’m tired of the name Genius Bar.) I don’t want them taking extra time in the store doing checks: if the repair depot can’t repair things properly, they should fix the problem at the repair depot instead of trying to inspect in quality after the fact.
• And I really don’t want them removing my memory when I ask them to fix a hardware problem that is clearly unrelated. Fortunately, I installed the memory myself, so I had the correct screwdriver to put it back in; otherwise, I would have been a lot more annoyed at the situation. (Especially given that they’d already come close to making me miss a daycare pickup time, so I didn’t have time to wait around for them to undo their work.)
• Next time, I’ll just avoid the store and send it in myself: that part of the process seems to be nice and speedy. (Unless the store really did send it in fast and Apple’s computers erroneously claimed that it was in transit when it had been received and was waiting for repairs.) Actually, next time I’ll ask around first and see if there’s a local Mac shop that can repair it under warranty on the premises, so I don’t have to wait for a few days.

I’m not too annoyed at the whole situation; it’s not going to significantly decrease the chance that I’ll buy another computer from Apple (if for no other reason that I don’t trust other manufacturers to do any better), and there were some actively good aspects of the situation. But they definitely missed some simple ways to make a good impression on me, or at least to fail to annoy me.

I recently learned that a Bay Area Go Players Association has been created, with the goal of ensuring that there’s at least one go tournament a month in the bay area. (And, so far, they seem to generally be holding their tournaments quite close to where I live.) I missed the first tournament, but the second one was this weekend, so I played a game of go for the first time since last summer.

Quite pleasant. Too many handicap games, and it was a bit odd having my AGA number of 6269 mark me as the person other than the organizer who’d been an AGA member the longest. And I didn’t do too well giving handicap stones; clearly I should be more careful in the future. But I had a lot of fun, and I’m planning to go again: going to the go club every week is a bit much for me, but playing in a tournament once a month (unless I have something else going on that weekend) sounds about right. And I think I’m doing a better job of trying to look at the board strategically; if I can build on that, I’ll enjoy the games even more.

I finished reading The Art of Agile Development, by James Shore and Shane Warden a few weeks ago. It’s a quite good book: if you’re looking for a well-written, prescriptive guide for how to do XP, this is what I would recommend.

Though I won’t go into the book in general any more than that. But, a few thoughts that it triggered:

One thing I liked about the book’s discussion of “once and only once” (pp. 316–317) is that it spent some time on the first “once”. I’m used to thinking of the sentence in terms of eliminating duplication, which is great. But the first part of that sentence is important, too: as they say, “don’t just eliminate duplication; make sure that every important concept has an explicit representation in your design”. That way, you have a place to store behavior related to that concept: e.g. having a Dollar class may not seem like a big win over, say, just representing dollar values as integers, but it gives you a place to stash code for formatted output, or parsing, or conversions to related types.

I also really liked the discussion of levels, rhythms, and frequencies of refactoring in the “Incremental Design and Architecture” section. (pp. 321–330.) They distinguish between standard incremental improvements and “breakthrough” changes: the latter are “idea[s] for a new design approach, which will require a series of refactorings to support it. … Breakthroughs happen at all levels of the design from methods to architectures.”

After which they talk about how frequently you do various sorts of refactorings, and how to make time for them. At the method level, “Method refactorings happen every few minutes. Breakthroughs may happen several times per hour and can take 10 minutes or more to complete.” At the class level, “Class-level refactorings happen several times per day. Depending on your design, breakthroughs may happen a few times per week and can take several hours to complete.”

The latter raises the question: several hours is enough to make a noticeable blip in your team’s work for the week, so where do you find the time, if you see something that looks valuable but isn’t on the shortest path to completing a feature? Their answer: “Use your iteration slack to complete breakthrough refactorings.”

And then there are architectural breakthroughs; I’ll quote them at length on the subject (p. 325):

In my experience, breakthroughs in architecture happen every few months. (This estimate will vary widely depending on your team members and code quality.) Refactoring to support the breakthrough can take several weeks or longer because of the amount of duplication involved. Although changes to your architecture may be tedious, they usually aren’t difficult once you’ve identified the new architectural pattern. Start by trying out the new pattern in just one part of your design. Let it sit for a while—a week or two— to make sure the change works well in practice. Once you’re sure it does, bring the rest of the system into compliance with the new structure. Refactor each class you touch as you perform your everyday work, and use some of your slack in each iteration to fix other classes.

Keep delivering stories while you refactor. Although you could take a break from new development to refactor, that would disenfranchise your customers. Balance technical excellence with delivering value. Neither can take precedence over the other. This may lead to inconsistencies within the code during the changeover, but fortunately, that’s mostly an aesthetic problem—more annoying than problematic.

Definitely something to think about: over the last few years, my team has made some progress in terms of lessening the technical debt that we’re adding to the system, but we still haven’t done nearly as well on that as I’d like, and we’ve done a bad job in terms of actually eating away at the technical debt. (As you might suspect from my using the term “last few years”.) We’re just not good at maintaining a good refactoring rhythm at the various scales that are necessary; maybe the authors’ advice on the subject will help, by giving an idea of what our behavior would look like if we were successful.

Note also the importance of slack in carrying this out successfully.

I recently was charmed (?) to discover that Jonathan Coulton likes big butts and he cannot lie. For those of you who want instant gratification, here’s a YouTube version, but the studio version linked to above has somewhat better audio.

On that vein, I don’t believe I ever linked to Alanis Morissette discussing her humps when that video was circulating through the blogosphere? I suppose I’ll take this as an opportunity to remedy that oversight.

I’ve now hit the 200 kanji mark in my quest to memorize all 1945 Joyo Kanji. Where by “memorize” I mean that, if my memory is working well (which it usually is), I can write down the first 200 characters (actually, the first 217 characters: I’ve been procrastinating on writing this blog post) in the order given by Hadamitzky and Spahn, in correct stroke order.

I don’t claim to have the readings and meanings memorized as reliably for all of them: they’ve all gone through my flashcard box (as have several hundred other kanji), but sometimes I forget. Still, I’m doing pretty well there, and at least it’s a line in the sand: if I run into evidence that I’ve forgotten a meaning of one of those characters, that’s a sign that I should take the corresponding card out of the box and put it back into my daily vocabulary drills.

But the point of memorizing isn’t really to memorize the meanings: it’s to make storage locations in my head where I can later put the meanings. I never get different English words confused, even if they’re unfamiliar to me, whereas, most of the time, when I “learn” a new kanji, if I see a kanji that looks kind of like it in an unfamiliar setting, I’m not sure if it’s that kanji or a different one. (Or I might not even recognize that kanji as familiar at all!) Whereas I can now reliably tell those 217 kanji apart from each other and apart from the thousands of kanji that I don’t have memorized. (The Joyo list is far from a complete list of all kanji that are used.)

So I’m pretty proud of myself! But I still don’t really believe that I’ll make it through the whole book. I memorize a page at a time (7 kanji per page; a side benefit is that I’m better at recognizing multiples of 7 than I once was…), adding one or two new pages a week. Say that averages out to 10 kanji a week (and it’s taken me more than 22 weeks to get this far); if that holds up, I can do about 500 a year, so I have three and a half years ahead of me. Which is a pretty long time; a lot can change between now and then.

I’m also not sure how the review process will scale. One reason why I’m sure I have them memorized is that I periodically go through the whole list and trace them all out with my finger. But it takes a long time to do just 217 of them; pretty soon, I’ll only rarely have time to write down the whole list. (I certainly don’t have time to do that most days today.) I break them up into groups of 70, and go through one or two of those groups at a time (a small enough chunk that I can do it most evenings after putting down my book but before falling asleep), but I think it’s important to periodically rotate through the entire list. Hmm, I guess I could do that in multiple settings? E.g. every night go through the current group and one other group where I cycle through the other groups in order?

The other Japanese-related thing I’ve done recently is place my first order from Amazon Japan. I’m still not particularly fluent with entering Japanese text using the computer, but I could do it well enough to muddle through. For the curious, my order consisted of a few volumes of Hikaru (I now have the first five in Japanese), a few puzzle books from Nikoli (we’ll be going on vacation in a few weeks, and I wanted some nurikabe puzzles to work through, which was actually the impetus for the order), an art book from Okami (which I felt a little bit silly about ordering, but which turns out to be totally beautiful; Miranda likes it a lot, too), and two DVDs: a pre-Ghibli Takahata movie and a Juzo Itami movie. (Most of his movies are almost as out of print in Japan as they are in the US, alas.) Comics are a lot cheaper there than here; DVDs are a lot more expensive, which I wasn’t expecting.

The most fun part was receiving the “your order has shipped” e-mail and realizing that I could actually make sense of most of it. (It, of course, helps that I’ve received hundreds of those from the US branch…) And then I went to the “your account” web page to track my order, was confronted with a drop down with five characters (four kanji and one kana), and I knew that they were pronounced “saikin no chumon” and meant “recent orders”. (Not “open and recently shipped orders”, as the US site says.) So that was pretty cool! Since then, I’ve tried to look at Japanese text in other contexts, and realized that my being able to piece that together was very much an aberration, but it was a pleasant one.

They only have one shipping rate for orders to the US; it’s pricy if you’re just ordering one thing (about 29 bucks), but the incremental cost of extra items isn’t too high. (A little less than 3 bucks each.) And the good news is: that shipping rate is fast. I placed my order on Monday night (US) and they arrived on Friday. So I didn’t need to order so far in advance of the trip after all! I’ll definitely do more of this in the future.

Warning: While I don’t intend for anything here to be a spoiler for the “primes and fractions” post I wrote before it (the post following this if you’re looking at the front page), it’s possible that you’d have more fun working out that brainteaser if you didn’t read this post first.

At the time when I wrote my last post, I’d gotten as far as figuring how the fractions in question turned into a computational machine of a much more traditional form than the form in which it’s stated there. After writing the post (and eating lunch), I wrote down that machine on paper, and went through a series of simplifications. (Refactorings, basically, with the extra benefit that I could increase the power of my computational machine, because I didn’t have to restrict my actions to those that could be carried out by the original computational model.)

After an hour or so of that, I had a much simpler model; I couldn’t yet completely see how the prime numbers were going to appear, but I’d gotten a lot closer, and in particular the concept of divisibility had appeared. Hoping to get a bit more insight, I decided to run a few dozen iterations of my new, simpler model.

And, of course, I started getting bad answers; if I’m remembering correctly, I ran into an endless loop or something, and at any rate I certainly wasn’t computing prime numbers. Oops.

Which was frustrating. I mean, I’d been pretty careful, breaking the work up into explicit steps, double-checking my work after each one (and frequently finding mistakes, which in retrospect should have raised warning signs). But I was pretty obsessed by then, so I wasn’t going to let a little problem like having a mistake somewhere within the last hour or two of tedious work stop me!

To recap my progress so far: I started with a computation, I transformed it to simplify it and to improve my understanding, and I made mistakes. Which is something I have experience with when programming, and I already know the answer in that context: I should be doing formal refactorings backed up by running tests after every step. I’d been doing an okay good job of the former, but the latter was too laborious to carry out frequently by hand. (For that matter, it’s just as easy to make mistakes when running through the algorithm by hand to test it as it is when transforming it by refactoring!)

So I brought my work over to the computer: I wrote a Ruby version of the initial state machine, with a driver program that ran through as many iterations of the steps as I wanted. And then I refactored the state machine, reloading it into irb after every change, and saving it into a new file when my little changes added up to a more significant one (e.g. eliminating a state), just so I could look back on what I was doing. I didn’t have formal unit tests: I just ran the program for a few thousand steps after every change and checked that the primes were appearing, on the theory that I was unlikely to make a mistake that let the algorithm still work. (In retrospect, maybe that was a mistake: it would have been easy enough to turn that manual test into an automated test.)

And it went a lot faster. I didn’t have to spend as much time double-checking my work: I could just make changes and run the test. I could work in smaller steps, so where on paper I would have composed two transformations in my head (involving changes to multiple variables at once), I could just write them out as a program one after the other and then merge changes to variables one at a time. If I wanted to eliminate a state, I’d just paste the code for that state into every place where it was used, and then do transformations after that. If I wanted to reorganize the control flow by moving a block into all the branches of a conditional, I’d just paste it into all the branches, then simplify the branches one at a time.

I still made mistakes, but even without the tests I made fewer mistakes than I’d been making on paper: it’s really a lot easier to avoid mistakes if you just cut and paste whenever you’re moving operations around, and then break up your subsequent simplifications into smaller steps. (At some point I cross-referenced my work with my paper copies; I’d made two important mistakes pretty early on, and in retrospect I’m surprised that so much of the mathematics managed to survive in the paper version even with those mistakes.)

So now I have a proof; yay. But I’m actually thinking that there are some lessons here that the mathematics community could learn from: I worked faster when I started using the same techniques that I use to write correct, readable code as quickly as possible, and I made a lot fewer results at the same time. And who wouldn’t want that? In particular, I think an automated engine that lets you correctly carry out refactorings on steps of your proof could help a lot, done right: it could let you do the relatively mindless work faster, so you could spend more time thinking at a higher level, and you could spend more time quickly exploring different paths in that mindless work to see if any of it leads somewhere unexpected.

And you would make fewer mistakes. My impression is that most math papers contain mistakes in their proofs; this doesn’t necessarily mean that the results are wrong, but sometimes it does, and even if they aren’t, it’s still good to get the details right. It would also make it easier to be honest with yourself about when you’re handwaving and when you’re being precise; I certainly wouldn’t say that handwaving is inappropriate in all circumstances, but you want to take care when doing it. (What I’d really like is for papers to be published in a format where the printed version contained the steps that are most useful for expository purposes but where there’s a more complete computer-verifiable version of much or all the proof.)

This isn’t a new idea, of course; I worked on a project called “VLISP” the summer after my freshman year of college that, among other things, had a theorem verification engine, and the idea wasn’t new then, either. (The proof of the four color theorem was almost a decade and a half before that.) So why hasn’t it taken off? I’m not sure, but here are some ideas I have:

• It’s hard to translate even relatively formal human reasoning into something that a computer can work with.
• Partly because of that, and partly because mathematicians are frequently reductionists at heart, people have been tempted to concentrate on the oldest, most well-trodden parts of mathematics, which means that their engines aren’t going to be of use to the vast majority of working professional mathematicians.
• People have conflated the concepts of “automated refactoring” with “a complete computer-verified proof (from first principles)” and (worse yet) “computer-generated proof”. I don’t think the latter is any more likely to be useful (right now) than having a computer automatically simplify the former for me; as for the former, a computer-verifiable proof starting from first principles is a nice jugendtraum, but you’d be a lot more likely to make an impact if you could produce a tool that could automate certain parts of proofs in ways that helped actual working mathematicians, even if it couldn’t automate anywhere near all of it.
• Automated refactoring tools only showed up in programming settings a decade and a half ago, and still aren’t widely used. At least I think they aren’t; I’ve never used one, and while most IDEs these days contain a few refactorings, I don’t get the impression that most programmers use them a lot. (Which is one reason why I lean so much on automated tests; in mathematics, I think the automated tests would be a good deal harder to carry out, but fortunately I think automated refactorings would be easier.) So it’s not surprising that they haven’t made an impact in other fields yet.
• Writing a tool like this that is easier to use than paper and pencil would be a lot of work, and require very good judgment.

For all I know, people are making a lot of progress in this area, and I’m just not aware of it. But I think there’s something there: I think our understanding of these problems in the programming realm has advanced enough that we probable have something to teach mathematicians. One of these decades, somebody will push the tools far enough to where using them starts to become almost as fast as working by hand, at which point people will get a lot more interested.

I was nosing around my home directory, and ran into a file with the following contents:

From The Mathematical Intelligencer, vol. 3, number 1, p. 45.

Problem is by J. H. Conway.

Problem 2.4:

A Prime Problem for the Ambidextrous

17/91, 78/85, 19/51, 23/38, 29/33, 77/29, 95/23, 77/19, 1/17, 11/13, 13/11, 15/14, 15/2, 55/1.

Write down 2 (with your right hand). Look at these fourteen fractions and use them to generate two sequences of natural numbers by iterating the following:

If r is the last number you wrote with your right hand, look for the first of the fourteen fractions which, when multiplied with r, gives you an integer. Write the integer down (with your right hand). If this integer is a power of 2, say 2^l, write down l (with your left hand).

1. What is the left-hand sequence?
2. Why?
3. How long does it take to generate the first 1000 primes this way?

I’ve just spent a while working through it; I’m not done yet, but I’ve decided it’s interesting enough that there was a reason why I kept it lying around, and perhaps some of my blog readers might enjoy it.

I’m trying to hire right now. Which means that I get to read lots of resumes, mediated by various pieces of technology. Which is annoying, among other things because the format in which the resumes are most easily read isn’t necessarily preserved by those mediating technologies.

Specifically, Sun’s internal tools only accept resumes in either text format or variants of Microsoft Word. Lots of people apparently don’t have a resume natively available in one of those formats, which means that their resume gets cut-and-pasted from another format into text, with the result that it looks like crap and is a pain in the butt to read. (In particular, resumes are typically full of bullet points and indentation, and neither of those reliably survives that journey.) I literally spend most of five minutes going through a typical resume changing the formatting so it doesn’t get in the way of my reading the resume; it isn’t a complete waste of time, because I’m doing a first pass at skimming the resume while reformatting it, but it also isn’t much fun.

So: what is a resume writer to do? It’s been ages since I’ve updated my resume (and no, I am not looking for a job: I just think it’s wise to update your resume every year or so, since I can’t reliably remember what I was doing much farther back than that); the last time I applied for jobs, I used a LaTeX file which I converted into PDF. (Which took a surprising amount of care to get looking right, if I remember correctly: some sort of font problem in the conversion.) Which means that hiring managers at Sun probably wouldn’t like me!

So what are good resume formats these days? Based on my experiences, it’s essential to have a good-looking text version: text is easy to e-mail, it’s a fallback that will always be available. You also want a version that’s nicely formatted; presumably PDF is the format of choice there. And you may want to put your resume on your personal web page, so HTML is probably a good third option.

But, of course, you only want one source representation. I used LaTeX for this in the past, and I’m still not convinced it’s a crazy idea: I think there are probably decent tools to go from LaTeX to HTML or text. Having said that, there’s also nothing about LaTeX that makes it uniquely well suited to the task. A resume is a lightly formatted extremely hierarchical document; any sort of markup language that lets you easily express that hierarchy while giving a reasonable amount of control over formatting should do the trick.

In particular, HTML should probably do the trick. You’d want to take a bit of care over the CSS that you use to style it with, but I don’t think resumes put any excessive demands on styling. You’d especially want to take care when converting it to PDF; PrinceXML seems to be getting a fair amount of buzz these days, so I’d be tempted to play around with that, despite its closed-source nature. Though my first line of attack would just be to provide a print-specific version of my CSS file; among other things, that would improve the way it looks to people who are printing out the resume from my web page. Were I to chose to put my resume there; not sure what I feel about that yet.

What’s the best way to convert HTML to text in a way that works well for resumes? I could have fun with XSLT, but that’s probably overkill. Honestly, maybe just loading the web page with Lynx would be good enough; I’d have to try it and see, once I get around to actually updating my resume. If not, there must be hundreds of other options.

One other tip for job applicants: when you attach your resume to something, the original file name of the resume will be available to the person receiving the resume, and it will probably be given as a default option for that person to save the resume under. So realize that, when you name your resume, you aren’t the main client of that name, the hiring managers are. In particular, if you want to give your hiring manager warm fuzzies, don’t call it “resume.pdf” or “David C Alternate Resume.pdf”: call it “DavidCarlton.pdf” or “DavidCarltonResume.pdf”. The details aren’t important—different hiring managers have different conventions about the names they’d use to save resumes under—but make sure that your full name is there and that there isn’t other extraneous garbage in the name. If you need to store metadata like that in your local copy, put it in the directory hierarchy, not in the filename.

In the spirit of “every long e-mail I send somewhere should be shamelessly recycled on my blog”, I present some random thoughts on testing.

Why do we release products with defects that we weren’t aware of? This is a sign of flaws in our testing; two possible causes are:

1. We don’t know what to test for.
2. We do know what to test for, but we’re not able to do enough testing before release.

For 1, how can we figure out where our blind spots are? Some tactics:

• Defect clusters.

If we can figure out in what areas we’ve historically had a large number of post-release bugs, then we can increase our testing in that area in future products. So if people can come up with useful suggestions for analyzing post-release data, that would be very useful.

• Different classes of tests.

One of the most interesting testing ideas I’ve seen over the last couple of years is the idea that you can analyze tests along two dimensions: are they business-facing or technology-facing, and are they intended to support engineering or to critique the product? (The idea comes from Brian Marick, I blather on about it elsewhere, and there’s also a section on it in Implementing Lean Software Development.)

This gives four quadrants. Technology facing tests designed to support engineering are unit tests, tests that narrowly focus on a specific internal interface. Business facing tests designed to support engineering are tests that are focused on a certain aspect of customer-visible behavior. Technology facing tests designed to critique the product are property testing, tests for “nonfunctional requirements”: load testing, security testing, combinatorial testing. And business facing tests designed to critique the product are various sorts of manual poking around: usability testing, exploratory testing, etc.

I know that, in the past, I’ve had huge blind spots in these quadrants. And we can gather data to figure out which quadrants we might be missing: if we’re either not implementing known basic requirements or taking too long for the product to stabilize its functionality in those basic requirements, then we might be missing tests in the two “support engineering” quandrants. If we’re running into lots of corner case bugs or stress bugs, we’re missing property testing. And if we’re producing products that behave according to spec, but isn’t what the customer wants, then we’re missing tests that are business facing and designed to critique the product.

The above assumes that we don’t know what to test for; what if we do know what to test for, we’re just not doing a good enough job? Here, testing is a bottleneck, and we want to speed it up. At least, it might be a bottleneck: it may also be the case that something else is a bottleneck, creating schedule pressure that isn’t caused by testing, and testing gets unfairly shrunk because it comes at the end of the development cycle. But, for now, let’s assume testing is a bottleneck.

There are certain obvious knobs we can turn here (hire more testers, build more machines to test on), and that may be what we have to do, but those knobs cost money. So we should also look at the testing value stream with lean eyes figure out where we can find waste, and eliminate as much as possible.

To that end, some questions:

• Are there manual tests that can be turned into automated tests?

Doing this would have three benefits:

1. If availability of human testers is a bottleneck, this helps alleviate that bottleneck.
2. Automated tests are generally faster than manual tests.
3. Engineers developing the product can run the tests more easily, which means that they can find defects sooner after introducing them, which has no end of benefits.

• Are there tests that can be sped up?

One technique that works really well on the software side is to directly test the smallest software interface relevant to the issue in question, instead of starting up the whole system: this can turn a 5 minute test into a 5 millisecond test. For example, every time I check in software, I first run a suite of 5000 or so automated tests; if I had to actually run the whole StreamStar system for each test, that would take weeks, but as it is it takes 15 minutes to run all 5000 tests. (And I wish they were faster than that!)

To be clear, we do have other tests that run the whole system. But, to return to the four quadrants above, try to move as many tests as possible to the “support engineering” side (by turning them into tests of clear functional requirements), and try to move as many of those as possible to the “technology-facing” quadrant (by shrinking the interfaces they test). You still need all four quadrants, but that’s the quadrant where you get the most bang for your time.

• Is the test analysis taking too long?

Maybe the problem isn’t with running the tests, it’s with making sense of the results of the tests. Do the tests give a clear pass/fail result? Failing tests take more time to analyze than passing tests (among many other problems, e.g. one bug can mask another); do we have too many failing tests? Do the tests not generate enough information in the failure case to make analysis easy (e.g. so you can tell different known bugs apart, or known bugs apart from unknown bugs)?

• Is the test writing taking too long?

If so, we should invest more time in test frameworks.

• Are people or machines idle inappropriately?

This is a dangerous issue to approach, because you don’t want to do makework for the sake of makework: for best utilization of a system, you should work your bottlenecks at as close to 100% as possible but explicitly allow slack in all other components. Having said that, sometimes waiting is just plain waste. For example, if you’re low on test machines, you want to separate running tests from analyzing tests as much as possible, so you can keep the machine busy running the next test while you’re still analyzing the previous one. (But if you’re not low on test machines, then if you can speed up the test writing/analyzing process by hogging the machine for a while longer, that’s a better choice. And still better is to make the writing and analyzing as easy as possible, so you don’t have to make that choice!)

• Do people have time to think about what they’re doing?

Overworked people make mistakes; even if they don’t make mistakes, it’s hard to devise a method to cut testing time in half in some area if your boss is harping on you to get dozens of things done today.

• Are good ideas spreading through the group?

We need a way to identify our best ideas and to get them adopted broadly.

Recently, we’ve been watching some videos on our TV via our Mac, using an appropriate video adapter and Front Row. Which works pretty well; I don’t plan to make a habit of it, but it’s nice to know that the option is available for the times when I want it.

Actually, I take that back: it works okay, but there are some flaws. Front Row works badly with multiple monitors: it’s unwilling to display on only the second monitor (or rather, I’ve gotten it to do that twice, but I can’t repeat that, and suspect that what I ran into was a bug rather than a feature). This means that, to watch stuff on TV, I have to have the laptop set to a small resolution, which is ugly. (And which takes more clicking than I’d like to switch out of, and it also rearranges my desktop icons.) Also, the laptop goes to sleep if the lid is shut, even if another monitor is connected (cooling issues?), which means that I leave it mostly closed and watch the movie on the TV while weird flickering shadows are mimicking it on the part of the laptop that I can see. And the remote is cute, but I think I’d want a few more buttons if I were using the computer as a DVD player. (Maybe not, though.)

Still, it’s good enough that I can easily imagine not buying another standalone DVD player: a Mac mini (or new Apple TV model?) that was plugged in directly to our TV would probably do just fine, and could also serve double-duty for other purposes. (E.g. iTunes storage.) I’m not sure Apple has just the product I want yet, but they’re fumbling around in that area, and maybe they’ll produce the machine/software I want soon enough.

Which raises the question: what is it that I want? I certainly need something that can work as a DVD player (which you would think is a no-brainer, but rules out the current iteration of Apple TV). My current DVD player is also a DVR, and I would prefer to keep that functionality. (Such as it is: the machine is complete crap, and while it has deigned to start reading DVDs again, it still frequently forgets to record TV shows.) The digital transition is happening in under a year, though, and I assume Comcast will use that as an excuse to force me off of analog, at which point my current DVR won’t work anyways, and I’ll probably have to go through Comcast to get that functionality. A lower price than the current Mac mini would be nice, too.

And then there’s always Blu-Ray: maybe I’ll actually care about that a year from now? And maybe Apple will have released a computer with a Blu-Ray drive? Which I have decidedly mixed feelings about: there seem to be significant technical costs to supporting it in your operating system, and while Apple might do a better job of navigating those costs than Microsoft did, I’m not convinced they’ll be able to really avoid them. So it looks more likely to me that I’d want to go the PS3 route if I wanted a Blu-Ray player. (And eventually there will be PS3 games that I feel compelled to play, won’t there?) Honestly, though, I feel not the slightest urge to switch off of DVDs right now, and I don’t see that changing for several years.

No clear conclusions, and I’m pretty sure we’ll stick with our current setup through 2008. But maybe we’ll fiddle around with this in 2009, depending on the effects of the digital transition and on what hardware is available.

Half a year or so ago, I retrieved my Nintendo 64 from a friend’s house, since I thought Miranda might enjoy some of the games on it. The one she settled on was Harvest Moon 64, a delightful little farming simulator. So, when Christmas came by, I thought she might want another Harvest Moon; I was interested in Rune Factory, but when she read the reviews, she (correctly, I suspect) thought that she would prefer Harvest Moon DS.

So we got her a copy of that, she’s been playing it off and on. (She doesn’t focus on one game the way I do, so she’s still playing both Harvest Moon games, both Animal Crossing games, and occasional spurts of other games.)

And then Liesl was bored last weekend, and picked it up. And now, every evening, I’m sitting here reading blogs while Liesl is next to me gathering wood to build a barn so she’ll be able to successfully woo the archaeologist, or something like that. Pretty convenient, actually, since it means we don’t have to fight for the computer. (Though it’s a good thing this week was a vacation week from school, because we’ve stayed up rather later than was wise on a few occasions.)

But I feel a bit left out of the fun! Harvest Moon DS only has two save slots (why?, why?); then again, I think I’d probably prefer Rune Factory, so if I really wanted to join in, I’d be better off buying a copy of that game instead. (Plus, we have two DSes, a fat one and a lite one, so it would avoid an unnecessary bottleneck if we had two games to play.) The truth is, though, that Eternal Sonata is taking up all my video game time (especially since we have too many series we’re in the middle of watching on TV), and there are other games I’d rather play on the DS when time frees up. (E.g. when we next go on vacation.)

But it is nice being surrounded by farmers. And I’ll join them eventually: it’s a charming series.

For the first time ever, or at least since the dawn of time, I listened to an episode of JapanesePod101 today that showed up in my RSS reader earlier this same month. So the end of that backlog is in sight! I’m not quite ready to declare victory yet—at my current rate, I still have another four weeks to go before I’ll be caught up, even if nothing intervenes—but I’m close enough to be getting excited about it.

Behind Closed Doors recommends that you have frequent one-on-ones with your team members: it says that

One-on-one meetings provide managers an opportunity to ascertain status, solve problems, and provide positive and corrective feedback.

Which I was a bit dubious about: daily standups provide a mechanism for me to get status from my team members every morning and for them to raise problems that I can solve, and having one-on-ones just to give “positive and corrective feedback” didn’t sit well with me. Still, I respected the authors, and thought that it was a good book in general, so, a year and a half or so ago, I decided to give it a try. (Incidentally, I still respect the authors and still think it’s a good book!)

I planned to have them once a month; they happened for about three months, and then petered out, and nobody seemed to miss them.

A few months later, though, I realized that I have one-on-ones with my boss every week and rather enjoy them: what’s the difference here? Part of the difference is that he doesn’t have a daily standup for his team, so there really is more room for getting weekly status updates. But I didn’t think that was the whole story: in particular, what I most liked about the one-on-ones was getting a chance to talk about matters that didn’t seem to fit into other interactions we had. (Sometimes issues that one of us had, sometimes just talking at random about this and that.)

Looking back at my team’s earlier one-on-ones with this in mind, then, it seemed like there were two differences. One is that I’d tried to have a goal for each one-on-one; another is that they only happened once a month, instead of once a week. I suspect that these differences are related: the less frequent something is, the more of an occasion it is, hence the more pressure there is to make something of it.

Having said that, I didn’t feel like there was much of a reason to have one-on-ones with my team members every week. But every two weeks, with no set agenda (except for rare exceptions), seemed like a good fit. That’s frequent enough for them to be routine and to fit in on a repeating schedule in my calendar, and it means that I rarely have anything specific that I want to talk about in the meetings, which reinforces their informal nature.

We’ve been doing this for several months now; I like the results. We usually find something to chat about; and, if we really don’t have anything to say, it’s fine if it only lasts five minutes! Not infrequently, it turns out that we really do have something to talk to that we wouldn’t have talked about without the excuse of a one-on-one, which means that issues are less likely to build up. (Don’t get me wrong, it’s not like we had huge buried minefields before.) On a few occasions, something big has come up; on one occasion, it’s been useful that I had scheduled opportunities to talk to team members individually, so I could sound people out on an issue that we’d had a hard time dealing with collectively without blowing it up out of proportion.

So: I am a fan of one-on-ones, but not for the reasons given in the book. Don’t get me wrong, if you aren’t getting status updates regularly through other mechanisms, then by all means talk about that in your one-on-ones, but there’s also real value in just having a regular opportunity to chat. And schedule them more frequently than you think would be necessary, frequent enough that they’re not an event and that there’s never any pressure to find something suitably weighty to chat about.

I’ve been going through some changes recently in my Japanese study. I finished the Manga-based grammar I’d been reading sporadically, and finished going through the characters in Read Japanese Today. Which I recommend (both of them, but I’m thinking particularly of the latter here): in my experience, you need as many methods as possible to get kanji characters to really stick, and learning about the origin of the characters is a good one. (I also recommend preferring to learn easy easy characters and focused memorization.)

Of course, now that I’ve finished those, the question is: what next? To follow up the latter book, I’m trying to spend more time imagining ways to link radicals to characters when memorizing characters. And Amazon commenters recommended the book Chinese for Begninners, which is apparently really about the characters rather than other aspects of that language; I’ve ordered a copy, we’ll see if I like it or not.

To follow up the former book, I’d been tentatively thinking that I’d start going through Hikaru no Go: I have the first volume in both English and Japanese, so it seems like a good place to start testing myself against the language more. The problem with that, though, is that I’m not sure exactly where in my schedule I’d find time to read it in Japanese! Fortunately, the problem doesn’t seem very urgent right now: I’ve found other ways to expose myself to Japanese (watching episodes of Hikaru and Twelve Kingdoms in Japanese with English subtitles; playing through Eternal Sonata in Japanese with English subtitles), and I’m learning a fair amount from them. I get the feeling that I’ve gotten over some sort of hump: most of the time, I very much rely on the subtitles, but more and more often I can figure out individual words after the fact or even decode whole sentences after the fact, and there are even some very simple sentences that I can figure out without the translation.

Another cause for optimism: for whatever reason, I’m going through the textbook faster than I had been. The last three chapters have taken me two weeks each to finish, instead of the three weeks pace that I’d been going at before, and the pace feels sustainable. My guess is that I’d run into a patch of unfamiliar grammar before (while the recent grammar has been stuff that I’d been at least somewhat familiar with from other sources), and I’m also getting better now at learning new vocabulary, and those have combined to speed up my progress. I wouldn’t be surprised if I occasionally go back to three week chapters (or longer, if illness/vacation get in the way), but I’m now a good deal more optimistic than I was that I’ll manage to finish the last twenty chapters in about a year.

I’m also getting more out of my use of JapanesePod101. I’d been annoyed by two flaws with their RSS feed: they only list the last seven episodes, which makes me worried if I ever go on vacation or need to send my computer in for repairs (previously, when I’d checked, they listed all episodes since their inception in the feed), and they started to throw in “premium lessons”, which I had to download by hand. (I do have a subscription, but I did that because I wanted to support them rather than because I wanted to have special access to stuff; I wished they’d just make the premium lessons available for free and stick them in the main feed!)

So I poked around a bit, and realized that there was a feed available for paid subscribers. And it not only remedied both of those issues, it also contained more material that I was aware of but hadn’t been using. The occasional “bonus audio” tracks are amusing but nothing special; having the lesson-specific PDFs available in iTunes, though, means that I actually look at them (since I see them in iTunes when deleting episodes that I’ve listened to), and they’re a good tool for helping reinforce my learning. I don’t actually generally use them to follow up on the grammatical points in the lesson: their main benefit for me is that they write out each dialogue in four forms: one including kanji, one kana-only, one in romaji, and an English translation. Which gives me a lot more reading practice, and in particular is a good way to test my kanji recognition skills in a safe environment. So now I’ll recommend a basic subscription to other people learning Japanese: it’s not just good for giving yourself warm fuzzies, the extra material in the RSS feed really is useful. (I don’t yet have an opinion about the premium subscription; the price difference is such that I didn’t seriously give it a thought, given that I’m learning enough via other means.)

So: I still have a long way to go, but I’m happy with the recent concrete signs of progress.

One other tweak that I’m considering: I write up lots of vocabulary flash cards (which is clearly useful!); when I’ve decided I know the word in question, I put it in the box in alphabetical order. This takes a noticeable amount of time (not a huge amount of time, but time I’d be happy to eliminate), and is largely drudgery; is it actually useful?

In some circumstances, the answer is yes: if a kanji has lots and lots of readings, I’ll only memorize a few on my first attempt, and take the card out later to add more. But for other words (kana-only ones, compounds), I almost never take out the card once it’s gone in the box. So I’m tentatively thinking that almost all of the time I spend alphabetizing those cards is waste; and, if I decide that I’ve forgotten a word that I once thought I knew and want to take out the card again, I could always just rewrite the card from scratch.

So now I’m thinking I might just throw away cards that don’t correspond to a single kanji. I’m going to think about it for a few weeks, since that’s not a step that’s easily reversible, but it might be a good opportunity to reduce inventory.

• I still haven’t figured out Cursor*10, but it has some neat ideas. (Via GayGamer.)
• An ActiveRecord tutorial; a nice little introduction to the concepts, I liked the way they get their hands dirty with showing you the SQL.
• Yeah, that’s kind of the way I feel about Obama…

• For no particular reason, a random bit of Sesame Street:

  (Via Cosmic Variance.)

• Who am I? I’m Andrew F***ing Jackson! (Via The Edge of the American West.)

• Crazy things with soccer balls:

  (Via Tim Ferriss.)

• Quite a case mod.
• Tim Bray on people involved in the XML standard, ten years after its ratification.
• Some charming, subdued flash games. (The web site isn’t working as I type this, but it was as of yesterday…) (Via Game|Life.)

I have rarely been so proud of my alta mater as when I learned that they (or at least Arts and Sciences) adopted a rule requiring faculty members to make their research be freely and openly available. I have no idea why academic presses didn’t start crumbling five or ten years ago; open access to scholarly research is so obviously the right thing to do.

In another report on the matter, I see Stuart Shieber’s name attached to the motion; I took a compilers course from him, he was my adviser in a compiler-writing project I undertook, and I graded/TA’d for him a few times. Glad to see he’s fighting the good fight!

We’re writing an IDL compiler at work, to generate C++ code from a file containing a list of attributes. We’re doing this in TCL, which turns out to be surprisingly easy and fun: just define some procedures that correspond to the syntax of your language, eval the contents of the file, and you’re off and running!

The syntax is pretty simple, looking like this:

  attribute int foo
  attribute string bar

(Well, that’s what it started out looking like; there have been some changes, as we extended the language to handle more cases.) Loading this invokes a procedure attribute that sticks the names and types of the attributes in some lists/arrays. And then we call some output code that does stuff like this:

  proc add_members {} {
    global attributes types

    foreach attribute $attributes {
      set type $types($attribute)
      puts "  $type ${attribute}_;" 
    }
  }

(I’m not cutting and pasting that, I may have some syntax details wrong. So don’t treat this post as a TCL tutorial!)

But after sticking in “foreach attribute $attributes { ... }” in a few places, it looked like a refactoring opportunity: I should define a procedure each_attribute that iterates over the attributes and binds the attribute and type to a local variable.

Here’s what the code currently looks like:

  proc each_attribute { block } { 
      global attributes types

      foreach attribute $attributes { 
        uplevel [list set attribute $attribute] 
        uplevel [list set type $types($attribute)] 
        uplevel $block 
      } 
  } 
 
  proc add_members {} { 
     each_attribute { 
        l "    $type ${attribute}_;" 
    }
  } 

Certainly the new definition of add_members is a lot shorter!

Some thoughts:

• I like using uplevel! For people who don’t know TCL, curly braces are a string quoting construct, so the block after each_attribute in the body of add_members is actually a string. And uplevel evaluates that string in the context of the calling function, letting us define new control structures.
• At first, I went back and forth about whether each_attribute should take another argument, namely the names of the variables that we want to bind the attribute name and type to. Ultimately, though, that would have let to writing each_attribute { attribute type } { ... } over and over again; why force people to constantly repeat those variable names? (Especially since a call to each_attribute within the block of another such call doesn’t make sense.)
• I do, however, feel guilty about injecting those variables directly into the calling scope: what I really want to do is create a new contour within the calling scope and have the new variables only defined there. Anbody know offhand how to do that in TCL? Can’t be too hard, I just need to look it up

I’ve linked to Hans Rosling before, but his 2007 TED talk makes him worth a second mention:

A good speaker, with some very interesting facts to present and ways to present them, and make sure you stay for the surprise ending! (A warning, it’s 19 minutes long.)

Found via Presentation Zen, who also links to several other of Rosling’s presentations; see also his web site, in particular the GapCasts.

I’ve been in a state of disillusioned disconnect about presidential elections for the last 15 years or so. (I was excited about Bill Clinton; that lasted until he made a 180 degree turn on his campaign promises within days of being elected.) To some extent, I was wrong about that: I underestimated George W. Bush’s insanity and evil, he really is an opponent where being the lesser of two evils leaves an awful lot of playing room. (Not that that’s the end of the story; I can’t come up with a happy view of what the country and world would be like if Al Gore had been elected in 2000.) But disconnected I was nonetheless.

Ever since the first primaries, though, I’ve been surprised to find myself caring about this election more and more; my party registration didn’t allow me to vote in the California Democratic primary, and for the first time I rather regretted that. In particular, quite to my surprise, I’m starting to get more and more actively hopeful about an Obama presidency.

I’m sure those hopes will be dashed soon enough. In the mean time, I present Lawrence Lessig’s views on the matter:

Xanh (warning: web site has completely gratuitous Flash usage), our local excellent upscale Vietnamese restaurant, just moved into larger digs; glad they’re doing well. The new digs are a bit hip for me, and in particular the room that pairs interesting artwork with green ceiling lights (washing out all the colors in the artwork; I wonder what it makes your food look like?) is not an inspired choice. But the food is superb, which is all that really matters; if it’s a little easier to find a seat at the new location and if it lets them have a slightly larger menu, I’m all for it.

Jordan, have you eaten there? If not, we should take you there the next time you’re in town. For locals, the new place is basically right across the street from the old place. (Same block, a few storefronts closer to the train tracks.)