malvasia bianca

I just finished Lean Thinking; it’s my current favorite lean book. One thing that made me jealous: they give several (to me) convincing examples of companies wanting to try out lean, and that brought in some people who really knew how lean worked. After doing what those people said, they immediately got some fairly impressive improvement. But they then managed to improve on this continually over the next few years.

Which, among other things, serves as a counterpoint to some thoughts I’ve been having, and that seem in the air in general. (See, for example, Martin Fowler on agile imposition.) It’s been clear to me for a while that my team’s agile adoption would have been vastly improved by bringing in an outside expert some time ago. (It would probably still be vastly improved by that.) But, among other things, doing so would be tantamount to my saying “we’re going to do XP, plain and simple”. People may hear me as saying that already, but I don’t intend to be saying that. What I would like to be saying is “here are some things that are really important to me” (high quality standards, sharing knowledge, responding quickly) and “I haven’t heard of anything that sounds as effective as XP to reach that goal”.

So one aspect is that I’m jealous of people who have built up the support where bringing in an outsider to show them what to do is effective. But another aspect is that I’m also jealous of people who have concrete touchstones that they can use to continually approve. This is something that, perhaps, XP isn’t so helpful at. The concreteness of the practices can be very useful if you need something specific to try. But they have a finality about them that (to me) makes it hard to use them as guideposts for continuous improvement. For example, we don’t pair program all the time. I’m willing to believe that we’d be more effective if we did, but I don’t have any great way to convince people (even to convince myself) that doing so would be a good idea, and taking it on faith will only go so far. In a current thread on the XP mailing list, Ron Jeffries proposed telling people to find a way to “deliver working software monthly”; a lot of people are willing to believe that that’s a noble goal, but getting from there to XP is a pretty big step.

So what are intermediate goals that can help you see ways to continually improve? (Through which you might end up at XP or might end up somewhere else; if we can continually improve, I really don’t care about anything else.) Here, I think lean manufacturing has a leg up on agile software development: they have goals at a similar level to agile goals (single piece flow or just in time / pull are probably at a similar level to incremental development and no bugs), but I get the feeling that they have more ways to see flaws and to translate those flaws into concrete improvements. (Categories of waste, or stop the line when you see a bug, combined with root cause analysis, for example.)

Or maybe agile is just as good at that; it could (in all honesty, I’m not being facetious) just be my lack of knowledge combined with my lack of skills in the appropriate areas.

Something to work on.

In response to a not-very-coherent question of mine on the lean development list, Tom Poppendieck posted an interesting response. From it:

Over a decade ago, when Jeff Sutherland invented Scrum, he was faced with a situation in which his product development process bottleneck was the capacity of skilled developers. He designed Scrum specifically to exploit the capacity of the bottleneck, to subordinate everything else to ensure maximum throughput through the software development process, and to elevate the capacity of the bottleneck by removing impediments.

I’d never really thought of Scrum in that sort of Theory of Constraints terms; hmm.

Results here not so good, alas.

In Jonathan Kozol’s earlier books, I’d already been appalled by the horrible physical condition of schools serving nonwhite populations. And, in The Shame of the Nation, we see that too:

In the years before I met Elizabeth, I had visited many elementary schools in the South Bronx and in one northern district of the Bronx as well. I had also made a number of visits to a high school where a stream of water flowed down one of the main stairwells on a rainy afternoon and where green fungus molds were growing in the office where the students went for counseling. A large blue barrel was positioned to collect rain-water coming through the ceiling. In one make-shift elementary school housed in a former skating rink next to a funeral parlor in another nearly all-black-and-Hispanic section of the Bronx, class size rose to 34 and more; four kindergarten classes and a sixth grade class were packed into a single room that had no windows. Airlessness was stifling in many rooms; and recess was impossible because there was no outdoor playground and no indoor gym, so the children had no place to play.

In another elementary school, which had been built to hold 1,000 children but was packed to bursting with some 1,500 boys and girls, the principal poured out his feelings to me in a room in which a plastic garbage bag had been attached somehow to cover part of the collapsing ceiling. “This,” he told me, pointing to the garbage bag, then gesturing around him at the other indications of decay and disrepair one sees in ghetto schools much like it elsewhere, “would not happen to white children.” (pp. 40–41)

During the 1990s, physical conditions in some buildings had become so dangerous that a principal at one Bronx school, which had been condemned in 1989 but nonetheless continued to be used, was forced to order that the building’s windows not be cleaned because the frames were rotted and glass panes were falling in the street, while at another school the principal had to have the windows bolted shut for the same reason. These were not years of economic crisis in New York. This was a period in which financial markets soared and a new generation of free-spending millionaires and billionaires was widely celebrated by the press and on TV; but none of the proceeds of this period of economic growth had found their way into the schools that served the truly poor. (pp. 43–44)

I don’t mean to be picking on New York here: he gives examples from elsewhere, here and in other books.

But the differences in teaching style are a good deal more chilling:

“Taking their inspiration from the ideas of B. F. Skinner…,” […] proponents of scripted rote-and-drill curricula articulate their aim as the establishment of “faultless communication” between “the teacher, who is the stimulus,” and “the students, who respond.”

The introduction of Skinnerian approaches, which are commonly employed in penal institutions and drug rehabilitation programs, as a way of altering the attitudes and learning styles of black and Hispanic children is provocative, and it has stirred some outcries from respected scholars. To actually go into a school in which you know some of the children very, very well and see the way that these approaches can affect their daily lives and thinking processes is even more provocative.

On a chilly November day four ears ago in the South Bronx, I entered P.S. 65 […]

Silent lunches had been instituted in the cafeteria and, on days when children misbehaved, silent recess had been introduced as well. On those days, the students were obliged to stay indoors and sit in rows and maintain silence on the floor of a small room that had been designated “the gymnasium.” The school still had a high turnover of its teachers […], but the corridors were quiet and I saw no children outside of their classrooms.

The words “Success For All,” which was the brand name of a scripted program used within the school, were prominently posted at the top of the main stairway and, as I would later find, in almost every room. Also displayed throughout the building were a number of administrative memos that were worded with unusual directive absoluteness. “Authentic Writing,” said a document called “Principles of Learning” that was posted in the corridor close to the office of the principal, “is driven by curriculum and instruction.” I didn’t know what this expression meant and later came back to examine it again before I left the school.

I entered the fourth grade of Mr. Endicott, a man in his mid-thirties who had arrived here without training as a teacher, one of about 15 teachers in the building who were sent into this school after a single summer of short-order preparation. […]

On the wall behind the teacher, written in large letters: “Portfolio Protocols: […]” To the left side of the room: “Performance Standards Mathematics Curriculum: M-5 Problem Solving and Reasoning. M-6 Mathematical Skills and Tools…”

My attention was distracted by some whispering among the children sitting to the right of me. The teacher’s response to this distraction was immediate: His arm shot out and up in a diagonal in front of him, his hand straight up, his fingers flat. The young co-teacher did this too. When they saw their teachers do this, all the children in the classroom did it too.

“Zero noise,” the teacher said, but this instruction proved to be unneeded. The strange salute the class and teachers gave each other, which turned out to be one of a number of such silent signals teachers in the school were trained to use, and children to obey, had done the job of silencing the class.

“Active listening!” said Mr. Endicott. “Heads up! Tractor beams!”—the latter meaning, “Every eye on me.”

On the front wall of the classroom in handwritten words that must have taken Mr. Endicott long hours to transcribe: a list of terms that could be used to praise or criticize a student’s work in mathematics. At Level Four, the highest of our levels of success, a child’s “problem-solving strategies” could be described, according to this list, as “systematic, complete, efficient, and possibly elegant,” while the student’s capability to draw conclusions from the work she had completed could be termed “insightful…, comprehensive.” At Level Two, the child’s capability to draw conclusions was to be described as “logically unsound”—at Level One, “not present.” Approximately 50 separate categories of proficiency, or lack of such, were detailed in this wall-sized tabulation.

An assistant to the principal remained with me throughout the class and then accompanied me wherever else I went within the school. Having an official shadow me so closely is a bit unusual in visits that I make to public schools. Principals who feel relaxed and confident about their teachers typically invite me to sit in on classes without constant supervision and to visit classes that have not been pre-selected. Also unusual, I realized later, was that Mr. Endicott, whom I had met before, did not say hello to me until nearly the final moments of the class and didn’t actually acknowledge I was there except by stoping by my desk and handing me the worksheet on perimeters.

[…] It is one of the few classrooms I had visited up to that time in which almost nothing even hinting at spontaneous emotion in the children or the teacher surfaced in the time that I was there.

I had visited classes that resembled this in Cuba more than 20 years before; but in the Cuban schools the students were allowed to question me, and did so with much charm and curiosity, and teachers broke the pace of lesson plans from time to time to comment on a child’s question or to interject a casual remark that might have been provoked by something funny that erupted from a boy or girl who was reacting to my presence in the class. What I saw in Cuban schools was certainly indoctrinational in its intent but could not rival Mr. Endicott’s approach in its totalitarian effectiveness.

The teacher gave the “zero noise” salute again when someone wihspered to another child at his table. “In two minutes you will have a chance to talk and share this with your partner.” Commuication between children in the class was not prohibited but was afforded time-slots and was formalized in an expression that I found included in a memo that was posted near the door: “An opportunity … to engage in Accountable Talk.”

Even the teacher’s words of praise were framed in terms consistent with the lists that had been posted on the wall. “That’s a Level Four suggestion” said the teacher when a child made an observation other teachers might have praised as simply “pretty good” or “interesting” or “mature.”

There was, it seemed, a formal name for every cognitive event within this school: “Authentic Writing,” “Active Listening,” “Accountable Talk.” […]

These naming exercises and the imposition of an all-inclusive system of control on every form of intellectual activity consumed a vast amount of teaching time but seemed to be intrinsic to the ethos here: a way of ordering cognition beyond any effort of this sort I’d seen in the United States before. The teacher, moerover, did not merely name and govern every intellectual event with practiced specificity; he also issued his directions slowly, pacing words with a meticulous delivery that brought to my mind the way the staff attendants spoke to the Alzheimer’s patients at my father’s nursing home.

[…]

I remember, too, another aspect of my visit that distinguished this from almost anyother class I’d visited up to this time: Except for one brief giggle of a child sitting close to me which was effectively suppressed by Mr. Endicott, nothing even faintly frivolous took place while I was there. No one laughed. No child made a funny face to somebody beside her. Neither Mr. Endicott nor his assistant laughed as best as I can recall. This is certainly unusual within a class of eight-year-olds. […]

When I was later looking at my notes, I also noticed that I couldn’t find a single statement made by any child that had not been prompted by the teacher’s questions, other than one child’s timid question about which “objective” should be written on the first line of a page they had been asked to write. I found some notes on children moving from their tables to their “centers” and on various hand-gestures they would make as a response to the hand-gestures of their teachers; but I found no references to any child’s traits of personality or even physical appearance. Differences between the children somehow ceased to matter much during the time that I observed the class. The uniform activities and teacher’s words controlled my own experience perhaps as much as they controlled and muted the expressiveness of children.”

Before I left the school, I studied again the definition of “Authentic Writing” that was posted in the corridor. Whaever it was, according to the poster, it was “driven by curriculum…” That was it, and nothing more. Its meaning or its value was established only by cross-reference to another schoolbound term to which it had been attached by “drive” in passive form. Authenticity was what somebody outside of this building, more authoritative than the children or their teachers, said that it shall be. (pp. 64–71)

There is a huge amount of education inequality within our nation, within our states, within our school districts, within our individual school buildings. Which raises the question: am I part of the problem, or am I part of the solution?

My daughter is part of a special program which is housed within one of the schools in our local school district. I’m all for different schools having different educational programs: I don’t believe in uniformity, since different students, parents, and teachers have different goals, different philosophies, are simply different people.

Having said that, different programs is one thing; different resources is quite another. Some of the resources I don’t feel guilty about: parents volunteering is one thing, grant money is one thing. And donations in kind can be fine, too. (Not always, but in certain contexts: I would be bothered by rich parents giving dozens of computers, but I wouldn’t be bothered by, say, people bringing in food to some sort of get-together.) But asking for a financial donation from parents who participate in the school? It’s not required—I hope that requiring a donation would be forbidden by law—but even the suggestion very much strikes me the wrong way.

The next question: how do I act on this in a responsible fashion?

I am pleased with the first round of the playoffs this year.

For better or for worse, I’m not planning to get ALCS tickets, but if the A’s win, it would be nice to see a World Series game…

I signed up for Netflix today. It fits in well with my current philosophies; the reason why I’d been holding off was that Liesl and I don’t watch movies very often, and we have a fair number of unwatched movies around. So we wanted to go through the backlog first.

But it’s becoming clear that much of that should simply be treated as sunk cost: maybe we’ll watch them, maybe we won’t, but we shouldn’t let it stand in the way of watching movies that we’re more interested in. So I bought a copy of the first volume of Haibane Renmei last week; after buying that, I realized that I wasn’t sure that I’d want to watch it over and over again, so why did I buy it instead of renting it? Oops. Having said that, we did enjoy it, and I could imagine buying the later volumes at some point, but for now signing up for Netflix and renting the rest of the volumes sounds like a better idea to me – for the cost of the remaining volumes, I could get seven months of a 1-DVD-at-a-time-Netflix subscription instead. (There are also a few other anime series that I’d like to dip into without spending hundreds of dollars.)

We’ll see how it goes, but I’m optimistic; and if that causes us to shift more of our free time to watching movies, that wouldn’t be a bad thing. I’m curious how their series management works; it looks like you can sign up for an entire series at once, and I hope that, if I do that, it will be smart enough to make sure to always send me volumes in order. So if, say, I’ve finished volume 1, and volume 2 is unavailable but 3 is available, it will wait instead of sending me volume 3.

I’ve been kind of obsessed with the theory of constraints recently, which has gotten me wondering about bottlenecks. One of their points is that, in general, a system has a single bottleneck; you should do everything you can to make that bottleneck as productive as possible.

For a simple bottleneck example, say you have a linear, five-step production process. So there are steps A through E, with A’s output being B’s input, and E’s output being the final product. Assume they can produce at the following rate (assuming the previous step has completed): A can produce 10 per day, B can produce 5 per day, C can produce 10 per day, D can produce 3 per day, and E can produce 10 per day.

If you look at where work is piling up, both steps B and D might look like bottlenecks: A can produce tons of stuff, so step A might get annoyed that B can’t consume its output fast enough. Looked at from the whole system point of view, though, only step D is a bottleneck: no matter how fast the other steps get, the whole system can’t produce more than 3 per day.

This example is very simple, but actually a lot of its similarity is irrelevant for this analysis: even if the paths branch and join in interesting ways, ultimately the rate of production of the whole system is limited by its least productive step. (For more complex graphs, though, the actions you take get more interesting – for example, typically there’s some day-to-day variation in the productivity of any given step, and you have to pay more attention to that on the steps feeding the bottleneck than elsewhere. But that is a discussion for another day.)

This, however, doesn’t mean that you can’t get a lot of good from improving the non-bottleneck steps. The first level of analysis is simply to look to make sure that the non-bottlenecks aren’t stealing resources from the bottlenecks. But, aside from that, the above discussion is about throughput; in many situations, latency can be just as important, or even more important.

So, in the above example, consider step C. It can produce at a rate of 10 per day. Maybe that means that it can carry out its step for 10 items in parallel every day; maybe that means that it can carry out its step for 1 item every .1 days. (Or maybe it can carry out its step for 100 items every 10 days.) From a throughput point of view, these are all the same; from a latency point of view, though, doing it for 1 item every .1 days is the best: compared to the 10 items in parallel version, that reduces the time from when raw materials enter the door to when a finished product leaves the door by .9 days. Which is great.

In fact, from the point of view of the whole solution, even a version of C where it could carry out its step for 1 item every .2 days would be better than a version that could carry out its step for 10 items every day. The latency of that step would go from 1 day to .2 days, which is good. The throughput of that step would go from 10 per day to 5 per day; that looks bad, except the throughput of the whole system is 3 per day, so that reduction in throughput is irrelevant.

Which doesn’t mean that you should go around reducing throughput willy-nilly: you’d better make quite sure you know you’re not the bottleneck before doing that! Fortunately, one lesson of lean and agile is that, if you work at it, you can increase latency without decreasing throughput. (Single Minute Exchange of Dies, as opposed to taking two hours to change dies.) Look for waste, find ways to reduce or eliminate it, and you’ll be happy.

It’s not at all clear to me how to apply this at work: I don’t know where the bottleneck is, and our process isn’t even well-enough defined for me to be confident that there isn’t something big that this analysis is missing. One tool to give clarity on this matter is a value stream map; I’d like to try producing one of those some time over the next week or two. If done right, that will help identify potential bottlenecks, and will probably help with ideas for reducing latency. The other tool is waste reduction; the value stream map will help there, too, and I should go through the traditional lean categories of waste to see what examples I can come up in my organization.

I was given a second hard drive for my computer recently. Some random thoughts:

• Annoying to have to buy a bracket to mount it in the computer, and even more annoying that Sun doesn’t want to sell me one directly; fortunately, it’s easy enough to get a used one for cheap.
• Having said that, once I got the bracket, it mounted very easily: no need to fiddle with cables at all. And the box is very easy to open up.
• You would think that Linux would be well-enough evolved to either pop up some message saying “you have a new disk, what do you want to do with it?” or, at the very least, have an appropriate item in a menu to deal with that. Not so, at least on Fedora Core 5.
• What they do have is a menu item for logical volume management. This does everything I want, once I type in the magic command # pvcreate /dev/sdb. (Your device name may vary.)
• LVM is pretty cool; further evidence that there’s nothing that an extra level of indirection won’t solve. I’m still not sure what the best configuration is for my situation; for now, I’ve got a separate volume group on each drive (so that I know what I lose if a drive fails). The original drive has one group with two volumes: one for swap and one for the standard directories (/usr, /home, …) on it; the new drive, for now, has one group with one volume, /backup. It’s nice to know that I can easily, say, increase the swap size if I should want to do so, or divide up the new disk once I think of useful ways to use up all that space. For now, I’m moving more backup stuff there and increasing the amount of stuff I backup; another idea might be to create a second volume there and RAID0 that volume with my main volume on the first disk. (Right now, I’m rsyncing my home directory nightly, which gets me most of the benefits of the RAID0 solution.)

After some discussions on the leandevelopment list, it would seem that slack has has more benefits than I realized. My current list:

1. If you’re not the bottleneck in your process, adding slack won’t decrease throughput and may well increase it (since it makes it easier for you to avoid stealing resources from the bottleneck).
2. Leaving a little slack makes your schedule more predictable: you can work harder to make a date if that should prove to be necessary.
3. It gives people time to recharge.
4. It gives people time to experiment, to try to find improvements that they might not have found if their nose was constantly at the grindstone.

One nice thing about lists like this is that you can turn them around to find places where you should be wary of applying it to your situation. In particular:

The first point suggests that, if you are the bottleneck, slack isn’t a particularly good idea: the bottleneck controls throughput, so the less the bottleneck works, the lower the throughput. And, even if you aren’t the bottleneck, you shouldn’t stick in so much slack as to become the bottleneck. (And you should consider spending some of your slack time figuring out how to help the bottleneck.) Exercise for the reader (actually, an exercise for the writer): what’s the bottleneck in your organization?

The second point suggests that you should have enough control over your schedule to know when you should eat into your slack. Mary Poppendieck mentioned a company that had been working on an 8-week cycle, with 6 weeks development and 2 weeks testing. They got the 2 weeks of testing down to one week; they turned the second week into slack intstead of adding it to development. That way, if problems show up after release, people can jump on them immediately. (And people are motivated to make sure problems don’t show up after release!)

The third point is a reminder that, if necessary, people will create their own slack whether you like it or not; if you try to avoid that, your reward will be burned-out employees. The flip side is that you don’t need vast amounts of slack to get this benefit.

And the fourth point is the most interesting one, and the one whose benefits are potentially largest and hardest to predict. Which means that I have nothing coherent to say about it.

I am currently awash in confusion about how we (my team, but also everybody working on the same product) should improve. Tough stuff; I hope I’ll have something more coherent to say here soon. Fortunately, the good folks on the leandevelopment mailing list are helping me sort through my difficulties.

The fact that I’m so confused suggests that a top priority should be getting more visibility into the situation. If I’m taking a lean point of view, we need to eliminate waste, which means that we should make that more visible. (Red cards? Probably start by just listing forms and manifestations of waste.) If I’m taking a theory of constraints point of view, we need to focus on bottlenecks, which means we should make that more visible. (Maybe create a value stream map?)

The latter is where I started: how do I figure out if my team is the bottleneck? (Is there really only just one?) I have no idea; I can see the card piling up before my team, but I don’t really know what happens after the card exits my team. So I can’t tell where work is piling up.

I’ll try to find time over the next few days at work to create some wiki pages on waste and bottlenecks. One fun thing about work is that enough people are subscribed to the wiki notifications that, if you’re thinking about something, you can just create a wiki page on the subject and you’ll attract some random, insightful questions or comments.

Now does seem like a good time to think about this. On the one hand, there is reason to believe that executing efficiently now could be particularly useful. And on the other hand, my boss has too many direct reports, so some sort of reorganization there may happen over the next month or two. So, if I could actually think of something useful to inform the reorganization, with solid reasoning behind it, I might be able to have an effect.

One of the times when I’m happiest is when I’m wandering around in a confused daze. (Don’t get me wrong, figuring things out and getting things done has its pleasures as well.) But I do need to get some concrete outputs from my wandering.

The two titles I was considering for this post are both military analogies. Sigh. So I will go with the title from the section in the book.

From How Children Learn, pp. 36–37:

The other day we went to Carlsbad Caverns, a strange and beautiful place. To get there, we rode many hours in the car. On the way, we played games. The radio was on, and with Lisa [2 years old – DBC] watching, I began to clap my hands in time to the music. She did the same. Then I began to clap one palm against the other fist. She watched a while, then made both her hands into fists, clapped together a bit, looked again, saw this wasn’t right, and soon did what I was doing. From this grew a whole series of games. I clapped hand against head; so did she. I clapped hand against stomach; so did she. I made my games more complicated. I clapped head with one hand and stomach with another; or clapped head with one hand while holding that elbow with the other, and so on. It was most interesting to see how she copied what I was doing. Each time she began by doing something fairly quickly. As she did it, she checked what she was doing against what I was doing. Then she made a change in what she was doing, checked again, and so went on until she was satisfied that what we were both doing was the same. Watching her do this, I was struck by two things. First, she did not feel that she had to get everyting right before she started to do anything. She was willing—no, more than willing, eager—to begin by doing something, and then think about fixing it up. Secondly, she was not satisfied with incorrect imitations, but kept on looking and comparing until she was satisfied that she was correct—which she almost always was.

I would seem to be in a quiet mood these days. Not feeling much like blogging, not feeling much like programming at home. Maybe because I’ve been programming a fair amount at work; I was worried that, with the new larger group, I’d have almost no programming time, but now that things have settled down (pleasantly!), that is fortunately not the case. (Incidentally, H.264 is charmingly eccentric. Or something.)

Part of the reason, too, is that Okami is tiptop stunning excellent. So I spent all of last weekend playing it, several evenings playing it, and am doing pretty well this weekend so far. I don’t think I’ll quite finish it this weekend, but next weekend certainly. So I guess the game isn’t going to fill the gap until the Wii launch after all; what next? Lego Star Wars II?

Another possibility: I could just not play video games for a month and a half. I would seem to be in a bookish mood these days; or I could spend more time programming. Or spend more timing thinking about stuff and writing about stuff. (Combined with the bookish bit above.)

The latter is increasingly attractive. My thoughts on some of the matters that I’ve been obsessing on over the last few years are starting to settle down. And I’m being reminded (e.g. by helping out with the PACT Parent Ed classes) that there’s stuff that I used to spend a lot of time thinking about that I haven’t recently revisited. So maybe it’s time to, say, go through the complete works of Alfie Kohn (who has a new book out, I should read it) and John Holt and see what, if any, points of contact they have with what I’ve been thinking about recently. Or maybe I should try to actually put some thinking tools into action. Or maybe I should spend every waking hour reading about lean. Or maybe I should spend time thinking about whether my actions are congruent with my stated beliefs and, if not, why not.

Or maybe I should play video games. That would certainly be easier…

By the end of the weekend, I’d copied enough of my CD’s that my laptop’s hard drive was beginning to fill up, so I thought it was about time to start backing them up. So I grabbed one of the USB drives that I bought for the purpose. (Half a year ago, when I bought the computer – this was before I got Just in Time religion.)

The computer didn’t recognize the drive. Oops.

I brought it upstairs to the Linux machine; it didn’t like the drive either. After playing around with it for half an hour, the Linux machine seemed happy with it, but the Mac didn’t like it. The other drive I bought at the time still worked, so I didn’t think it was a usage problem: the drive was bad.

So the drive went into the trash. But, actually, this is a blessing in disguise, because it pointed out a whopping big hole in my backup strategy. One that I was vaguely aware of, but had previously been able to ignore by sticking my fingers in my ears and saying “la la la I can’t hear me”. Namely that a single backup isn’t any good: you want lots of the suckers. Or at least two.

I’d originally chosen USB hard drives over optical media because I was doing this for the purpose of backing up degrading optical media, and I didn’t have any confidence that the backup optical media would last any better than the original. But obviously the hard drives had problems; and if one of them were to really fail, it could wipe out a big chunk of my collection. (Hmm: buy a bunch of USB drives, RAID em, and stick them in the safety deposit box? Nah.)

So I think what I’ll do now is back them up to both DVD-Rs and hard drives. Hopefully they won’t both fail simultaneously, and the worst thing that can realistically happen is that my house burns down, the hard drives fail completely and a handful of DVDs fail. If that happens, I’ll still have 95% of my CD collection to comfort me while sitting in the ashes of my house, and the missing 5% will be the least of my worries. Hopefully this won’t be too labor intensive; presumably the Mac comes with some sort of dead-easy DVD burning software. And I’ll keep MD5s of everything around, so I can tell which backup is correct if I get bit rot on one of them.

And I’ll revisit the issue in another decade or so, at which point it will cost next to nothing to back up everything on, say, flash USB storage and a couple of remote hosts providing reliable backup service. We’re only talking 300GB or so, after all. (At which point I’ll be worrying about backing up my DVD collection, I suppose.)

Silly me, though: I really should know better than to make a mistake like that.

As regular readers are aware, Yosha hadn’t been at his best for a while. A couple of weeks ago, though, he started getting even more lethargic, and pretty much stopped eating. We tried giving him food that actually tastes good; that helped for a while. But then it stopped helping, and he was having a hard time standing up. And was spending even more time asleep. And Zippy was really worried, and was grooming him a lot.

By last Monday or Tuesday, it was clear that the end was near, and while he didn’t seem to be in too much pain, things could get a lot worse any time. So we made an appointment to have him put down in the middle of the day on Friday. (Friday had the advantage that we could stay home with Zippy for three days in a row, to help him make a transition.)

As it turned out, he didn’t quite make it until then: he died Friday morning. In bed, quickly, with Liesl, Zippy, and me all right there.

It will be hard on Zippy – he has basically never ever been apart from Yosha. (Together at the vet, together getting haircuts, together at the kennel, …) Having said that, he seems to be handling it quite well: no despondent howling, no wandering around wondering what’s going on. He knew Yosha was sick, he was there when he died. (And groomed him for a few minutes afterwards, either as one last try to help or as a way to say goodbye.) He basically hasn’t left my side since it happens, but I think he’ll do okay.

It’s hard on the rest of us, too. I’m honestly not sure if I’ve spent more time with Liesl or with Yosha – Liesl and I spend time together out of the house without Yosha, of course, but Yosha and I spent time together in the house without Liesl, especially when I was in grad school.

Having said that, I suspect that most of the grieving went on in the two weeks before he died. It wasn’t a shock that something like this would happen eventually; we’d hoped it wouldn’t come quite soon, but we had enough warning to spend a little more time with him saying goodbye, cuddling with him even more more than normal. And it was very clear by the end that his time had come.

No, we aren’t planning to get another dog soon. For one thing, it has been my position for years that I’m going to need some dog-free time at some point. (Not that we won’t get dogs eventually, I’m just going to want a break for a few years there.) For another thing, I don’t think we’d be doing Zippy any favors by that: he’ll miss Yosha, but he’s 12 years old, and I don’t think he has enough energy to really enjoy playing with a puppy.

As a Nintendo fanboy, I must of course comment on the Wii launch news. Packing in Wii Sports makes sense for their market-broadening strategy; wouldn’t it have helped if they could have included a second wiimote, though? The controller prices are exorbitant. I wish I didn’t have to wait until November 19th.

The news that surprised me the most, though: the Mii Channel. When you turn on the system, you can do various things other than play games; one of those is create an avatar of yourself, called a “mii”. Here are some movies; the avatar editor looks pleasantly accessible, enough so that, when I buy my Wii, the first thing I’ll do will probably be create a mii instead of, say playing a game.

And this avatar can be used in games. (Not all, of course, just those where it makes sense.) So all of a sudden, the low-quality graphics in Wii Sports make sense. As Scott McCloud has taught us, sometimes you can identify better with less detailed representations of people than with high-fidelity ones; a tennis game sounds more fun with four cartoony representations of the people playing. I’m sure the Wii version of Animal Crossing will use this as well, and this will encourage the development of other friendly titles in new genres targeted at non-traditional gamers.

Having added extra memory to my Mac (1.5GB, instead of .5GB), and having finished the other games I was in the middle of, I went out and bought a copy of Civilization 4 on the Saturday of labor day weekend. I opened the box, gazed with pleasure upon the technology tree diagram, popped it in, bumped up the graphics settings, and began the tutorial.

Two turns later, the game crashed. For lack of a better idea, I bumped down the graphics settings somewhat; this time, it lasted a good twenty or so turns before crashing. So I bumped it down still more; it finally let me finish the tutorial.

This is one of the reasons why I almost exclusively play games on consoles. Computer game manufacturers are apparently perfectly happy to release crap that wouldn’t make it through even the most basic of playtesting: that’s what patches are for, right? Well, no: I buy a game because I want to play it right now, not because I want to play it in a few months when most of the bugs are ironed out. To be sure, the developer’s job is made more difficult by the various possible system configurations that people might try to run the game on.

Despite that unpleasantness, I gave the game a go. It seemed stable enough, and was quite playable at the lowest graphics settings, so I left it alone – I don’t know for sure if the graphics settings were the trigger, but I really wasn’t up for determining the stability boundaries. And the game proved to be as excellent and as addictive as its predecessors. I stayed up far too late that night with my first game, and woke up early the next morning to finish the game; I then stayed up far too late Sunday night with my second game, and spent Monday morning finishing it.

As is common in its genre, it is addictive out of proportion to its quality. The gameplay is turn-based, and there’s always something little going on to pull you to play just one more turn. Also, there are never any large events – ends of levels, reaching a rare save point, etc. – to give you a push to stop playing. Fortunately, unlike some games in the genre, the gameplay really is excellent. And pushes some of my buttons – I really do like the whole city- / world-building thing, and exploring unknown territory.

So if you’ve never played any games in the series, do yourself a favor and go out and buy a copy: you’re in for a treat. If you have played other games in the series, you know what to expect; this one has more of the same, with a few tweaks (for the better).

Having said that, after spending a weekend playing it, I had a dilemma. On the one hand, I quite enjoyed the game, and would continue doing so for a few more plays. On the other hand, I’ve seen the ideas before, there are a few ways in which it’s not my complete fave, so staying up until midnight playing it isn’t the best idea.

If I could have continued playing it in moderation, I would have done so. But it’s not clear that I’m capable of doing so. So, for better or for worse, I didn’t play it last weekend, I didn’t play it this weekend, and Okami is about to come out, so I’ve decided to go cold turkey on the game. The upshot: it’s the best game that I’ve stopped playing after three days.

One way in which it isn’t the best fit for me: I don’t really enjoy games with its sort of combat model. I’m happy to play, say, turn-based strategy games, as long as I have complete information. But once concepts like fog of war get thrown into the mix, I don’t enjoy the combat nearly as much. In Civilization, you fairly early on reach situations where, in order to expand, you need to take over at least some of your neighbors’ territory. And you don’t really know what you’re up against; also, in general, you have to worry about your neighbors invading you, where you also don’t really know what you’re up against. I can deal with this sort of thing, it’s just not my fave.

Actually, what I really like about this game has nothing at all to do with combat: I like building cities, building an empire, developing land, researching technology, and the like. So maybe the lesson here is that I should spend more time with games that focus on that sort of thing: I should give the latest Sim City another look, or recent Harvest Moon games, or something. Hmm. We’ll see how Spore works for me, when it eventually comes out.

I’ve finally started backing up my CD collection, only half a year after I planned to do this. The goals are:

1. To have lossless backups, including offsite ones, of the entire collection.
2. To do whatever error correction is possible when creating the backups.

(The vast majority of the CDs are between 10 and 20 years old, and are showing their age.)

If it weren’t for the second point, I would backup the CDs by doing dd if=/dev/cdrom of=.... But I don’t trust that to produce as clear a copy as possible.

Digging around, there seem to be various tools which specialize on doing good copies: cdparanoia for Linux, Max, for MacOs, and Exact Audio Copy, for Windows. Since my CD collection is located in the room where my Mac is, Max seems like the way to go.

Unfortunately, this is where my naivete about just what is on an audio CD begins to show. My naive method produces one big file for the entire disk. Max, however, wants to generate a collection of files, one per track. My understanding is that, if I tell Max to generate WAV files, then that’s more or less the same data as the naive method (on a disk where reads are error-free). But is it exactly the same data (modulo trivial packaging), or is there some sort of extra data that I’m missing? I know CD’s contain almost no metadata, but I’m worried about stuff like gaps between tracks – iTunes has a habit of screwing that sort of thing when ripping CDs (though they claim to have improved that with the latest release), and I have several CDs (e.g. operas) where that is quite annoying, so I’m a bit worried that I’m missing something there.

Does anybody know of a good FAQ on these matters? I’m having a surprisingly hard time piecing together the information that I’m looking for: what exactly is on an audio CD, what the relationship is between that and a collection of WAV files.

Anyways, I’m glad I’ve gotten started. It may take months for me to actually finish the process, but it should be a relatively mindless one from now on. (Assuming that a collection of WAV files proves sufficient.)

What are good formats to use for purchasing and storing music and movies? I remember a time in the past where it was possible to pretend that Ogg Vorbis was a reasonable choice for many of your audio needs; that is, unfortunately, no longer the case.

Desirable qualities for a format, in no particular order:

• Quality should be as high as possible.
• File size should be as small as possible.
• The standard should be open.
• The standard should be unencumbered.
• There should be a wide range of software tools (including free ones and best-of-breed ones) to play the format, on all platforms I use.
• There should be a wide range of hardware devices (including best-of-breed ones) to play the format, in all locations where I’d like to use it (home, car, bus, jogging, …).
• There should be a wide range of software tools (including free ones and best-of-breed ones) to convert to/from the format.
• It should be possible to easily purchase content in this format.
• It should be possible to easily find free content in this format.
• I should be able to easily copy, excerpt, etc. the content for the forseeable future.
• The format should support adding metadata (titles, composer, performer, album art, …)
• The format should support aggregation (e.g. a podcast with multiple pieces of music should be a single file with indexing).

These are, of course, incompatible goals, but never mind that. Given the above, what are the conclusions?

One is that DRM is to be avoided: it fails on the copy/exceprt criteria, and probably also fails on open standard and free tools criteria. Having said that, it’s almost impossible to purchase video without some sort of basic DRM being involved. So, in some circumstances, I guess I can live with DRM, as long as tools for cracking it are widely available. Which is, fortunately, the case for DVD’s, though I’m somewhat worried that hardware might start getting in the way there. (For example, my understand is that many current DVD drives force you to respect region encoding.)

What are suitable audio formats? The aforementioned Ogg Vorbis unfortunately doesn’t look so hot to me. The big reason is that best-of-breed hardware devices don’t play it: I can’t use it on my iPod. There’s very little content available in the format. And it’s lossy, so I can’t store content in it and convert it to other formats as necessary.

Also, its main advantage over MP3s is that it’s unencumbered by patents. The problem is, in this crazy day and age, I don’t know if I can even be sure of that. There are so many overbroad software patents being granted that I can’t be confident that any decent a/v compression format couldn’t be attacked by patent trolls.

So, basically, it only helps me if I want to burn a CD and play it on a computer using particularly purist tools. And that’s a situation where I never find myself. The only Ogg Vorbis files I had on my computer were from Lambda Expressway, but now I see it’s available as MP3s, so that won’t be necessary any more.

So, in practice, there seem to be three obvious candidates for music: Audio CDs, MP3s, and AAC files. Audio CDs are widely available for purchase, playable in lots of ways, as high quality as is easily available, easily convertible to other formats. The downside is that file sizes are somewhat large, and they’re lacking in the metadata department.

MP3s are quite widely available, too. File sizes are smaller; quality is generally acceptable for my listening purposes. They’re lossy when compared to CDs, though, which makes them less suitable for archival purposes. Better than CDs for metadata, but there’s room for improvement there. Bad for aggregation. Also, it’s quite difficult to purchase MP3s; I hope that will change in coming years (decades?), but maybe I’m over-optimistic. Tons of players.

AAC is one of my current favorites: it has most of the advantages of MP3s, but does better in metadata and, especially, aggregation. So it’s a great format for podcasts. Slightly fewer players, but enough of them for my purposes. The patent situation seems somewhat better than that for MP3s, but not perfect.

MP3s and AAC suit my needs for free stuff. For purchasing, CDs are good, but it would be nice to have a format that I liked that would enable me to purchase music digitally. Unfortunately, all the choices suck: Apple’s and Microsoft’s DRM solutions are both loathsome. So, for now, I’m buying music on CDs: I’d rather do that, wait a couple of days to have it shipped to me, and rip it myself, than have music delivered quickly and painlessly in either of those formats. I have to think that this suggests that the recording industry could find a way to make more money off of me if they were willing to give up on DRM; the recording industry is not, alas, well-known for its forward-thinking business acumen.

I am a little worried that, at some point, music that I care about won’t be available on CD. Recently, for example, I wanted to get music of some of the artists I liked from Next Big Hit. They’re all independent, so it wasn’t too surprising that I couldn’t find all of them at Amazon; that’s what CD Baby is for. But I couldn’t find any of the music from one of them there, either; looking at her website, I did find her music for sale, but not on a CD, and some of it was apparently only available from iTunes.

So it sucks that new artists are getting caught up in DRM protection that’s really designed to serve (or “serve”, perhaps) the interests of large labels. Fortunately, in this case I could just e-mail the artist directly, and it turned out that she did have a few copies of her CDs left over. But a taste of a world that I’d just as soon avoid.

For video, the story is less pleasant. There’s no long-standing open format like CDs. Videos that are on the web are generally either in Quicktime (which encompasses many different formats, but these days I can usually play it under Linux) or Windows Media (which I can’t play under Linux). Actually, that’s not even true – videos are frequently hidden behind Flash front-ends to the extent that I don’t know what the underlying format is, and can’t get at the bytes short of doing a tcpdump or something. MP3s and AAC both have analogues, namely MPEG-2 and AVC (a.k.a. H.264), which are as acceptable as their audio compatriates. And there are new physical formats coming out that do look noticeably better than their predecessor, and that will probably be harder to copy and play on free players. (And there’s no reason to believe that this is the end of video formats.)

The upshot is that, for video, I just stick my head in the sand. Fortunately, most stuff on the web I click on once to watch but have no desire to save. And almost nothing I really want to watch is in Windows Media format, so I don’t have to install that viewer on my Mac. I like watching DVDs, but will happily avoid upgrading to newer formats for the indefinite future. (And I’m cautiously optimistic that the general public will be slow to adopt either of the new formats.) I haven’t yet grappled with the whole backup issue: my DVDs aren’t showing signs of age the way my CDs are, and disk space isn’t quite cheap enough for me to want to back them up wholesale anyways.

I suppose I should look on the bright side: no matter what, we live in a much better world now than when we had to deal with LPs, cassette tapes, videotapes. New media are much more robust, much easier to copy, much higher quality, much more broadly available. And the current RIAA leadership will retire eventually.

I just upgraded to iTunes 7; I wish I hadn’t. Downloading fancy album cover is nice (though I wish I hadn’t had to give a credit card number to do so); the reflected images of the album covers are a bit much, but whatever. We’ll see how gapless playback works.

Unfortunately, it turns out that a certain piece of core functionality is broken: as far as I can tell, there is no way to tell it to automatically sync most podcasts with my iPod but to let me manually manage some of them. The previous mechanism that I’d been using for this (only sync checked episodes) has gone away. (The checkboxes themselves are still there; maybe the option will be restored in a future update?) The user interface lets me drag and drop episodes, which should also work just fine, but they don’t actually show up on the iPod.

So: buggy software. If it were free software, I’d probably be able to easily find some place I could get a definitive answer and/or file a bug; with Apple, there seems to be no way to do that. There do seem to be customer forums where I can whine; not clear that doing so will help, but who knows. I do hope there will be an update fixing this soon; if not, I guess I’ll look into free software for managing the beast.

Two e-mails I received today:

An e-mail from CD Baby with the name associated to the from address given as “CD Baby loves David”, saying (among other things) the following:

Your CDs have been gently taken from our CD Baby shelves with sterilized contamination-free gloves and placed onto a satin pillow.

A team of 50 employees inspected your CDs and polished them to make sure they were in the best possible condition before mailing.

Our packing specialist from Japan lit a candle and a hush fell over the crowd as he put your CDs into the finest gold-lined box that money can buy.

We all had a wonderful celebration afterwards and the whole party marched down the street to the post office where the entire town of Portland waved ‘Bon Voyage!’ to your package, on its way to you, in our private CD Baby jet on this day, Wednesday, September 13th.

I hope you had a wonderful time shopping at CD Baby. We sure did. Your picture is on our wall as “Customer of the Year”. We’re all exhausted but can’t wait for you to come back to CDBABY.COM!!

I am amused.

And Payseur & Schmidt e-mailed to double-check that I received a book I’d ordered a month ago. Apparetnly they’d been having problems with their ordering system, information had been lost, and they wanted to make sure I’d gotten my order. (And, if not, ship me a replacement ASAP.)

Nice to see people who care.