I ran into a situation like that last night, and I decided to try to take that big step apart; here’s what I ended up with. The problem in question was exercise 3 on page 84: it told me to print out the first word of each line of its input. My first two tests were as follows:

firstWordsTests =
    TestList["empty" ~: "" @=? firstWords "",
             "one line" ~: "first\n" @=? firstWords "first words\n"]

which I got passing with this implementation:

firstWords "" = ""
firstWords line = head (words line) ++ "\n"

The third test was where I ran into trouble, though:

firstWordsTests =
    TestList["empty" ~: "" @=? firstWords "",
             "one line" ~: "first\n" @=? firstWords "first words\n",
             "two lines" ~: "one\ntwo\n" @=?
                         firstWords "one line\ntwo lines\n"]]

How can I write a pattern which will match the third case? Not at all clear to me, but at least it points out the direction I’m going in: I want to add lines to the test cases until some sort of looping construct falls out. Given that, let’s try to refactor against one red bar in a way that brings out the decomposition into lines that’s latent here.

Fortunately, Haskell has a function lines that transforms a string into a list of the lines making up that string; let’s rewrite firstWords to use it. The smallest step that I managed to come up with to do so was this:

firstWords input = concatFirstWordsOfLineArray (lines input) where
    concatFirstWordsOfLineArray [] = ""
    concatFirstWordsOfLineArray [line] = head (words line) ++ "\n"

Which is a larger step than I’m comfortable with, but at least it’s small enough conceptually that I don’t feel like I’ve leapt into the unknown. I’m not sure what to call this sort of transformation—maybe “Insert Intermediate List”?

(Incidentally, the observant reader will note that this transformation isn’t a refactoring: it preserves the red and green bars, but the nature of the red bar goes from an unexpected result to an exception being thrown. That’s okay with me; what I’m doing is still useful as an implementation pattern. Hmm, maybe I should try writing these transformations up in an Alexandrian style?)

And, after that step, it’s now obvious how to get my test to pass:

firstWords input = concatFirstWordsOfLineArray (lines input) where
    concatFirstWordsOfLineArray [] = ""
    concatFirstWordsOfLineArray [line] = head (words line) ++ "\n"
    concatFirstWordsOfLineArray (line1 : line2 : []) =
        (head (words line1) ++ "\n") ++ (head (words line2) ++ "\n")

At this point, we have some obvious code duplication; Extract Method turns it into

firstWords input = concatFirstWordsOfLineArray (lines input) where
    concatFirstWordsOfLineArray [] = ""
    concatFirstWordsOfLineArray [line] = firstWordLine line
    concatFirstWordsOfLineArray (line1 : line2 : []) =
        firstWordLine line1 ++ firstWordLine line2
    firstWordLine line = head (words line) ++ "\n"

At which point it’s pretty obvious that we’re doing something quite similar to all elements on the list, so we want to transform this into a map plus a subsequent operation. And, in fact, the subsequent operation is concatenating all of the list elements together; keeping that in mind, we do an Insert Intermediate List on the third case, giving us:

firstWords input = concatFirstWordsOfLineArray (lines input) where
    concatFirstWordsOfLineArray [] = ""
    concatFirstWordsOfLineArray [line] = firstWordLine line
    concatFirstWordsOfLineArray (line1 : line2 : []) =
        concat [firstWordLine line1, firstWordLine line2]
    firstWordLine line = head (words line) ++ "\n"

That makes the structure clear for the third branch, and also makes it obvious that we can write the first two branches the same way:

firstWords input = concatFirstWordsOfLineArray (lines input) where
    concatFirstWordsOfLineArray [] = concat []
    concatFirstWordsOfLineArray [line] = concat [firstWordLine line]
    concatFirstWordsOfLineArray (line1 : line2 : []) =
        concat [firstWordLine line1, firstWordLine line2]
    firstWordLine line = head (words line) ++ "\n"

So now we have our map operation:

firstWords input = concatFirstWordsOfLineArray (lines input) where
    concatFirstWordsOfLineArray lines = concat (map firstWordLine lines)
    firstWordLine line = head (words line) ++ "\n"

(These last three steps seem like they should all go together: Extract Identical List Operation?)

Now the code is looking nice (and, in addition, would pass more tests should we choose to write them), and the challenge turns towards expressing it as tersely and clearly as possible. First, Replace Unaltered Parameter with Composition to get:

firstWords = concatFirstWordsOfLineArray . lines where
    concatFirstWordsOfLineArray lines = concat (map firstWordLine lines)
    firstWordLine line = head (words line) ++ "\n"

Do it again (throwing a bit of currying into the mix):

firstWords = concatFirstWordsOfLineArray . lines where
    concatFirstWordsOfLineArray = concat . (map firstWordLine)
    firstWordLine line = head (words line) ++ "\n"

And, by now, concatFirstWordsOfLineArray clearly isn’t pulling its weight, so we Inline Method:

firstWords = (concat . (map firstWordLine) . lines) where
    firstWordLine line = head (words line) ++ "\n"

I still want to make this shorter, which in Haskell land frequently seems to mean using Replace Unaltered Parameter with Composition; to that end, we rewrite the latter definition as

firstWords = concat . (map firstWordLine) . lines where
    firstWordLine line = (++ "\n") . (head (words line))

That lets us turn it into:

firstWords = concat . (map firstWordLine) . lines where
    firstWordLine = (++ "\n") . head . words

at which point we can Inline Method again, and get:

firstWords = concat . (map ((++ "\n") . head . words)) . lines

Which is short, but you have to think a little bit as to what it means; what I’d like to do is find a way to get the unlines function in there, which is a function that takes a list of strings and concatenates them with newlines between. The next step in that direction is to realize that map distributes over function composition; so we Distribute Map, giving us

firstWords = concat . (map (++ "\n")) . (map (head . words)) . lines

and, indeed, the first half of that is exactly unlines:

firstWords = unlines . (map (head . words)) . lines

Phew! The code is now about as terse as I can think of while passing those three tests, and it passes several other new tests that I might think of to boot. And, as a bonus, this version is clearer than any of its predecessors: we split the input into a list of lines (lines), then we grab the first word out of each of those lines (map (head . words)), then we smoosh all of those lines back together (unlines). Though, as Bryan pointed out to me, I forgot to write one test (which I’ll leave as an exercise for the reader), but getting the code to pass that last test was a lot easier in this form than it would have been in forms further up the blog page. (Bryan also had some suggestions for how I might use QuickCheck instead of HUnit to test this, which I hope to be able to follow up over the coming months.)

If this sounds interesting (or if it sounds bizarre but if the idea of learning Haskell sounds interesting despite my peculiar approach), it’s not too late to join the reading group: none of us are moving at a very fast pace, so it shouldn’t be much trouble for a newcomer to catch up.

So we’re forming a book club. And I figured a few of my readers might be curious about the language as well. If you fall into that bucket, let me know and I’ll add you to the mailing list.

• Joe Armstrong and Simon Peyton Jones discussing Erlang and Haskell. (Via @timbray.)
• SuperBetter, Jane McGonigal’s serious-injury-recovery game. (Via @therealfitz.)

• Elite Beat Agents, meet Phoenix Wright!

  (Via GameSetWatch.

• A light-and-mirror puzzle in a tabletop RPG.
• Cloud pictures. (Via Dubious Quality; the fungus-infected violin link is pretty good, too.)

• A music video version of Danc’s Flash Love Letters.

• Mitu Khandaker on immersion and controllers.

• The best Zork walkthrough I’ve ever heard.

  (Via MTV Multiplayer.)

• And, because I seem to be incapable of doing one of these without some sort of random Japanese offering:

  (Via Pink Tentacle, of course.)

The first was given by Jon Dahl, on “Aristotle and the Art of Software Development”. You can see video and slides of an earlier delivery of the talk. (I don’t think it changed much between deliveries; I also don’t know how much sense the slides will make out of context.) He viewed this subject through a lens of ethics: a Kantian view focusing on actions (and, in particular, universal principles motivating them); a Millsian view focusing on outcomes; and an Aristotelian view focusing on actors.

As you might suspect from the title of the talk, Dahl was most taken with the third point of view. (Which begins about 15 minutes into the video or 70 slides into the deck.) He presented Aristotle as linking ethics with happiness and a life well lived, and with virtue (in the sense of performing your function well). Honestly, it may be the case that many of these words had significantly different meanings for Aristotle than they would for us; in particular, Dahl glossed the Greek term that he translates as “happiness” as coming from “good” plus “spirit”; depending on how one takes the compound, I could imagine that as meaning “one whose spirit is good”, which makes the idea that ethical people are happy potentially more of a tautology than an an interesting statement.

Leaving these glosses aside: the point is that being good (virtuous) at something is inherent in the people who are good, rather than being rule-based. Which raises the question: how do you become a virtuous person? In the talk at Agile 2009, it sounded like at least some of Aristotle’s answer to that is fatalist: virtuous people are born, not made. (And—stop me if you’ve heard this before—virtuous people always turn out to be rich non-barbarian males. Sigh.) Watching the video of the RubyConf version, though, there’s a less fatalistic point of view: you gain virtue through practice and through modeling. So, to become an expert, you should hang out with experts, and practice; as you gain expertise, use your improved understanding to improve both your expert detection and your practice.

In contrast, if you’re a Kantian, instead of looking for good people, you look out for good principles, and follow them rigorously. In an Aristotelian point of view (again, with the caveat that it’s been decades since I’ve read any Aristotle myself, I’m just going by the talk), it would be possible to have virtues without formalizing the principles at all; furthermore, Aristotle’s virtues are means between excesses and defects (see slide 95; this gives me a Buddhist vibe, too), so following rules too strictly may actually be a sign that you’re straying from virtue.

The other talk was one by Mary Poppendieck on “Deliberate Practice in Software Development”. It discussed the nature versus nurture question of becoming an expert. My memory of the talk is that she strongly came down on the nurture side of that; rereading the slides, I’m not sure my memory is correct, but what is correct is that she claimed that nature alone is not enough: the way you get to Carnegie Hall is, indeed, to practice. Specifically: people need to practice their field for about 10,000 hours before becoming experts; and those who do eventually become experts practice longer and harder than those who don’t.

This last sentence, of course, doesn’t settle the nature or nurture debate: it may be that those who aren’t naturally gifted won’t become expert violinists no matter how long they practice. On a more meta level, it may also be the case that, without the appropriate nature, you’ll find practicing violin for 10,000 hours so offputting that you’ll give it up long before then. So it’s not clear (at least to me) to what extent nature is necessary; it does seem to be the case, however, that nurture (in the form of practicing) is necessary to become an expert, and in fact quite a bit of nurture is.

It’s not the case, however, that any old 10,000 hours of practice will do. As the title of the talk says, it should be “deliberate practice”, and Poppendieck listed four key factors in making the practice successful: a mentor, a challenge, feedback, and dedication. You want somebody else to guide you; you don’t want to be complacent; you want to know how your work is turning out; and, even with that, you need to put in the sweat. And she gave examples of how you might structure your work to support this. (Looking at it through a lean viewpoint; the slides don’t mention them, though the talk may well have, but I’ll bring up A3 reports.)

I was planning to turn this post into some sort of grand overarching pulling together of the above with some other articles that had come across my twitter feed. But, after listening to Dahl’s talk again, I’m having a hard time finding any nice symmetries. So, I’ll conclude with talking off the top of my head (as if I ever do anything else!):

Poppendieck’s recommendations don’t contradict the Aristotelian point of view: in particular, it sounds like Aristotle would wholeheartedly support having a mentor. Can we make other links: maybe Mill’s utilitarian point of view resonates with feedback, for example? Actually, I’m surprised at how hard it is to find explicit Kantian resonances with the deliberate practice model; is that a sign of something deep, or is that just a sign that the idea of having principles to guide us is to obvious as to not need stating? (As a side note, right now I’m finding the idea of following quite detailed Kantian principles in software development to be strangely appealing.)

The nature versus practice question is one that I’m interested in for both personal reasons and parental reasons. Take the question of how nature affects our practice habits: I’m a decent musician, but I was never one who was drawn to spend hour after hour after hour in practice rooms honing my art. I’m sure I could have become a better musician than I actually am, though I don’t know where my ceiling would be (if that concept even makes sense), but I do know that doing so would have required quite a bit more desire out of me. And I certainly support Poppendieck’s claim that not all practice is created equal: I’ve seen more than enough of people doing “practice” in ways that strike me as noticeably suboptimal.

As for nature and raw talent: I’ve seen enough exceptional people that I don’t believe that nature is irrelevant to becoming a world-class expert, from an outside point of view. But, from an internal point of view, it’s probably best to pretend that it is irrelevant: Poppendieck’s deliberate practice recommendations sound pretty solid to me, and I’m fairly sure that anybody following them and putting in the time would improve anybody’s skills. They also suggest a meta-approach: improve your skills at deliberate practice. Though, of course, you can’t just do that in isolation: you want to deliberately practice deliberate practice, which (among other things) means actually learning stuff.

Looking at holes in my own practice: of Poppendieck’s four recommendations, the area where I probably do worst is in finding mentors. (Indeed, in retrospect I probably should have spent rather more time acting as a mentor while managing the last few years. Sorry!) I imagine I shy away from feedback (both giving and receiving), too.

Important stuff; I hope we can all figure this out.

Buying an iPod Touch changed that, though. Once I took a bit of care to add a stylesheet that looked good on the device, I found that it was just as easy to go through my vocabulary on the iPod as it was on the computer, and it was hugely easier to fit the review into spare moments of the day. So I switched from doing one big review a day to at least three a day, plus a few more when I had spare moments around the house.

So that was a huge help. I still couldn’t review vocabulary during spare moments at work, though; and once I got an iPhone, the number of places where I could conceivably review vocabulary but couldn’t actually do so got even larger! This made making the program available safely on the internet a lot more urgent.

I ended up using authlogic for authentication support; there were a few odd testing issues, but in general it worked quite well. (I think I’ve figured out at least one of the testing issues; I hope to have time soon to confirm my hypothesis and submit a patch.) And once that was in place, adding the general plumbing to link items with users was straightforward. (And I learned something about Rails in the bargain.)

I still don’t necessarily recommend that anybody else use it. For one thing, I suspect that the spacing of my review sessions isn’t optimal: I’m using exponential spacing more out of optimism than any strong evidence that it’s the best pattern to use, and in fact I get the feeling that some of my gaps in the months-out range are a little bit off. And, for another thing, I’m sure that there are features that I don’t need that others would find important; I can’t promise to be able to add them.

I will, however, strongly recommend that, if you have an iPhone or an iPod Touch and are considering using some sort of timed review program, that you don’t choose a desktop-only solution (unless you have a desktop computer in your bathroom): it makes a huge difference to be able to nibble away at your items when you have free time instead of having to do larger batches at less frequent intervals. And I hope that I’ll be able to add feature requests that I deem sensible without making you wait too much; alternatively, I’m happy to make a Mercurial repository available if you want to tinker with it yourself. I will also back up your data offsite nightly (though I make no uptime guarantees for the server my program is hosted on: it’s my home computer, and if something bad happens while I’m on vacation, it could take a couple of weeks to get it fixed). And you should be able to export your items / timing information as an XML file at will, if you want to change solutions. (Though it’s your problem getting that XML file translated into whatever format the other solution wants!) Having used it for more than a year gives me confidence that I’ll remain interested enough that it shouldn’t disappear for the forseeable future.

So if any of you wants to give it a try, let me know and I’ll set you up with an account.

Incidentally, I mentioned a few months ago that I was feeling a bit swamped with projects and conference prep, and was planning to step back some from my normal blogging schedule. (Especially the video game related parts of it.) Fortunately, Agile 2009 is now over (though, don’t get me wrong, I enjoyed both the preparation and conference very much!), and some of my side projects are winding down, so things should be getting back to normal around here. In fact, over the last week and a half things have been much more prolific than normal on this blog; I don’t plan to keep that up, but it is very nice both to have the time and energy to blog like that and to remind myself that I really enjoy doing so.

The relevant models are a User model and an Item model. A user has_many :items; part of the Item class definition was the following:

class Item < ActiveRecord::Base
  ...
  def self.next
    find(:first,
         :order => "next_review_time",
         :conditions => pending_item_condition)
  end

  def self.pending_review_count
    count :conditions => pending_item_condition
  end

  def self.pending_item_condition
    ["next_review_time <= ?", Time.now.utc]
  end
  ...
end

As written, those would, for example, add up the total count of all users’ pending items; I wanted instead to be able to ask those questions on a per-user basis.

The has_many declaration gives me a collection user.items for each user. And if I pass a block to has_many, I can add methods to that collection; that seemed more stylish than, for example, adding a user parameter to the above methods. So I modified my User unit tests to make sure that there were items owned by multiple users, and copied over the Item unit tests for the above methods, replacing, e.g. calls to Item.next with calls to user.items.next.

Which I expected to fail, complaining about an undefined method “next”. But not only did the tests not fail for that reason, they in fact passed, ignoring the items for other users! At first, I assumed I’d made a mistake in my tests, but no: when I replaced the calls to user.items.next with calls to Item.next, they failed as expected. So the functionality I wanted really did seem to be working without my having to lift a finger.

But how? I could imagine some sort of method_missing implementation that forwarded functions on the collection to functions on Item. But if it were doing that, how was the functionality getting restricted to items owned by the user in question? I would hope that it wasn’t doing a broader search in SQL and then subsequently ditching inappropriate items in Ruby; sure enough, a look at the logs shows that the SQL queries that are issued do include the user_id restriction.

I haven’t verified it yet, but the only hypothesis that I’ve come up with so far is that the collection objects that are returned by user.items start off life as clones of Item; so they have all of the class methods on Item. And then some key method (find, say) is overridden so that the user restriction always gets applied, and all other lookup methods are implemented in terms of that key method, so they get the restriction in question for free.

Which, once I thought about it, makes sense. Consider Item, not as a class but as an object: you can think of it as a collection object, namely the collection of all items. With that in mind, it makes sense for other objects that are also collections of items to be closely related to it, and if we can express that tight linkage in terms of the implementation, by thinking of Item as the primordial collection of items, so much the better!

An interesting journey. At first, I thought I must be writing bad tests; then I thought I was seeing magic; and eventually I came around to a realization that what I was seeing made a lot of sense conceptually, so I was happy that I was working with a language and a framework where I could get such sensible behavior for free. And, no matter what, it was a useful reminder that there’s a reason why the TDD cycle is red, green, refactor, not just green (with simultaneous test+implementation changes), refactor: if I’d done the latter, I would have missed a useful learning opportunity.

Though there’s a postscript to the story. I’d learned that my original implementation idea was unnecessary; the question remains, though, whether the original implementation idea was a good one or not. And, after giving it some thought, I decided that I preferred having these methods in the User model rather than in the Item model. If I left them in Item, it would be easy to write code representing concepts like “what’s the next item that any user will be asked?” or “what’s the total pending item count across all users?” Which aren’t natural questions to ask, so I was setting myself up for bugs by leaving those methods in Item.

So, with my tests in place, I moved them to User:

class User < ActiveRecord::Base
  ...
  has_many :items do
    def next
      find(:first,
           :order => "next_review_time",
           :conditions => pending_item_condition)
    end

    def pending_review_count
      count :conditions => pending_item_condition
    end

    def pending_item_condition
      ["next_review_time <= ?", Time.now.utc]
    end
  end
  ...
end

Leaving me where I expected to end up when I began, but having taken a much more interesting route.

Star and Griesemer’s first type of boundary objects are Repositories:

These are ordered ‘piles’ of objects which are indexed in a standardized fashion. Repositories are built to deal with problems of heterogeneity caused by differences in unit of analysis. An example of a repository is a library or museum. It has the advantage of modularity. People from different worlds can use or borrow from the ‘pile’ for their own purposes without having directly to negotiate differences in purpose.

At first, I thought this type was kind of banal, corresponding perhaps to collection objects in software, but now I think it’s more interesting than that. Reading their description more closely, I don’t get a collection object vibe: collections in the programs that I write usually contain a quite uniform group of objects, and those objects are used for one or two specific purposes; the above, however, emphasizes heterogeneity and differences in purpose.

That last sentence, in particular, reminds me of mashups; if you combine that with standardized indexing, I’m getting a very strong RESTful vibe from this. In a RESTful application, names are key but internal structure can vary from location to location, and outsiders can use standard tools to access the data that the application exposes and borrow it for their ends.

Next in their taxonomy is the Ideal Type:

This is an object such as a diagram, atlas or other description which in fact does not accurately describe the details of any one locality or thing. It is abstracted from all domains, and may be fairly vague. However, it is adaptable to a local site precisely because it is fairly vague; it serves as a means of communicating and cooperating symbolically—a ‘good enough’ road map for all parties. An example of an ideal type is the species. This is a concept which in fact described no specimen, which incorporated both concrete and theoretical data and which served as a means of communicating across both worlds. Ideal types arise with differences in degree of abstraction. They result in the deletion of local contingencies from the common object and have the advantage of adaptability.

My first reaction here was to try to make an analogy with abstract types; indeed, they use the word “abstracted” in their second sentence, and I can easily see their species example being used as an example in an OO textbook. The only thing that gives me some amount of pause when proposing this analogy is their use of it as a talisman communication tool between multiple parties, many of which may turn out to want to know more about the details of the objects in question.

In contrast, in my programming experience, if I have a concrete subclass of an abstract class, it’s more typical for almost all users to only care about the abstraction, while perhaps only one user cares about the concrete class. Though, rereading what they say, maybe their species example suggests that the correct analogy to the Ideal Type is to (any sort of) class, with the non-ideal objects being instances, rather than subclasses? Either way, though, I get the feel that I’m missing something in the way that I was missing something with my earlier analogy between a Repository and a collection: is there another analogy waiting to be found here that’s a bit grubbier in the way that the RESTful example is?

Third up are Coincident Boundaries:

These are common objects which have the same boundaries but different internal contents. They arise in the presence of different means of aggregating data and when work is distributed over a large-scale geographic area. The result is that work in different sites and with different perspectives can be conducted autonomously while cooperating parties share a common referent. The advantage is the resolution of different goals. An example of coincident boundaries is the creation of the state of California itself as a boundary object for workers at the museum. The maps of California created by the amateur collectors and the conservationists resembled traditional roadmaps familiar to us all, and emphasized campsites, trails and places to collect. The maps created by the professional biologists, however, shared the same outline of the state (with the same geo-political boundaries), but were filled in with a highly abstract, ecologically-based series of shaded areas representing ‘life zones’, an ecological concept.

This one was hard for me to grapple with. (And I’m not the only one; that link, incidentally, gives references to extensions of this taxonomy.) Even in the physical world, it’s a bit hard for me to tell examples of this: is the concept really restricted to large-scale geographic areas? That seems a bit limiting.

The paper in question discusses the Museum of Vertebrate Zoology at the University of California, Berkeley; is it a boundary object? I tend to think so: quoting from the definition on page 393,

This is an analytic concept of those scientific objects which both inhabit several intersecting social worlds (see the list of examples in the previous section) and satisfy the informational requirements of each of them. Boundary objects are objects which are both plastic enough to adapt to local needs and the constraints of the several parties employing them, yet robust enough to maintain a common identity across sites. They are weakly structured in common use, and become strongly structured in individual-site use. These objects may be abstract or concrete. They have different meanings in different social worlds but their structure is common enough to more than one world to make them recognizable, a means of translation.

And certainly the MVZ is robust enough to maintain a common identity, but it means something different to a postdoc working there, to somebody who has spent her career there, to a visiting researcher, to a university administrator, to an outside funder, to a janitor. (Indeed, much of the paper is devoted to showing such differences in meanings.) Given that, I would treat the MVZ as a Coincident Boundary: though not spread over a large-scale geographic area, it’s still a place which means different but related things to different people.

Which, to be honest, doesn’t help me directly with finding programming analogies; maybe a function body in the input to a compiler that means different things to the lexer, the parser, the optimizers, the code generator, the debug info generator? Actually, I think maybe the more important analogy is a bit more conceptual and not internal to programming: maybe we could think of a domain object as an example of a Coincident Boundary that means one thing to a programmer, another thing to a database administrator, a third thing to an system architect, a fourth thing to an XP Customer, a fifth thing to a marketer, a sixth thing to an end user. I’m not completely sold on that, but I do think that domain objects are boundary objects of some sort, and they’re a better fit to Coincident Boundaries than anything else in Star and Griesemer’s taxonomy.

The last entry in their taxonomy is Standardized Forms:

These are boundary objects devised as methods of common communication across dispersed work groups. Because the natural history work took place at highly distributed sites by a number of different people, standardized methods were essential, as discussed above. In the case of the amateur collectors, they were provided with a form to fill out when they obtained an animal, standardized in the information it collected. The results of this type of boundary object are standardized indexes and what Latour would call ‘immutable mobiles’ (objects which can be transported over a long distance and convey unchanging information). The advantages of such objects are that local uncertainties (for instance, in the collecting of animal species) are deleted.

Class interfaces (whether concrete or abstract) are examples here, as are generalizations such as duck types or the sorts of dependencies that C++ templates impose on their parameter types. For example, templates don’t care about the details of an iterator as long as it exposes its increment operator under the name ++, its equality operator under the name ==, and so forth.

Network protocols are another example: indeed, what is better than TCP/IP at ensuring that data “can be transported over a long distance and convey unchanging information”? Stick it on the wire in a Standardized Form, and it will come back out the other end. We can use our RESTful example from above in this context as well: if you want disparate clients to all be able to talk to each other, it helps a lot if everybody speaks in terms of GET, POST, PUT, and DELETE.

And, as we did with the last example of the taxonomy, we can step away from the code a bit. An acceptance test is a Standardized Form: if the Customer and the engineers want to agree on what it means to complete that task, it sure helps if they can point to a Standardized Form for specifying that completion, and an acceptance test that both sides can read (and run!) is an excellent form for that agreement to take, for deleting uncertainties.

Interesting stuff. I’m curious what other classes of boundary objects people have come up with, and I should probably spend more time thinking about examples outside of the strict domain of code. And I really like the lens it gives on the messiness, the grunge, the lack of sterility of the RESTful approach: if you pin down enough so that people can talk to each other while leaving enough of the details undetermined so that different groups can use the entity in question for significantly different ends, unexpected synergies can flourish.

Ivermectin is a popular drug for deworming animals. Onchocerciasis (river blindness) is a chronic illness that’s a particular burden in sub-Saharan Africa. Since river blindness is caused by a worm susceptible to ivermectin, the manufacturer (Merck) desired to donate ivermectin to fight the disease. That presented some problems. For example, it would not be in Merck’s interest if the bulk recipients responsible for redistributing ivermectin to people instead resold it into the lucrative veterinary market. On the other hand, it would also not be in Merck’s interest to tell the recipients (including national governments that are markets for other Merck drugs) that they are not competent or trustworthy enough to receive ivermectin. Merck needed organizational distance.

The solution was for Merck to donate the drug to a non-profit non-governmental organization. An independent expert committee would make the decision about which applicants (both governments and non-governmental organizations) would then receive the drug. This committee is a boundary object. To Merck, it provides distance: Merck donates the drug, reaps the benefits in good will and tax deductions, but is insulated from political repercussions. To the bulk recipients, the committee is the dispassionate judge of applications, end-point of an application process, and advisor during implementation.

This situation and its solution immediately reminded me of the notion that “All problems in computer science can be solved by another level of indirection.” But it’s not just that broad aphorism: the example reminds me of Bob Martin’s SOLID principles in particular. (The first five principles listed here; see also this Hanselminutes show on the topic.) I don’t think all his principles apply to the drug example, but more than one does.

The most obvious example is the Single Responsibility Principle. Merck is a big company that does lots of things; to handle this problem, they created a separate organization that has only one job, to deal with dispensing ivermectin.

I’ve squinted at the the Open Closed Principle a few times, and I can’t see how it applies to this situation – the types of modification that the OCP is talking about don’t seem so relevant here.

The Liskov Substitution Principle also doesn’t seem particularly relevant. It’s about base classes and derived classes; the only example of that that I see here is that you could think of the abstract concept of an applicant as a base class, with concrete applicants as derived classes, but that’s a pretty weak LSP situation: the base class is so abstract, the derived classes are so concrete.

Skipping ahead to D, the base and derived classes are a much better fit for the Dependency Inversion Principle. In fact, the second paragraph quoted above is all about that principle: rather than having the concrete company Merck deal with concrete recipients of donations of the drug, we introduce multiple abstractions. From Merck’s point of view, the NGO is something of an abstraction: as long as Merck knows enough about the NGO to trust that they’ll do a reasonable job dispensing the drug, it doesn’t have to worry about the details of the process. Similarly, from the applicants’ point of view, the NGO is a relatively abstract organization compared to Merck. (Or is it? Am I conflating this with the Single Responsibility Principle? Certainly there are fewer opportunities for linkages between the applicants and the NGO than between the applicants and Merck as a whole.) From the NGO’s point of view, the very notion of “applicant” is an abstraction placed on real people, real organizations.

Going back to I, the Interface Segregation Principle also seems relevant, though admittedly my justification for it here seems very similar to my justification to the Single Responsibility Principle: the NGO is exactly the interface to Merck for clients who want free ivermectin.

Does this analogy hold up in other examples of boundary objects? Can we relate the Open Closed Principle and the Liskov Substitution Principle to examples outside of programming? Can we run our analogy in the other direction, finding properties of boundary objects that suggest principles of good programming?

Being a good GTD devotee (or a good lean/agile devotee), this means that something has to go. Fortunately, I’m actually pretty well on top of things right now—in particular, my Next Action list is about as short as I can ever remember its being—so I shouldn’t have to prune too much; but I have to prune something. And I’m certainly not going to take a break from learning Japanese—in fact, one of the unintended consequences of the memory project has been to make there be pretty serious consequences if I take even a couple of days off from my study. (One could make a sensible case that I am being a total idiot in subscribing to ChineseClass101 right now, however. Though I certainly don’t intend to treat that as seriously as I’m treating learning Japanese.)

So I think my only choice is to cut down on my video game playing for the time being. Don’t get me wrong: I’m not going to stop completely, you’ll still find me every Thursday evening at the VGHVI play nights, and I’ll keep up with VGC activities. I imagine I’ll do some playing and blogging outside of that, too, but for those of you who read this blog for game-related content, don’t be surprised if there are relatively slim pickings here for a while.

But don’t unsubscribe, either! In particular, my conference activities won’t continue forever, so by the fall I should be back to normal. Heck, I might even be back to normal after Agile 2009—I certainly want to find time in early September to play a certain game.

Actually, I subscribed to several of the company’s series: I was pretty sure that the lowest level they offered was too basic for me, but the next five levels (going from 2-year-old through 6-year-old) all seemed plausible. So I subscribed to all five, planning to unsubscribe from the lower levels as I got more confident. In fact, I subscribed to them several months before I finished the textbook, so I had a backlog built up before I started reading any of them.

So I started working through my backlog of the 2-to-4-year-old fiction level, こどものとも年少版. (Which means something like “child’s friend early years edition”?) It was surprisingly hard, in some ways harder for me than later levels: it uses an awful lot of onomatopoeia (which Japanese uses much more than English in general), and I’m fairly sure that some of the speech forms are somewhat nonstandard parents-talking-to-kids forms rather than what I’d learned in grammar books. Fortunately, the books were totally charming, and while I wouldn’t want all books to be as repetitive as those ones are (a lot of doing the same thing on different pages with different numbers or colors or animals or whatever), it really helped me to have the same sentence structure and half of the same words to cling to while figuring out the rest of what’s on the page. And I’ve gotten a lot better at reading books in that series over the intervening months; the onomatopoeia words are even starting to stick.

Once I made it through my backlog of books at that level, I started on the next level: ちいさなこどものとも (little science’s friend?), nonfiction for 3-5 year olds. This was a great level for me: the sentences didn’t have the word usage quirks that previous level had, and the sentences were a bit more interesting while still not requiring me to look up an overwhelming number of words.

About a month ago, I made it through my backlog of those (I’d been reading one every weekend), and moved up to the next level. It’s called こどものとも年中向き (child’s friend targeted at intermediate years?), and is fiction for 4-5 year olds. And I’m enjoying the transition: the books are a bit longer than previous volumes (28 pages instead of 24 with more words to a page), but my practice from previous levels is paying off, as is my memorization practice, so they’re not taking too long. I’ve only read three books from that level so far, but they’re really quite varied: one was a regular story that confused my a lot until I realized that some of the word endings were in regional dialect; one consisted of scenes from a train station that might have fit better in the science series; and one was a counting/animal story that, honestly, probably would have fit better at an earlier level.

I’ve subscribed to but not started reading two more levels after that (one nonfiction, one fiction, both going through age 6); for now, I’m staying subscribed to the earlier levels, but I imagine at some point I’ll unsubcribe to those and add a subscription to something still more advanced. Also, for what it’s worth, all of the levels I’m subscribed to are kana-only, so my kanji practice isn’t paying off here yet. Though it’s paying off in other areas: for example, it’s kind of weird looking over at the spines of my Japanese go books and realizing that I actually recognize most of the characters I see there, even the non-go-specific ones.

I’m still listening to JapanesePod101, of course (incidentally, they just added a Chinese sister site, if you’re interested in learning Mandarin), and I’m spending a lot of time (almost certainly an unproductive proportion of time) memorizing vocabulary in general and Kanji in particular. In particular, I basically haven’t skipped a day using my memory program since it went live almost 10 months ago. (I usually use it during my lunch break at work.)

Which has been an interesting experience: in particular, at first, I ignored some of Wozniak’s suggestions, and I’ve learned that I was wrong to do so. To be clear, I don’t claim to be following any of his algorithms at all—I’m sticking with the algorithm I outlined here—but there are recommendations he makes that would apply to my algorithm that I ignored. In particular, he suggests a floor of 1.3 for the exponent; initially, I figured I’d put in a floor of 1.0 instead. But, after a few months, that turned out not to work at all: it was taking more and more time each day to review stuff because, once an item got tagged as “most difficult” (not too hard with kanji), I’d review it every single day for a month, and that clogged up fast. So I bumped the floor up to 1.2, and things got better; I then figured I should stop reinventing the wheel and bumped it up to 1.3, and I’m glad I did.

I’m also doing a better job now of following his suggestion of breaking up items into small chunks to memorize. Before, I would list all of the readings of a Kanji as one item: e.g. for the question 問 I would list the answer “もん、と（い）question, problem; と（う）matter, care about”. But now I break that up into three pairs: Q: 問, A: もん question, problem; Q: 問い, A: とい question, problem; Q: 問う, A: とう matter, care about. That has several advantages: individual items are smaller (as Wozniak recommends), I naturally focus more on the readings that are harder for me to remember, and I’m testing myself on something that actually matters when reading instead of an abstract skill. (I.e. I will encounter 問う when reading, but I will never be in a situation where it matters if I can list all the endings that you can stick after the Kanji 問.) In particular, the previous method wasn’t good at training me to tell whether, say, 上る was the reading のぼる or あがる. (It’s the former, the latter is written 上がる.)

Also, I made another Japanese-specific change while breaking up the kanji into multiple questions: I started writing the On readings (derived from Chinese) in katakana and the Kun (native Japanese) readings in hiragana. (So the answer to 問 is really モン.) It’s actually usually pretty obvious whether a reading is On or Kun, so that’s not important from a memorization point of view, but it meant that every day I was exposed to hundreds of katakana characters, so my katakana recognition speed has increased dramatically. (Incidentally, if any of you are learning Japanese, a recommendation: learn how to use your keyboard input method. Under Linux, you can convert a word to katakana by hitting F7; under OSX, by hitting control-k.)

Another surprise: I’d sort of assumed that some sort of geometric series magic would mean that I would be able to keep adding items to the database without increasing the amount of time I need to spend reviewing each day. Which, if you think about it for a minute, isn’t the case at all: e.g. if all items are at exponent 2 and I never make a mistake, then every day I need to review all the items I added yesterday, all the items added 2 days ago, all the items added 4 days ago, all the items added 8 days ago, all the items added 16 days ago, etc., and the growth here is unbounded. (Or rather, is bounded only by my lifespan!) I don’t think this is a big problem, but it might be; it does suggest that if I have too many items with small exponents then I’m in trouble. I hope that that problem will naturally ease: there’s a limit to the number of Kanji I have to memorize (I’m almost halfway through the official common usage Kanji list), and as I start reading more, I’ll get exposed to vocabulary more frequently in other contexts, which should manifest itself by the vocabulary seeming easier from the program’s point of view. We’ll see how it goes; if it gets too bad, I’ll cut down on the forced memorization and spend more of my time just reading and not worrying much about words I don’t know.

I had plans to quickly spiff up this application and make it multiuser, but that didn’t happen: basically, it became useable shockingly quickly, and I really didn’t have much of an impetus to improve it past that stage. It’s amazing what I’ve managed to leave out: for example, I assumed that I would have to implement a search functionality early on. But part of the basic Rails CRUD functionality is a URL that lists all the items, and combining that with browser search still works acceptably for search even though I’ve got over 3000 memory items listed. Or I assumed that I would have to secure it (and probably naturally add multiuser functionality as part of that) to get it useable while at work or travelling, but ssh tunnelling to an unsecure deployment was working fine for me until I got my new iPod and wanted to be able to use the program from the iPod’s web browser.

That’s changing now: aside from the iPod issue, I’ve recently gotten a bit frustrated with some UI elements, Miranda has shown some curiosity in using the program, and I just finished reading the paper version of the third edition of the Rails book. So now I’m pretty excited to start up my tinkering again! And in fact I started that last weekend (I continue to be impressed at how easy it is to write functional tests in Rails, incidentally), and I plan to continue with that on future weekends until the program looks/works a lot better. So: Jim and Praveen, I apologize for the delays, I’ll have a multiuser version available soon if you’re still interested! And anybody else who is interested, let me know; I’ll announce it here when it’s ready for use by people other than myself.

To counter these catastrophic possibilities, efforts are made to routinise component actions either through technicians’ training or by automation. Once a string of operations has been routinised, one can look at the figures obtained and quietly forget that immunology, atomic physics, statistics, and electronics actually made this figure possible. Once the data sheet has been taken to the office for discussion, one can forget the several weeks of work by technicians and the hundreds of dollars which have gone into its production. After the paper which incorporates these figures has been written, and the main result of the paper has been embodied in some new inscription device, it is easy to forget that the construction of the paper depended on material factors. The bench space will be forgotten, and the existence of laboratories will fade from consideration. Instead, “ideas,” “theories,” and “reasons” will take their place. Inscription devices thus appear to be valued on the basis of the extent to which they facilitate a swift transition from craft work to ideas. The material setting both makes possible the phenomena and is required to be easily forgotten. Without the material environment of the laboratory none of the objects could be said to exist, and yet the material environment very rarely receives mention. It is this paradox, which is an essential feature of science, that we shall now consider in more detail. (p. 69)

The production of a paper depends critically on various processes of writing and reading which can be summarised as literary inscription. The function of literary inscription is the successful persuasion of readers, but the readers are only fully convinced when all sources of persuasion seem to have disappeared. In other words, the various operations of writing and reading which sustain an argument are seen by participants to be largely irrelevant to “facts,” which emerge solely by virtue of these same operations. There is, then, an essential congruence between a “fact” and the successful operation of various processes of literary inscription. A text or statement can thus be read as “containing” or “being about a fact” when readers are sufficiently convinced that there is no debate about it and the processes of literary inscription are forgotten. Conversely, one way of undercutting the “facticity” of a statement is by drawing attention to the (mere) processes of literary inscription which make the fact possible. (p. 76)

A fact only becomes such when it loses all temporal qualifications and becomes incorporated into a large body of knowledge drawn by others. Consequently, there is an essential difficulty associated with writing the history of a fact: it has, by definition, lost all historical reference. (p. 106)

Can we profit from focusing on objects/processes that “facilitate a swift transition from craft work to ideas”? I spent a few pleasant hours this afternoon doing some Rails programming; that framework shines because of the small amount of craft work necessarily to see a manifestation of your ideas. Does a software framework count as an “inscription device”? Does a programming language? Does a compiler, an interpreter? If not, is there some generalization of that concept that we can use here?

Agile processes value a swift transition between the programmer’s craft work and the Customer’s ideas. (A transition in both directions, I should add.) What are the inscription devices here? Ironically, one of the key mechanisms that agile uses to speed this transition is to remove certain inscription devices, or at least inscriptions, in favor of people talking directly to each other.

Can we relate tests to inscriptions and inscription devices? Test runs can certainly lead to thousands, millions of inscriptions over the course of a day; most of those inscriptions are internal, in that the software is noting that an assertion passed, but I label them as inscriptions nonetheless. They’re a very good form of persuasion; if you’re on a project where test runs act as a reliable safety net, then your worry level decreases, you can treat the software’s behavior as a “fact”, and spend time in idea land. Until, of course, a test failure (or, much worse, a failure that your tests didn’t catch) undercuts your software’s facticity.

I’ve been pretty obsessed with A3 reports for the last few months, which are certainly a form of inscription. And one of the strengths of the process is the extent to which the A3 report doesn’t serve as a source of persuasion, the extent to which the “sources of persuasion seem to have disappeared”: if the process is doing well, it’s a summary of facts to which all participants agree. Or have the sources of persuasion disappeared? Perhaps better to say they’ve been distilled down to a trace, as with a scientific paper; I don’t want to underestimate the importance of that trace.

I don’t suppose I can relate this to video games somehow? One issue that I struggle with, especially in games with a large variety of techniques to reach a goal, is how to internalize the various gameplay options that are available to me. Most of the time, I end up leaning on a few standard ways of progressing through a game’s levels; I suspect my experience would be richer if I had a broader tapestry of “facts” to choose from in the form of live tactical (or, better yet, strategic) options. What can games do to help me reach this state? What inscriptions can they present me with to ease this journey? How can I modify my own play styles to reach this state?

The talk that did the most to crystallize these thoughts was the Harmonix one. The speaker started the talk off by addressing the issue that everybody at the company has an idea of how the games should work, should evolve; rather than treating this as an imposition (how dare those programmers tell the designers what the game should be like!), they apparently treat this as an opportunity, a resource. This doesn’t mean that they let everybody go off and stick their own ideas into the game—indeed, most of the talk was about addressing that tension, about how to get a coherent game in the face of so many cooks. But what it made me realize is: I’m totally one of those people who has opinions about everything, and who gets unhappy if his opinions get brushed off. (I’m quite happy to have my opinions argued against and discarded after due consideration, though!) So I’d better spend my time working at companies like Harmonix.

And that has nothing to do with the video game industry: on any but the smallest project, there will be multiple voices that want to be heard, that (in my opinion) deserve to be heard. One of the reasons why I’ve stayed with my current job for so long (coming up on six years!) is that I’ve never (or at least rarely) felt that my voice wasn’t heard: people have given my ideas due consideration, and a reasonable number of them have been adopted by the project. (And a reasonable number have been discarded, too. That’s okay. Incidentally, none of what I’m saying here should be taken as arguing against the agile idea that there’s a Customer who ultimately decides what goes into the project: I have nothing against somebody other than myself having final authority over certain decisions, I just want those decisions to be made after appropriate Consultation.)

This also shed some light on the sort of organizational role that I’d prefer. For the last four and a half years, if I’m counting correctly, I’ve been managing people, but I’ve maintained a job label that puts me in Sun’s engineering track rather than it’s manager track. And this isn’t just a conceit: I will immodestly suggest that I know the details of our product’s code as well as anybody, and I’ve touched the code quite regularly over the course of those years. (And continue to do so: some of the back story behind my last post is my thinking that I should increase my technical efforts in certain areas.)

This puts me in an uncomfortable position career-wise: I like what I’m doing now, with a foot in the programming world and a foot in the managerial world, and I’m all-too-well aware that the vast majority of jobs out there make you choose the one or the other. And I won’t rule out the possibility that I’ll eventually leave the programming world entirely—in particular, I’m getting more and more curious about how organizational change works, which fits better in the managerial world. (This is one of the ways in which my current job continues to hold my interest: over the last half-year or year, I’ve gotten to talk to more people in other parts of the organization, and started thinking about how we might do things differently.)

But, for the time being, I think the main aspect that I like about being a manager is that it makes it a bit easier for my voice to be heard, about the organization of the team as well as the organization of the code. And this brings me back to the thoughts at the start of this blog post: I’m pretty sure that I’d be happy in any sort of position where my thoughts were heard, irrespective of whatever formal power my position within the organization gave me. So I shouldn’t be thinking about being a manager versus being an individual contributor: from the former, I should take the idea that I want to be heard, and from the latter I should take the idea that I want to be in touch with the beauty of the details of what I’m working on. And wherever that leads me is okay.

Back to GDC and the video game industry: in the GDC Confab, Michael asked me if I was thinking of entering the gaming industry; my answer was that I wasn’t sure, and one big worry I had was the industry’s belief that it’s appropriate to force people to work extremely long hours. (I didn’t have anything to say at the time about the “But What I Really Want to Do Is Make Games” panel, but one of the comments that’s stuck with me was a panelist saying, basically, games journalists are already used to working ridiculous hours, so that’s a way in which they fit right into the game development world!) I don’t think that’s a good way to produce software in general—I’m pretty sure I can’t maintain disciplined, creative thoughts on one project for more than 40 hours a week—but even if it were, I’m not so obsessed with my career that I’m willing to let it dominate all other aspects of my life, my family in particular. I’ve recently heard the statistic claimed that the average amount of time people spend in the game development industry as a whole is five years, while I’ve already been at a single job for longer than that; the humane working conditions are a big part of that, and any industry would have a hard time learning and growing if it churns through its workers as quickly as the games industry apparently does.

It is certainly the case, however, that a lot of what I saw at GDC excited me. People were talking about all sorts of interesting things, things that I don’t necessarily get a chance to think about so much in my current job. And its artistic nature is a big draw for me: I’d like to work on something beautiful, something that nourishes my soul. Though, to be sure, code can be beautiful and nourish my soul, even if the beauty is hidden within the product: it may well be the case that I’d get more out of revealing the beauty hidden within some of our code at work than working on something more overtly artistic! Also, working on games would be a change of pace, which I like; I’m sure the fact that my role at work has changed significantly every year or two is one of the big reasons why I’m still there. (And has been changing again over the last year, the last few months, even seeing the seeds of new ideas over the last couple of weeks.)

If I had to say what sort of position appeals to me most right now, here’s a stab at what my list would look like:

• I’d be getting my hands dirty with software
• but would be able to think and talk about aspects of the product beyond just the code
• (and have people listen to me!)
• including the structure of the organization.
• (Said organization would ideally be small,
• and in particular I’d be on a small team.)
• That structure would be an agile one
• or, better/broader yet, a lean one,
• including working at a Sustainable Pace.
• (Working close to home would be good, too.)
• The product would be a work of art
• (a game would be nice),
• working on it would nourish my soul.
• And I’d be learning something new,
• both technologically
• and at a domain level.

Which is a lot to ask for. (But if there are any companies in Mountain View using agile methods to write soul-nourishing games in Erlang, please get in touch!) In the mean time, I’m doing pretty well on that checklist right now…