(Disclaimer: I in no way speak for Playdom.)
There’s a certain strain in the criticism of social game development that has always sounded bizarre to me, and I think I’ve finally understood why. It’s really a two-fold strain: on the one hand, it complains about the metrics-driven approach to game development that social game studios typically take, and on the other hand, it complains about the focus that social game developers typically have on how exactly they’re going to make money. With these two hands frequently united in an allegation that all that social game makers care about is whether a feature increases or decreases revenue.
These complaints were so foreign to me that I ignored their existence for quite some time, until some smart people that I have quite a lot of respect for caused me to realize that yes, many people really do have that point of view, and it was one that I should take seriously. And, once I started taking it seriously, I realized that it’s actually a completely natural point of view! I like making analogies on this blog; and if I replace video games by, say, books or music, then the idea of a metrics-based approach and a business focus would sound pretty weird/unpleasant to me, too. So what I’d discovered was that I had a pretty big blind spot in that area myself; where did it come from?
The answer that I’ve come to is that it’s because I’ve spent the last seven or so years immersed in agile software development; and a metrics-based approach and a business focus both fit into agile quite naturally. One of agile’s key tenets is that you should have a precise notion of what ‘done’ means: agile recommends this in your tight coding loops, with TDD; it recommends this at longer-term levels, with acceptance criteria for stories. And a metrics-based approach fits into that point of view very well, helping to extend the importance of ‘done’ from the programming side to the product owner side: it encourages the product owner to also have green bars of their own that their feature prioritization process is trying to attain, it encourages them to come up with formal criteria that they’re trying to satisfy.
Furthermore, looking at metrics is a way of paying attention to what your customers (and potential customers) are actually doing. And this, too, is firmly established within agile: XP has a “Customer” role, with a strong suggestion that that person be an actual customer of the product. This, of course, doesn’t make sense in all circumstances, but even when it doesn’t, similar ideas appear: e.g. Toyota famously sent its designers of their first minivans and first Lexus models to live with their target customer base.
So, the upshot is: paying close attention to customer-focused metrics makes a lot of sense from an agile viewpoint. Indeed, not doing so when such metrics are easily available (as is the case for web-based software) would be rather odd.
That’s the metrics side of this puzzlement; what about the business focus? A business focus is, honestly, something that I couldn’t have cared less about when I was younger—I was all set out to be an ivory-tower academic—but here too, agile was an awakening for me. Take the Customer concept I mentioned above: that person may not be an actual customer, but she’s definitely somebody with the business goals firmly in mind. This business/engineering collaboration is firmly established within agile: the business side sets the priorities, the engineering side takes charge of getting those priorities done. And both work together to proceed in an iterative approach, to deliver business value as quickly as possible and to discover actual (not hypothetical) business value as quickly as possible, allowing them to sculpt the product’s goals to meet business needs.
And this, it turns out, is pretty fascinating. It’s especially fascinating for me because of my personal history: I worked on the same product for several years before joining Playdom, and it was never a business success. Some of that is the fault of engineering execution (for which I shoulder as much blame as anybody), but a fair chunk of that is that we never managed to find a way to engage the market that allowed our engineering strategy to work. And that wasn’t because people thinking about the business side of the product were stupid, or made obviously bad choices: it’s because it’s a really hard problem to solve. So, from that point of view, working at Playdom is a huge relief: I can frequently point to something that I worked on last week that is making money for us this week, and that feels great.
So: agile has trained me to like metrics, agile has trained me to take a business focus. But where does that leave art, where does that leave craft?
Here, too, we run into a blind spot of mine: I wouldn’t normally think of those (especially metrics) as being in any way antithetical to art or craft. The reason for that again, of course, being agile: it has strong statements in favor of the craft of programming, paired with quite a lot of explicit guidance to that end. My favorite agile practice is TDD, and craftsmanship is right there on an equal footing with measuring your results: yes, you have to come up with a metric (“red”), but you also have to make your code look good (“refactor”). And lean also has an extremely strong craft focus.
It is, of course, the case that working to satisfy metrics and business concerns puts a constraint on your choices. But this isn’t inherently antithetical to art or craft: in general, I think that constraints are as likely to help art as they are to hurt it.
So, that’s where my blind spots in this area come from: my recent background means that certain questions that are natural to others just aren’t questions that I’d be likely to think of. At least if I’m primed to be in an agile mindset by being in a programming context; as I said above, I’d be quite likely to raise those complaints in, say, a literary context. Which raises the question: which is the best model to consider here?
And, to that, my best answer is “it depends”. If you’re driven to create games that come out of your experiences and vision, if you couldn’t care less what most people think or whether or not you make money, then I will in all sincerity say that that’s wonderful. (On which note, everybody please chip in some money for Jordan Magnuson’s Game Trekking project!) Even when doing so, you may find the occasional metric useful—see this report of how dramatically differently users may approach our games from what we expect for an example of why—but you may not.
But, for better or for worse, most video games take rather more work to produce than most books. So I personally would not be comfortable working on a project that multiple people depended on for their livelihood without a hard look at business considerations. (Heck, forget “multiple people”: I wouldn’t be comfortable working on a project that I depended on for my livelihood without a hard look at business considerations.) I don’t care about business considerations at all when writing this blog; I care about them a lot when working.
Our art form will become better the more tools there are in our bag. Being aware of tensions is great; treating tensions as creating separate worlds, not so much.
Post Revisions:
This post has not been revised since publication.
Posts
This seems like a very timely article, as I’ve recently seen some Agile-related terms (like references to the morning “standup” meetings, for example) start to leak into various game development feeds in my Twitter account. As you’ve pointed out, there are a number of desirable qualities in Agile development that line up very well with the environment of social game development; the development teams are (presumably) very small, the technological foundation of the product is incredibly stable, and any rapid changes in the consumer environment can likely be addressed through tuning changes that easily fall into the usual “sprints” toward delivery that happen through an Agile process.
I wonder how applicable the Agile SDLC is to larger game projects, though, especially projects that generate one extremely large fire-and-forget AAA game deliverable. It’s not impossible to undertake Agile on larger projects, but the increase in scale brings about a commensurate increase in risk on multiple fronts: more requirements mean more opportunities for scope creep and mismanagement or leaks in traceability, larger teams mean more possibilities for hiccups in chemistry and more difficulties for efficient management of resources in concert, etc.
Integrating Agile into large-scale game development isn’t impossible, of course, and there are some advocates (such as Clinton Keith) that have written articles outlining how Agile policies like Scrum can fit into a larger game development context. But, on the surface, it doesn’t quite seem to be as easy of a fit as it is with the social game development environment and, given how risk-averse many of the game publishers are with their larger projects, it would appear that Agile would have a long road ahead versus the traditional waterfall process for the time being.
At any rate, I enjoyed your post…and it wasn’t just because I happen to be a software engineer who enjoys talking shop. :)
As we start to develop a more robust language for game criticism and analysis, I think that an understanding of the creative processes that produce these games can become incredibly impactful to the discussion, especially as the media that carries these games changes and evolves. Writings like these can bring some light to ideas that deserve more attention than a first-glance judgment of “player exploitation.”
9/11/2010 @ 10:44 pm
I think there are definitely some aspects of agile that don’t match as well to larger game projects. In particular, the notion of continuously releasing just doesn’t work if you want to release a long narrative arc at once; there are ways to work around that if you want (episodic content), but I think choosing not to do so is perfectly reasonable. (It’s the same as with novels: sure, you can serialize your novel if you wish, but most authors choose not to. Though, of course, in both the novel case and the video game case, you may still benefit by showing incremental rough cuts to your trusted circle.)
But other aspects of agile can be a good match, and you can still rethink how to best get the benefits of iteration with a longer release cycle. My favorite talk of GDC 2009 was the BioWare Mass Effect 2 talk that I took notes on here: http://malvasiabianca.org/archives/2009/03/gdc-2009-friday-bioware-talk/ And, having seen how Mass Effect 2 turned out (both in terms of quality and schedule), they’re clearly doing a lot of things right.
Amen to your last paragraph; it’s an area where I would certainly like to do better. It also fits with one of the gaps I see in agile: it doesn’t devote much time to giving explicit guidance to how the product side of the business should carry out its craft, beyond a general recommendation to proceed incrementally and recommendations for how the team should interact. This is in stark contrast to its detailed recommendations for the engineering side of the team; given how productive I’ve found the latter, I’d really like to understand better if there are similarly useful recommendations for the creative processes on the product side.
Thanks for the thoughtful comments, I really appreciate them!
9/12/2010 @ 9:44 am
I hope you’ll write more on this subject in the future. It’s fascinating to hear your perspective on these two increasingly important issues, especially given your years of experience. There’s a host of very intelligent people discussing this subject, but I’ve been hard pressed to find honest critiques of the prevailing wisdom.
I agree with the central points of your post. I take these to be: (1) listening to metrics can maintain focus on the user experience and (2) constraints can–and often do–benefit craft. The first of these is especially appropriate when building products that run on user satisfaction. Metrics can be incredibly helpful if they’re used wisely. The second–business priorities–has driven the development of every modern mainstream media format. Pursuing business goals often produces a great product in unexpected ways.
However, a lot of the criticism directed at social game development seems to be rooted in the perceived lack of a mature infrastructure to guide developers as they attempt to make use of these new tools. My entertainment background is in film, which isn’t a stranger to these issues. Brilliant and creative people have struggled for decades in the film industry to support the integrity of their craft against perceived attacks from focus testing and business priorities. Nevertheless, determined people continue to create great works in this conflicted environment.
I have little doubt that game development will, in time, find its own way to reconcile these concerns. From an outsider’s perspective, it seems that the most probable solution is for executives, producers, developers, and designers to take a more holistic approach to metrics. I’ve seen a crass focus on lackluster data damage good decision making in game production. However, if metrics are interpreted carefully, they often counsel actions that benefit the craft of the game, the user experience, and the priorities of business at the same time. I have faith that, as game design matures as a profession, social games will find their own equilibrium.
9/12/2010 @ 1:54 pm
Thanks for the encouragement, both here and in our conversation on the VGHVI night. And yes, we don’t have a mature infrastructure in this area, and developing it is a very important task. I’d just like to see us work on that by exploring surprises (especially positive ones) a bit more, and by not writing off broadly applicable tools/approaches unless we’re very sure indeed that they’re a bad idea.
9/13/2010 @ 10:26 am