playtesting and design process, pt. 2

I realized I wasn’t done from my previous post. I’m trying to work out whether Ben’s post makes me watch to change my own process. Heavier doses of playtesting is more useful for my process now, but probably does introduce its own turbulence.

agile and iterations and synergizing

My design process is obviously impacted by my experience in software development, which is unsurprisingly heavily “Agile”. This is a process that’s about lots of flexible, repeated iterations of work. Instead of emphasizing an ironclad design up front, you keep producing working prototypes and “iterate” repeatedly, adjusting your direction based on the previous iteration.

I know this provably produces quality software, but there are obvious limits. In terms of game design: sometimes what you create by only building part of a game is too limited to give you real feedback. (Imagine testing Mafia Wars if your iterations has nothing but the ability to buy and collect from real estate. Does this teach you much of use?)

Even when you’re creating agile iterations of a product, I’ve found some useful caveats. Its critical to have a strong vision of what you want (even if you are willing to change that vision); I’m distrustful with iterating in order to uncover a vision.

It’s also a very useful skills to be able to decompose the big, complicated system you invision into multiple steps/layers/tiers. This creates useful checkpoints in your design without requiring you to put it all together at once. You probably dont know what your overall game is like until you have it all in place, but it’s good to say that components A, C and D are basically stable, even if components B and E are in shambles.

real world

As it turns out, I’ve had some freelance software work that involves game design, and some of these discussions – when does playtesting work? how much design up front? – have come to the fore. I expect that my thinking on this will evolve in the next few months.

playtesting and design process

Ben has an interesting post: “Stop Playtesting”. More specifically, he’s targetting where playtesting is actually toxic to a design process.

Rebuilding This Warp Drive In Mid-Flight

So thinking about Spacerpunk: it has evolved in a decent fashion as I put the game in front of good play groups, and it has changed significantly in response to the group’s feedback. However, caveats to this:

(1) I have come to carefully filter player feedback to tell the difference from core problems with my game vision (that I should change), vs personal preferences or mismatched visions (which I should probably not change in the game itself, but certainly could drift in an actual game to suit the group; but this doesn’t get me further in terms of design).

(2) I try not to immediately jump on feedback until it has recurred in a multisession pattern. (Or unless it is so catastrophic and clear to be unavoidable.)

(3) Interestingly, my resolution mechanics have been all over the place during the design of this game, but the structure of the setting and player situation has been pretty steady. I suspect that the fluff of my game is the important part!

It’s obvious, though, that playtesting mid-design can result in jittery, mixed feedback from players who may not share your interests or may simply have a skewed experience at the table. Scrambling at the whim of every respondent quickly gets you to Design By Committee and, thus Design Hell; I get there easily enough on my own.

Drinking My Own Kool-Aid

I think a flaw in my game design abilities – perhaps due to a relatively unrefined process? – is that I’m not the best predictor of how a system or mechanic will work in a social context. I easily get caught up in the narrative of what I’d like the system to do, and do not predict what is the more obvious course of action.

A key example of this exists in The Dance and The Dawn. I initially had an economy of Ice Tokens where players were rewarded for being cruel by sabotaging other players (but at the cost of arming them with the very resource they had just used).

I imagined this could have lead to some delightfully escalating bouts of sabotage with a sort of prisoner’s dilemma, but in practice the currency did not motivate well. There was so little incentive to attack other players – and a near guarantee of retribution – that abstaining from the Ice Tokens was the most obvious strategy. Players only sought out Ice Tokens if they wanted to engage in sabotage for aesthetic reasons, rather than competitive ones.

I’d like to think I’ve gotten better at this kind of objective analysis, but I think my playstyle also impacts my ability to playtest. I instinctively drift a game to fit the style of a group and engage with mechanics on a somewhat emotional/fictional level, rather than trying to optimize my play. (Ex: suboptimal raises in Dogs in the Vineyard to reflect my fictional intent, rather than playing to win and creating suboptimal fiction.) When I’m not more stringent than this, the result is brittle design.

math is hard

I agree that playtesting is an expensive place to figure out how mathematical probabilities work out. I wish it was easier for layfolk to figure out probabilities – hell, I generally just write a script to generate results naively and take the average – but it’s still vital. If you have a rule that’s “keep doing X until you roll doubles”, and you don’t know whether that averages to 2 die roll or 12, then your design is missing something huge.

Math doesn’t have to come first of course, and thinking that way probably doesn’t work for most designers. If you want to defer on the probability work, design your game and, once it has taken shape, take note of what kinds of probabilities you hope for each mechanic to produce. (Is it more like 50/50 or a sure thing? Do you steadily get better, or does your chance of success cap out somewhere?) You can doublecheck the actually produced probabilities, and where necessary work backward from desired probability to a mechanic that will work.

(Hey readers: what tools are there for figuring out math probabilities in an easily accessible way? I write stuff in Ruby, but that’s rather niche.)

ETA: Continued in part two.

Test early and often!

I’ve been working on two games, and even had a chance to put them both through an early test. I got great feedback: both tests revealed some critical problems in the rules thus far. I’m thankful for this greater insight at an early stage. (It definitely helps that despite being “failures” in some sense, both instances were fun.) Let me first introduce the two games, which we may see again soon.

Governor: Rise to Power. This game is about the leaders of a young planetary colony, how they seek power, and how the world changes as a result. (Certainly influenced by Agora, as well as video games in the tradition of Alpha Centauri and Civilization.)

Wushu: Remix! Little relation to the martial arts form, actually. This is my recurring attempt to reconfigure Wushu (a game that’s good for high-action stories) into something that better suits my tastes.

How’d the Governor test go? This was actually a solo test: me playing through my own ruleset with actual dice (well, virtual dice). I found that the mechanical representation of factions gave the players a good way to interface with and care about the fiction. However, once a conflict began, the core mechanic revealed some “Death Spiral” behavior that was fun at the time, but would probably be rather frustrating the second or third time. Moreover: in play, it became more clear that the mechanics, as used, weren’t focusing on the right aspects of the story I wanted. (I’ll post more later about the mechanic I used, and some alternatives I’m looking at.)

How’d the Wushu test go? This was a full playtest I ran for some friends in a scenario and setting we pulled together ad hoc. The game itself was insane fun, in the way that Wushu tends to promote over-the-top crowd-pleasing action. My new rules worked better than expected, but not good enough. The new structures I put into place, while meaning one thing in my notes, emerged as something different when translated into actual play. Your game text can say what it likes, but attaching meaning to in-game events is something that as an emergent property of the gameplay. (The next iteration of this will probably detach from the Wushu ruleset, finally.)

So, both tests actually let me refocus on what mattered to me and the other players, and which things were ultimately not so important.