The fellow on the right looks like a pretty nice guy, no? Well, take my advice: don’t go to a party where he is lurking outside.

Meet Paul and Ian, my elite SpyParty playtesters. Both have played around 10 hours of SpyParty across multiple sessions. They’ve played the game more than anybody else, including me if you don’t count my daily code testing. Paul, on the left, is a designer on DarkSpore, and he specializes in playing the Spy. Ian, a designer on The Sims, is partial to playing Sniper.
When you’re trying to make a really deep player-skill game, you need to have people like Paul and Ian who will play your game a lot and get very good at it. I’m really lucky to have these two friends who are happy to play my game so often and for so long. Given my depth-first, accessibility-later development philosophy, I am basically designing the game for these two guys. If anybody else has a good time playing, that’s a great bonus, but right now, Paul and Ian are the most important SpyParty players in the world. If I come to a decision where I have to choose between making the game better for newbies or better for Paul and Ian, I choose the latter. It’s an audience of two.
For most of the development, everything was rosy, and Paul and Ian were evenly matched. This was a huge relief, because SpyParty is a highly asymmetric game, so balancing the Spy versus the Sniper is a subtle and difficult challenge. Every time I’d add a significant feature, I’d test with Paul and Ian, take a bunch of notes, go back and tune the feature if it was unfair (sometimes while they waited), and then we’d test again until it was balanced. The fact that they’d played the same amount, and they’re both ultra hard-core gamers and very competitive, meant I could trust that if they were each winning about 50% of the time, I had a balanced mechanic.

The Good Old Days
Naturally, when I got the Action Test feature working, I called up Paul and Ian and the rest of our playtest gang and had them come over to try it out. We ordered pizza, and everybody had a good time casually testing the new feature and discussing its pros and cons. Finally, it was time for Paul and Ian to sit down and see what impact the Action Test had on the elite level game.
They hadn’t played each other for a few months because I’d been doing so much traveling, but last time we played they were at the same level, so I figured after a few warm up games they’d be back to form and the intensive balance testing could begin.
Except Ian kept shooting Paul almost instantaneously. Over and over again.
What the heck was going on? Had I screwed something up with a recent feature? Did the Action Test actually do the opposite of what it was supposed to do? Had Ian planted a mirror behind Paul?

Spy Bloodbath
I decided to sit down and play the Spy. I’m not really a fair person to play against, since I wrote the code, and I know exactly what percentage of the time people take books from the shelves, and how fast they sip their drinks. These are huge advantages when I’m playing Sniper, but even for Spy, I can usually beat anybody. However, even though I did a bit better than Paul, Ian managed to kill even me over and over again.
Okay, something was wrong.
Most people who hear about the game think the Sniper must have an advantage, but this isn’t actually true. The metrics are pretty clear on this: even for newbies, the Spy and Sniper are pretty evenly matched. For example, in the recent playtests at NYU, the Snipers won 48% of the time to the Spy’s 52% over 188 games, and at an even more recent playtest at Valve, the Spy won 55% of the 104 games played.
However, at PAX, an observant fan in line noticed a trend…the Sniper was winning more and more as the show went on. Day 1, the results were 47%/53% Spy/Sniper. Day 2, the score was 35%/65%. Finally, on the afternoon of day 3, the issue was noticed and debugged, just in time for the tournament, but overall the day was 36%/64%. What was going on was quite interesting and subtle: because there was a huge line with a long wait, everybody had plenty of time to read and digest the Four Page Instruction Manual and then watch a whole bunch of games before they got to play. This, it turns out, is Sniper practice. The Spy is a kinesthetic experience; you need to be holding the controller to get better at playing Spy. But the Sniper is mostly about perception, so simply watching 10 or 20 games while you waited in line, seeing the various tells from different angles, and watching what worked and what didn’t for each side, gave the Snipers a huge advantage. We fixed the problem by not giving verbal help to newbies playing Sniper, figuring they’d had enough practice while waiting in line, and only helping out the Spies. This was enough to restore balance for the people playing the game for the first time.
However, did you happen to notice anything about all the pictures from the PAX days linked above? Here, let me help you out with a couple examples:
Our pal Ian was one of two friends who helped out in the SpyParty booth the entire three days of PAX. Without Ian and Tom doing the bulk of the work, I’m sure I would have died right there on the show floor.
Well, this means Ian got something like 25 hours of additional Sniper practice over the course of PAX by tutoring people and observing games the whole time. Look at him in those pictures, sucking up information about how the NPCs move and how often they do and don’t do things.
This is the current best theory explaining why he’s winning all the time now. He’s an unstoppable machine because he’s played three times as many hours as the next best player.
This is good news and bad news. The good news is if this theory is true, the game is working exactly as I’ve designed it. He should be beating Paul (and even me) every game now, because he’s played so much more, even if his hand didn’t touch the controller more than a few times at PAX. I’m designing the game to have an intense player-skill ramp, and this is what should happen if I’m successfully achieving that design. The bad news is it’s hard to know if the theory is true without me or Paul practicing enough to get to the same level, which is a lot of practice!
So, I’m going to need to practice my own game, which seems a little cosmically unfair, but such is life when you’re striving to make a truly deep game, I suppose.
I’m also using this as an opportunity to fix as many of the subtle places where the NPCs and the Spy diverge due to lazy coding. Ian noticed some of these during PAX, and I need to deprive him of that advantage.
An example of this kind of divergence is the Ambassador would only have his or her personal space violated by the Spy character. Ian observed that the Ambassador would run away sometimes, and deduced this was a tell. Now the Ambassador will move away from any character that stands too near for too long. This is subtle stuff!
I’ve also changed the way the conversation groups work. Now NPCs look for open areas, and randomize within the entire open arc, rather than just aiming for a slightly random point near the center of the arc. Also, there are no longer any hard coded maximum number of characters in a given conversation group, the NPCs look for an opening big enough, and if there is one, they can go for it. Ian was asking me the other day about how many people would be in a conversation at once, so now there is no straight answer to the question. Take that, Sniper! There is a probability table that tells NPCs how likely they are to want to join a conversation given the number of people already there, but I’m not going to publish that, sorry Ian.
One thing I’m not going to fix is how NPCs won’t go out of their way to be near the Ambassador, while the Spy needs to sometimes. This is happening at the behavioral level where I want the game to live, so the onus is on the elite Spy to bide his or her time waiting for the Ambassador to be in a plausible position to walk near.
One other good thing to come out of this is an interesting modification of the “3 out of 4 missions” game mode. When newbies start playing, they play with the Spy required to do 4 out of 4 missions; it’s too overwhelming for newbie Snipers if the Spy doesn’t have to do everything. Once the Sniper starts winning a lot, usually around 10 or 20 games in, we switch to the Spy picking 3 out of 4 missions. With this “subsetting”, the Sniper can’t camp any individual mission, and there is interesting yomi in which missions the Spy picked. However, given Ian’s skill gap, I realized a good new mode would be “any 3 out of 4 missions”, where the Spy doesn’t have to pick the 3 up front, all 4 are available, and the Spy can be opportunistic during play. This should be slightly harder for the Sniper because now the Spy has more options, yet still only has to do a subset of the missions to win. I’ll need to playtest this to be sure, but I think it’ll work well at elite levels.
In the meantime, watch out for this guy:
