I just wrote this post for the Bugs subforum on the Early-Access Beta site.  I figured you all would be interested in reading it, since you’re all going to be invited into the beta soon.1

"he probably would"

Finding and reporting bugs is a learnable skill, and you don’t have to be a programmer or even a particularly technical person to do it right, you just have to be methodical and thoughtful. Reporting bugs the Right Way means I can spend more time making SpyParty awesome and less time trying to get a bug to “repro” locally.

What is a “Repro”?

The goal of a bug report is to describe the steps to reproduce the buggy behavior (whether it’s a crash that deletes your entire hard disk or it’s a word that’s not pluralized correctly in the user interface somewhere) well enough that I can step through them on my machine and see the behavior first-hand. A bug that doesn’t repro is very hard to fix, whereas a bug with a “good repro” is usually pretty trivial to fix. It can be the difference between 10 minutes to fix and 10 days to fix, literally. So, when posting a bug report, the key is to do some work on your end to be methodical and give steps to repro the bug as simply and as clearly as you can. It’s basically a recipe to make the buggy behavior happen for you (and me!).

This does not mean you shouldn’t report bugs that don’t always happen the same way or happen infrequently, or that you can’t get a good repro for, but it does mean those bugs will be much harder for me to fix. In other words, it’s more important to report the bug poorly than it is to not report it at all, but a good repro makes a huge difference.

Bug Reporting Steps

I’ll be updating these steps as the beta proceeds, and eventually I’ll switch this part of the forum over to a real bug database, but for now, threads in this forum will have to do. The basic rule is one thread per distinct bug.

1. Try to reproduce the buggy behavior on your machine. If you crashed, write down what you were doing right before the crash before you forget it, and then relaunch the game and try to get it to crash again. If you can’t repro the bug, you should still report it with as much description as you can, but as I say above, it’s way easier to fix bugs with good solid repros. The shorter the recipe for the repro, the better. As Einstein said, you want the simplest possible repro, but no simpler. A bug that repros in Practice Mode is easier to debug than one where I have to connect over the network, for example.
2. If screenshots look like they’ll help to repro or describe the bug, feel free to take them (you can do it by hitting F5 when the game is running, or use FRAPS or your favorite screenshot tool). If you hit F5, the screenshot is put in your AppData/Local/SpyParty/screenshots directory, which you can get to by going to the Start menu and typing “%LOCALAPPDATA%\SpyParty\screenshots” into the little edit box, and hitting return. I’ve enabled attachments in the forums, so please attach the images to your bug report (as opposed to using a 3rd party image service, which might delete the images after a while).
3. Make sure you have the latest SpyParty build. The easiest way to do this is to launch the game and hit “Play” on the main menu, which will try to take you to the lobby. The lobby will update your build if you’re out-of-date. If you update, try your repro steps again to make sure they still work.
4. Search this subforum for similar bugs. If you find the same bug, then read the thread on that bug, and see if there is a workaround, or if I think I’ve fixed it in an update, etc. If I think I’ve fixed it in that thread I will mark the subject with ” – FIXED”, and I really want to know if you have the same bug again! This is called a “regression” and it means I’m a bad programmer!
5. If you find the exact bug, feel free to add more information at the end of the thread. I will try to add a “vote for posts” feature so you can simply say “me too”, but until then, feel free to add a post if there aren’t already a bunch of others doing the same, or if you have additional information or slight modifications or a simpler repro that might help debug the problem.
6. If you don’t find an exact match in the current bugs, but do find bugs that seem related, keep the links to those threads so you can insert them into your bug report. A real bug database allows you to reference other bugs, so this is our ghetto version of that feature.
7. Create a new thread in the bugs forum if your bug is a new one. Use a descriptive thread title. “Game buggy” is a bad one. “Main Menu text disappears after window maximized” is a good one. I might change the title after we’ve discussed the bug a bit, but starting off with something descriptive will help.
8. At the top of the post, describe the end state of the bug, meaning what’s wrong, like “the game crashes”, “the statues are floating in mid-air”, or whatever. Describe what you expected to happen, and what happened instead. Attach an image showing the bug, if appropriate.
9. Then put your repro recipe in the post, written as clearly and as simply as possible. You can use the bbcode list tags, or just write it out, either works. Attach any more images you have to show how to repro the bug, but you don’t have to go crazy (saying “hit Play on the main menu” is fine, you don’t need to screenshot the main menu, take it into Photoshop and add an arrow with “click here” text unless the exact place to click matters to repro the bug, for example).
10. SpyParty writes out a log file every time it runs. You can find these log files in the “logs” directory (“%LOCALAPPDATA%\SpyParty\logs”). The file name for the log files has the date and time of the playtest session start (e.g. SpyParty-20120412-11-54-49.log). Attach the log file that goes with you reproing the bug with the steps you’ve posted. Similarly, if your game crashed but said it could not upload the crash dump, then attach the correctly dated dmp file from the logs directory. Otherwise, SpyPartyHelper uploaded the dmp file to my server so I can access it, but mention this in the bug report.
11. If there’s private information you need to send with the bug report, mention this in the report itself, and then send mail to support at spyparty.com.
12. If you want to attach a really big file, say over 2MB, please mention this and I’ll figure out a place for you to put it, please don’t mail giant files to support at spyparty.com.

That’s it for now! Feel free to reply to this post with suggestions of how to improve the bug reporting process and I’ll edit this post.

Eventually I will add a way to report bugs from inside the game that will take a screenshot and everything automatically. Won’t that be nice?

1. for a suitable definition of “soon” [↩]

I decided at pretty close to the last possible minute to have a small “thank you” party for the first set of SpyParty beta playtesters, and I invited some friends from the press who have been covering the game for a while.  Scheduling things during the evening at a big tradeshow like GDC is basically impossible…there are so many other conflicting events and parties and meetings that you have to just pick a date and go with it.  I picked Tuesday night, and tried to make it a short party, scheduled in-between other big events, but in a bizarre twist of fate I still ended up landing right on top of the SimCity 5 launch event, a game being made by all my old friends from Maxis.  Oops.

Anyway, the idea behind this party was to get the hardcore playtesters playing together in-person and say “Thanks!” with beer and snacks, but also to get them mentoring the folks from the press, who haven’t seen much of the mid-game yet because they haven’t played enough games.  I don’t know how well the mentoring thing worked out since it was not exactly a quiet contemplative training environment, but everybody seemed to have a good time, and that was the main goal.  Huge thanks to Jonathan Blow for letting me use his suite for the party, and to Luis & Mylene and John & Alice for bringing the food1 and drinks!

I took some pictures, maybe you can recognize some playtesters and press…

And, now we’re in the home stretch for the next batch of invites.  I’m plowing through the list of bugs and feature requests from the first batch, and the next invites are going out really soon™.  Here’s the current todo list:

• put portraits up along side for cast characters
• also spy!
• how to center text? allow two lines worth of space? uneven spacing?
• have box with ? on side for casting characters – progress status and neat
• don’t get that click on guy selects during cast characters
• character/level chooser mouse tip for select versus next/prev
• mouse overrides cursor keys in menus if over item…don’t override if key change but no mouse change
• why is volume not working on last 30 seconds timer on sniper machine?
• sniper review tells on waiting screens – sort of
• basic infrastructure for this
• chooser through animations playing on loop!
• text title and description support
• require sniper okay to continue
• bug_ambassador
• double_agent
• inspect_statues
• seduction
• steal_statue
• transfer_book
• check_watch
• mouse clicks are going to background window on spy side
• does spy side keep mouse cursor on results screen?
• sniper camera moves on results screen
• pgup->PageUp
• fix statue pad size
• multiple affordances controls
• display bumpers/QE/mousewheel?
• update action test text as affordance changes for Signal Double Agent?
• or kill it if it’s not an active affordance, so have to restart?
• document action test is same button?
• put small icon next to it?
• update while in lobby
• Update Failed on download bar…needs to test more
• remove briefcase verbs? or ignore them?
• write forum welcome/intro/faq posts
• update readme, or just empty it and point to the focum post?
• how to report bugs post
• post threads for known bugs
• tutorial video, walk through Spy and Sniper practice mode as if I was showing you how to play at a playtest
• ctrl-delete delete next word, use ctrl-right code…uh, and fix that
• updated lobbyserver won’t accept old connections, even to update, idiot!
• start sniper while other on results -> no -> sniper -> assert(level) – yay, found it!
• needs to zero play id on starting sniper because still current
• need to fix gameid/playid thing…I’m constantly stomping the gameid
• store gameid/playid pairs from server, then look up based on playid
• store playid on server as well
• can’t escape from sniper results after play attempt? IsLobbyGameOver busted
• add string wrap to property parser, or general line continuation?

I haven’t decided how many people to invite in the next batch, but it’ll be somewhere between 20 and 50, depending on how confident I’m feeling.  Yikes, 50 strangers from the internet!

2012-03-23 15:19:45 – Crossed some stuff off.  Working on the missions preview code, so people will be able to see what all the tells look like before playing.

2012-03-30 13:24:10 – Added a bunch of stuff I’ve been working on that wasn’t on the posted list!

2012-04-03 17:18:30 – Well, it turns out the mission tell preview took a lot longer than I thought, and I still didn’t manage to hook it up to the Sniper waiting screens, so it’s just in the practice menu for now.  I shouldn’t have done this one now, but once I started I failed to stop working on it.

1. Mylene’s homemade guacamole was awesome! [↩]

Cast Character Display

Update:  If you ended up here looking for the open beta, I have no idea why wordpress is sending you here when it should be sending you here. Sorry, and I’m trying to figure it out!

So, I’m motoring along through my beta todo lists,1 and I start to investigate a bug Rachel found during a recent playtest:  “suspected double agent on rollover but no triangles?”

Seems simple enough, and I’m able to reproduce it quickly.  Allow me to explain the bug, and then the naive fix, and then the real fix that doesn’t mess up the game’s tuning…

Missions, Casting, and Game Modes

First, some missions have cast characters, meaning the Spy (or the code) gets to pick a character at the party to cast into a role, for example the Ambassador, the Seduction Target, or the Double Agent.  These cast characters are displayed to the Spy and Sniper with a triangle over their heads, and the Sniper also gets a bit of rollover text when the target is on the cast character.  You can see an example to the right. 

The Sniper sometimes doesn’t get to see the correct cast character, so while both players get to know which character is the Ambassador, only the Spy gets to know who the Seduction Target is.  For the Contact Double Agent mission, the Spy gets to see the real Double Agent, and the Sniper gets to see multiple Suspected Double Agents, and doesn’t know which one is really it.2

There are also multiple game modes on most maps, and some of the game modes are known as subset modes, where the Spy can select for play a subset of the total available missions for the map, and then optionally enable a subset of the selected missions.  This is for more advanced play, and subsets prevent the Sniper from camping a given mission, or at least make it a very high risk strategy to do so because he or she can’t be sure the Spy is actually attempting that mission.   The Sniper can see which missions the Spy has selected, but not which he or she has enabled, so in the shot below, you can see the Transfer Microfilm, Swap Statue, Inspect Statues, and Seduce Target missions are selected, but the Spy only has to do 3 out of 4 of them.3

The Spy’s Selected (but not necessarily Enabled) Missions

The rollover text but no triangle!

The Bug

The bug reproduces when I deselect the Contact Double Agent mission on the Ballroom map, so there is no real Double Agent.  The mission code still randomly casts the Suspected Double Agents and sends them to the Sniper.  The Sniper code does “the right thing” in the code that draws the triangle and checks if the mission is selected, but doesn’t check in the rollover text code, and so you get the latter but not the former.  It looks like the shot to the right.

I obviously shouldn’t be displaying the rollover text to the Sniper for Suspected Double Agents when that mission isn’t selected as available.  Or should I…

The Fix(es)

My first thought was the obvious:  check whether the mission is enabled, and if it is, don’t display the text.  But as I was looking into making that fix, I started thinking about the ramifications of that change.  None of the cast characters can be the Spy (currently), so that eliminates a bunch of people at the party from the Sniper’s point-of-view, including the Waiter, the Ambassador, the Security Guard, and yes, the Suspected Double Agents.  If I fixed the bug and removed the rollover text, it would actually increase the number of potential Spies at the party, and change the tuning to make things more difficult for the Sniper.  The more partygoers the Sniper has to worry about, the more information he or she has to sift through to find the real Spy.

But, if I go the other direction, and just remove the number of Suspected Double Agents from the party if the mission is not selected, then things get easier for the Sniper, because even if the Sniper knows the suspects can’t be the Spy, having that density of people in the room increases the cognitive load.

After bouncing this issue off a friend, it seems like the right thing to do in the near term is to still send over the Suspected Double Agents, and display the triangle and the rollover text, even if the mission is not selected.  This way the number of people at the party stays constant regardless of whether that mission is selected, and the game stays tuned correctly.  It’s actually a kind of interesting solution, because it means the Sniper has just a tiny bit more cognitive load, because the suspects will be at the party, but the Sniper will have to either lowlight them or remember to ignore them if the Contact Double Agent mission is not selected.

Here’s the fix that actually removes the enabled check on the triangle draw code:

=== modified file 'project/spyparty/code/situations/double_agent/double_agent.cpp'
--- project/spyparty/code/situations/double_agent/double_agent.cpp      2012-02-24 03:13:37 +0000
+++ project/spyparty/code/situations/double_agent/double_agent.cpp      2012-03-14 21:20:46 +0000
@@ -414,8 +414,8 @@
     }
-    if(enabled && ( (GameMode == MODE_SNIPER) || (GameMode == MODE_DEBUG)) ) {
-        // highlight the suspects
+    if {
+        // highlight the suspects, even if we're not enabled, else we're affecting the tuning of the map...sigh
         for(uint32 i = 0; i < NumSuspectedAgents; ++i) {

Eventually, once I’ve got enough people in the beta to get valid statistics, I can experiment with other solutions, like removing the suspects. Another thing I’m not sure of is whether I need to display the Double Agent on the Spy view in this case…currently I don’t. It’s hard to see how that would matter since the Spy’s gameplay is so different, but I’m not quite sure.

This was not the blog post I was planning on making today, but I found it interesting that such a simple bug could have subtle tuning impact like this, and decided to share.  I firmly believe basically all game programming is game design, and the little decisions made while writing code are where the game design rubber hits the road, but this one surprised me with the subtlety of the ramifications.  I’m trying to imagine what would have happened if this was a big game with a team of designers, a team of producers, and a team of programmers, and this bug got entered by QA into the database, shunted to a programmer via an associate producer, and then “fixed” and checked off the list.  How long would it have taken to figure out why the Sniper was winning or losing more often than before after the next official beta build rollout?

I’ll post GDC pics next time!

1. Yes, sadly, there are multiple todo lists. [↩]
2. The real Double Agent is always a suspect.  The Spy doesn’t get to know who the other suspects are, just to make things even crazier!  Asymmetry everywhere! [↩]
3. There are actually two kinds of subset modes, one where the Spy has to pick which subset he or she will attempt before starting, and another where the Spy can opportunistically attempt the missions at play time. [↩]
4. GameMode == MODE_SNIPER) || (GameMode == MODE_DEBUG [↩]

I really should be working on my GDC1 talks. Instead, I am writing up yet another beta playtest, so I will make this quick!

On Friday my friends and awesome game artists Luis Antonio and John Cimino playtested.  They were only supposed to play 4 or 6 games, but kept playing for an hour, which is a good sign.  Of course, I took a bunch of notes, so here are the tasks from that:

• don’t get that click on guy selects during cast characters
• mouse overrides cursor keys in menus if over item…don’t override if key change but no mouse change
• have box with ? on side for casting characters – progress status and neat
• why is volume not working on last 30 seconds timer on sniper machine?
• sniper review tells on waiting screens
• chooser through animations playing on loop!
• require sniper okay to continue
• detailed stats stay on mouseover and off…good?
• mode on chooser that says if mouse moves off, and not keyboard select, then help goes away?
• pgup->PageUp
• ambassador was doing tapdance again…is it always the ambassador? I think so…!2
• microfilm get animation is same as put, so you get it out of your pocket, which makes no sense

I need to consolidate the recent punch lists so you folks can see what I’m working on.  I occasionally check items off the older lists in the older posts, which is kind of silly.  Sadly, the tasks on this list aren’t going to get checked off for a while due to the conference this week.

The best anecdote from the playtest was when John shot Luis and then said, “I lowlighted all of the men at the start because you’ve only ever picked women as the Spy.”   Luis realized his meta-mistake, and had to consciously start randomizing his character choices more. That is exactly the kind of know-your-opponent yomi gameplay I want to encourage!

If you notice the laptop to the left in the picture below, John is actually chatting with us using Google Talk.  Luis and John didn’t notice because they haven’t played that much yet and I didn’t want to point it out, but I could clearly hear the Spy’s mouse clicks through the connection, and they are huge tells, so I’m going to have to figure out what to do about that before I enable voice chat as a feature.

SpyParty causes bad posture.

All the auto-update, crash dump, and login stuff seems to be working really well, so I’m super happy about that.

Now, back to my talks!

1. The interview with Notch is supposed to be live-streamed on Gamespot, no link as of yet. [↩]
2. I’ve been seeing this rare bug where one of the characters would slide across the floor, jittering.  I noticed it again during this playtest and realized I always see it on the Ambassador, which is a huge help in debugging it! [↩]

Last Thursday we did a small playtest, the second so far in the Early-Access Beta!  This time it was Rachel, Lauren, Ian, and me, and we played and chatted (and restarted after crashes) for a little over 3 hours.

As it should, the playtest generated a ton of tasks, but this time I actually fixed a bunch of them before I got around to posting this, so here is the pre-struck-through punch list!  It seems much more satisfying this way, if only all my task lists were already mostly done before I posted them…

• coming out of settings menu doesn’t eat the final up and it performs action
• what is invert-y’s key? – was shift-y, made ctrl-y
• notify new chats if pageupped
• /ring bell when message comes in if in background?
• display match or lobby [channel] on /say and /emote
• escape should clear timestamps on all chats if displayed
• floating briefcase
• change white chat help to darker
• stop echoing command on /say /emote /whisper
• also do /reply and /retell local emulation
• chat help on sniper machine adjust up like chat line for portraits
• gameid sync is because server sends, then get request to change, which clears game
• include gameid in request?
• this is because of the clearing of the gameid in ClearGame
• the server gameid comes in before the be_role packet, which toasts it
• uh, undo previous CurrentGameID.Zero() change at least, still see sync issues?
• if page up, then escape, new chats don’t show up, and new chats disappear
• need to reset paging on escape?
• suspected double agent on rollover but no triangles?
• assert IsChatAllowed(), File: .\spyparty.cpp, Line: 5565
• when I chatted when cancelling invite
• go back on selection screens
• enter focus for chat?
• or just use it for chat and fuck it for menus
• fine, make it /say
• enter on empty chat should close
• crash dump!
• pigo says: i hit play again, and when i selected the sniper it crashed right away
• i’m in results, he e2mm, hit sniper, i hit no, he hits sniper again
• no confirm on end match in match menu
• just do SpyWins & SniperWins for lobby stats
• some whipsers getting eaten
• tolower username in game
• MOVEFILE_COPY_ALLOWED

2012-02-29 16:16:25 – I don’t think whispers were actually being eaten, I think it was /say in matches, which now describe which [channel] they’re talking to.  I think.

Of course, there are still some tasks left on the previous list, not to mention the list before that, but this is forward progress.

The funniest issue from this playtest came from chatting in the lobby.  I use Counter-Strike keys for chat, so ‘y’ is public chat, and ‘u’ is whisper.1  Lauren and Ian are big World of Warcraft players, so they would constantly hit ‘enter’ to chat, which when you’re in the lobby would start a match with somebody. About the third or fourth time one of them ended up playing a game in the middle of a chat conversation, I acquiesced and made ‘enter’ a synonym for /say just like ‘y’, and made ‘space’ the key for selecting menu items.2

Maxima

When you’re working on software, you get a feeling when you approach a local maximum, meaning a point where you’ve got the thing working pretty well given its current overall state.  This is always a long way from the global maximum3 but it means the game is pretty robust and does all the things it’s trying to do at this point in time reasonably well.  The current SpyParty builds need a bit more testing before they’re really at that local maximum, but I feel like they’re getting close.  The builds feel like this towards the end of PAX, after I’ve shaved off the rough edges found from 20 hours of playtesting.

This means I seem to be on schedule for the near-term beta rollout plan, which is nice.  I gave the “invite-some-fans-in-before-GDC” idea mentioned in that post a little more thought and I realized I was completely insane to even think about that…I have no time to fix or support anything during GDC,4 so everybody’s going to have to wait until the week of the 12th for more invites to go out.  Sorry!

Of course, with numerical optimization, moving off a local maximum means things get worse for a while until you find another maxima, and that’s true with software as well.  My plan is to simply bring you all along for that ride in the short term.  Once there are enough people in the beta who complain about me breaking stuff all the time, I will probably allow people to opt-in to the more bleeding edge builds, and keep them running on a separate server.  This is a pretty common thing to do to minimize pain during open betas, but it reduces your pain by increasing mine, so I hope I can put it off as long a possible.5

And no, I still haven’t gotten my first $14.26 from PayPal.

1. really team-chat in C-S, but whisper is the closest I have to that. [↩]
2. mouse click works as well, obviously. [↩]
3. if such a thing even exists for works of art and entertainment! [↩]
4. or even sleep, sadly. [↩]
5. especially since my server deployment team is me. [↩]