Late Sunday night, I was looking at the lobby server logs, and saw a login from “rachel” but no logout.  Uh oh, I thought, this must be a bug, since with only 9 people invited into the Early-Access Beta so far, the chance of actually finding another person in the lobby without making a pre-arranged date to play was basically zero.

I don’t have great tools for inspecting the running lobby yet, so I started up a local copy of SpyParty and logged on to poke around.  Hey look, it wasn’t a bug, it was actually my friend Rachel Reynolds, the real live beta tester!  She’d started up the game after registering, and was just checking things out.  She invited me to a game, and away we went!

According to the stats, we were in the match for 1 hour, 30 minutes, and 17 seconds, and we played 10 games for a total of 44 minutes and 31 seconds of playtime.  We spent a lot of time testing the new chat system and finding bugs and things that need cleaning up.  We also spent a lot of time re-learning how to play because neither of us had played since last summer!

Here’s the punch list we came up with from this official First Early-Access Beta Playtest:

• chat
• make local echo look like remote print, no /say
• actually, echo command and print message
• help on cursor line feels weird
• put “hit ‘y’ to chat, page up/down to scroll history” at bottom
• at least until person has done it a couple times?
• nah, just in small text, want to encourage it
• hmm, this is just going to be annoying if it’s always there…
• mask keys from game when chatting!!!
• sniper camera
• spy movement!
• don’t fade messages if app is background
• do a “checker is typing” meta message?
• can only do this out-of-game or it’s a tell
• word wrap text to screen
• uh, where do I cap the length of the chat message?!
• no where…need to do it in HandleKey, and have help?
• need to scroll the chat line too, including the help!
• or word wrap it? yikes
• statue in hand anim bug
• repro: grabbing statue on veranda in overlap area?
• fix statues to use the pad instead of radius
• character/level chooser mouse tip for select versus next/prev
• laser event still sending on sniper finished screen
• add back 3 of 4 known missions on ballroom
• have another menu mission state to pick available known missions
• pick 4 of 6 that will be the available
• then pick 3 of 4 to accomplish
• make sure this journals
• need to store some way of telling which map and game mode in db!
• a single dword?
• document action test is same button?
• put small icon next to it?
• remove briefcase verbs? or ignore them?

Not bad for an hour and half’s “work”!

However, for me personally, the biggest (re)discovery of the playtest was that I really like playing my game!  That probably sounds odd, but in the seemingly endless months of getting the beta ready, losing myself in server code that is necessary but feels very far away from making a work of art and entertainment for other humans to experience and enjoy, I’d actually forgotten how much I love just plain playing SpyParty, and it was great to be reminded of that.

Also, goddamn this game is stressful to play!

2012-03-02 13:59:07 – Just checked off the statue bugs.  Nobody is reading this old entry anymore, though, are they?

I still do not have my $14.26 from PayPal. 

But, I have fixed most of the bugs that Jonathan Blow—the first official SpyParty Early-Access Beta invitee—found, so I guess I’m not going to wait for PayPal1 and hope things are working now.

There have been a lot of questions here on the blog, on Twitter, and on Facebook about how the beta rollout is going to work, and a few misunderstandings based on ambiguous statements I’ve made, so I figured I’d try to clear all that up in this post.

First, and most importantly, everybody who signed up will get invited into the beta, it just might take a while.  Just getting to the point where I could send a single invitation email was way more work than I thought it would be, so it is going to take some time to get things to where I can invite in all 11,147 people who have currently signed up (as of this blog post).

I can, however, say how the very near future of the next few weeks is going to go…

Near-Term

Today, I need to finish writing up some basic documentation, including the Welcome Message in the private beta forums, the How to Report Bugs post in the forums, the Beta Test FAQ, and I need to update the game’s README.txt.  Hopefully I will grind these out in the next few hours.

Then, this evening, I’m going to send out a small number of invitations to my hardest-core and longest-term playtesters, folks like Paul and Ian.  This will be sent to only about 8 friends, all of whom are game developers and who have played a lot of SpyParty already, and who are very comfortable playing with very early software.  This group’s job is to play a lot, to find more subtle bugs—especially if I’ve broken any tuning since I haven’t playtested since PAX West 2011—and to make sure all the new score ranking and player database stuff works.  I’ll be fixing the bugs they report and implementing the features they request on a daily basis, in addition to playing a lot myself so I can stay even remotely competitive in my own game.

While these folks are testing, I’m going to finish the Invite-a-Friend feature, so they can test that as well, which will mix a few more people into the beta.

At this point it’s probably around March 1st, and I’m hoping the extremely-low-hanging bug/feature fruit will be mostly picked and everybody will be having a fun time playing and I’ll be ready to invite some more people in, including people I don’t know personally.  Unfortunately, this is also when GDC hits, which means I lose a week of productivity, so I’m going to hold off on new invites until after that’s over.  I’ll be doing a little SpyParty press at GDC, but I’m going to keep it pretty mellow.  I also might set up the laptops at some parties again, not sure.

After GDC, or maybe just before GDC if I’m feeling really confident and things are going swimmingly, I’ll invite in a small group of the earliest and hardest-core fans from the invite list, the blog, Twitter, Facebook, GDC, and PAX, and a small hand-picked batch of press who’ve been covering the game closely since the beginning.

Then the real bulk invites will happen shortly after that.  I’ll start with 20 people from the beta sign up list, mostly in order from the beginning, but also probably randomizing 25% of them from the whole list so at least a few people won’t have to wait quite as long to get invited.  Then, once the dust settles from those 20, I’ll invite 50, and then 100, and so on.

Less-Near-Term

At some point, my server will start melting from too many simultaneous players, but I don’t expect that to happen until I’ve got 1000 or more people invited in, so this first group should get in pretty quickly.  I will look at the bottlenecks and optimize the server, and that might get me to 2000 invited, or around 200 simultaneous players.2  By that point I want to have my server infrastructure scalable and running in the cloud, but that’s going to be a significant chunk of programming, so there might be a delay at this point in getting more invites out.

Finally, once the backend is scalable, I can start inviting larger groups of people in at the same time.  When I can send out 2000 invitations and everything runs smoothly as those people sign up, download the game, and start playing, it will be time to invite the rest of the beta list in, shut down the list, and let people join the beta directly without an invite. I think I’ll probably have a couple week period where I invite all the confirmed signed up people in, but don’t allow new signups, as a way of saying Thank You for signing up.

But, that’s all months and months away.  The Near-Term section above should be reasonably accurate, but this part is mostly me making stuff up right now.

1. Give me my $14.26, PayPal!  And yes, I am going to provide other payment provider options soon. [↩]
2. That is, if 10% is a reasonable estimate for peak concurrent users relative to registered users, which it seems to be. [↩]

A 11:47:53 pm, February 15th, 2012, the first SpyParty Early-Access Beta invitation was sent across the internet.  I was typing into a laptop at my dining room table in Oakland, California.  I typed the necessary commands into an ssh window, which was connected to a server in Texas.  A perl script generated an invitation email which landed in Sunnyvale, Calfornia, in search of Jonathan Blow.  Finally, Jonathan checked his email by clicking a button on his laptop, which was sitting on my dining room table, right next to mine.

He registered, and my PayPal account went from $0.00 to $14.26 and he was able to login to the private beta website!

Success!

And then the bug reports started.  The login page didn’t restrict the case of the username.  There was no link to the beta homepage page in the email.  I forgot to upload a working build of the game before sending the invite.  The server’s auto-detection of new builds was broken.  And then, to cap it all off, PayPal decided to reverse the charge and put the payment on hold.  I didn’t even have my $14.26 anymore!

Oh well, that’s what shipping software is all about, and I expected at least this many problems.  The important thing is that I can now say The SpyParty Beta Has Started!

I live tweeted the whole thing from the @spyparty account:

Okay, I am switching my php from the paypal sandbox to the live paypal servers…
Feb 15, 10:58pm

PayPal balance: $0.00 USD
Feb 15, 11:03pm

This first invite is going to @Jonathan_Blow, who (hopefully) won’t mind if my paypal script accidentally charges him $15000 instead of $15.
Feb 15, 11:22pm

I said I’d try to start invites before 11111 beta signups, and I’m at 11101, so I’m going to make it. Also, @spyparty is at 3995 followers!
Feb 15, 11:29pm

I am sending the invite to @Jonathan_Blow…hopefully it will work…
Feb 15, 11:47pm

Invite away…he is sitting right next to me, so hopefully the email can make it that far…
Feb 15, 11:48pm

He’s in. Forgot to upload the latest build, oops. Plus a login bug…
Feb 15, 11:53pm

PayPal balance: $14.26 USD
Feb 15, 11:56pm

And, PayPal reversed the charges and has them under investigation. LOL Give me my $14.26 back!
Feb 16, 12:07am

Status: Being Reviewed By PayPal … they are really earning their $0.74 on this one…
Feb 16, 12:25am

Why the hell isn’t the lobbyserver finding the updated build on s3?!? That code used to work…sigh. Current new bug count: 8
Feb 16, 12:38am

I guess I need to handle reversed charges with a temp ban…currently @Jonathan_Blow has a valid beta account and I don’t have his $14.26…
Feb 16, 12:39am

I can understand Minecraft's $600k setting off some flags, but $15?!

Sorry I haven’t updated in a while, but the good news is I’ve been totally cranking on the Early-Access Beta!  I really appreciate everybody’s patience, especially since I’m so behind schedule on inviting people,1 but it’s getting close, it really is!

I have a zillion blog posts queued up in draft or idea form, but I don’t want to take any time away from working on the beta to finish any of them, so I came up with this (probably ill-advised) idea:  I will post the remaining items left to do before I can invite the first players to join the beta.  Then, instead of taking time to write new posts, I’ll just edit this post and strikethrough the items as I do them.  Sadly, as with most software development task lists, the items below are at vastly different time and difficulty scales, so they won’t all get checked off at the same rate, and they have an annoying tendency to spawn unforseen subtasks as one digs into them, so the list will grow occasionally too.  But, at least this way you can see my progress towards the finish line at a finer granularity than my intermittent tweets.

Even though I’m stupendously late, I’m pretty excited about SpyParty’s development right now, because almost all of the big tasks have been finished.  Stuff like auto-updating, crash dump reporting, the installer, and the security system are done, or at least beta quality.  Most of the rest of the stuff to do is the myriad loose ends that come along with getting anything ready for public consumption.  The two big remaining things are the invite-a-friend feature that I originally wasn’t planning to do for launch, but after your comments here on the blog, on twitter, and on facebook, I decided I simply had to add it, and the text chat feature, which is sort of stubbed in but needs to be improved, since communication is going to be an important part of the experience.  It’s going to be really rough to start with, but hopefully it will be good enough to get things started.

I’ve broken up the todo list into three sections:  the Website, which means anything you talk to through your browser, including payments and the private beta foums and homepage; the Server, which is the lobby, player and game database, and that sort of thing; and the Game, which is the thing that runs on your machine.

Without further ado, here is the list.  You can ask questions in the comments, but do keep in mind that time I spend explaining these things is time I’m not spending checking them off!  Also, these are just the tasks to get the first beta invites out, I have infinitely longer and less well defined task lists for later in the beta and beyond.  But let’s not think about those right now…

Website

• invite friend to join beta, one invite per person
• mail and homepage needs notes about whether you can talk about game during beta
• please do, indie games need your help to spread the word
• if you have criticisms, I just hope you’ll also tell me (post in forums or email) so I can fix them
• invites
• invite.pl
• need to update the intInviteStatus, uncomment the execute line
• do initial invites with email
• change invite.pl to not need any email, and just take a count and invite the next group
• cheating
• many ways right now, please just don’t, will make me deal with cheaters now instead of making game
• spectating
• listening for control clicks?
• even multiple people watching sniper side monitor
• switch to CentOS 6
• get backups in order, good test for this!
• audit all the code
• add user to announce mailing list
• block test ipns from doing real work
• leaderboard
• test newline in display name? need to escape it
• style cosign pages
• login, login error
• logout
• change password
• only want while logged in for now? yes
• style
• test cosign logout
• definitely does not work in firefox the first time? wtf? look at the cookies?
• beta homepage
• style
• stats
• link change password
• style rss
• display nicer date
• forums
• edit account settings – disable change password & email address
• update phpbb to 3.0.9
• welcome message, introduction to beta
• stats on profile page
• phpbb how to report bugs
• phpbb faq – update faq.php to add styles/<active>/faq/more_faq.php to the array
• error_log in all php code
• paypal note about anonymous payments, credit cards, etc.
• retry-registration needs to display a page before going to paypal
• mail with details of which cancellation on paypal error?
• test paypal flow with non-sandbox
• enable IPN, redirect back to page, etc.
• what does cc bill show as outside of sandbox?
• start downloading logs
• forward mail? probably better than running pop
• or upload to s3?

Server

• when shutting down, don’t allow new connections, just reply with failure message
• why doesn’t the stunserver work when resolving addresses?
• test relay server, at OAK?
• kerberize stun and relay servers
• initially just show spy/sniper games played, so user name is displayed as checker (45/67) everywhere
• upgrade krb to 1.9.2
• client
• server
• write lobbyclient stress tester app that spews data at server
• log in first sometimes
• use known packet types
• make this the lobbyclient app, and have it be console control as well
• send message to all connections
• don’t make this too general for now, fine to recompile for different stress tests
• need to fix pointer output in log for connect/disconnect
• send client state info to server for debugging?
• yes, at game menu, playing, etc.
• server database
• async_db clear semantics for db return codes, reg/unreg
• store game/match data to db
• users records
• ranking info
• date stamp for login
• last login, logout
• $inc login count
• $inc failed count?
• cumulative time logged in! login/match/game times & counts
• add usernames to journal filenames (urlescape!)
• checker/test for testing
• turn verifies in the lobby ctor into logs so can debug
• clients only allowed one game journal per game id
• don’t need the random number anymore, given session id in filename
• overwrite file on multiple submissions
• async_krb5 and async_db error log…have a varargs Log and pass it, also for game
• finish updating to new protocol
• client<->server
• client<->client
• hmm, track memory usage and make sure we’re not leaking
• output to log file occasionally?
• info mallinfo – dev/test/mallinfo.cpp will work on linux
• GetProcessMemoryInfo
• async_krb5
• clean up code
• disable all the built-in accounts? or leave them randkey?

Game

• “game” -> “match” in ui
• handle errors in client<->client authn
• switch to — for command line parms?
• windowed maximized borderless
• network time sync?
• it can lag by 5 seconds, which plays fine but seems terrible, better to run Zach’s simple net sync code
• merge reliable packets
• ooi in reliable packets
• test motion in joy ps3 driver
• make a list grid mode for the chooser, or just obey tabs?
• then display other players in lobby and their ranks
• want to display release notes during autoupdate?
• have lobbyserver fetch them?
• displayed while downloading, option in main menu
• send gl versions and cpuid during login?
• need to version check on clients
• journal match id on client and server
• net pause – pause is a nice thing, no?
• test keys disable
• figure out what stuff to collect in the journals, careful of machine name, etc.
• no marketing, or sharing (might publish anonymous stats)
• add the appropriate command line parms to the –help! window stuff, etc
• fix chat
• /me /e[mote] str – print “username str”
• /played /age – print playtime stats
• /? /h[elp] – print commands
• /w[hisper] /t[ell] /m[sg] <username> str – private message to username
• /r[eply] str – reply to last person who whispered me
• /retell str – send to last person I whispered
• /away /afk /dnd [str] – displays str to anyone who whispers me, /away to cancel
• /time – lobbyserver time and local time
• the chat text is totally unreadable with those colors!
• /fps – basic net and video info
  
• completion, cut & paste
• support esdf, y invert, config screen
• if not fully remappable keys, mention ablegamers list in readme and say will comply soon
• logging
• async_krb5 and async_db error log…have a varargs Log and pass it, also for game
• log printf output to file instead of console
• and gzip and send to server?
• chat messages to console?
• convert to use log function instead of stderr
• disable logging by default
• or, log to file always for now so crash dump can upload it instead of asserts log
• log to pipe and store in spypartyhelper’s memory instead of file?
• update docs/README.txt
• put the character name in highlight text? try it.

2011-11-14 18:02:07 – Checked one off already, although I must admit I did the minimum necessary to get it working.

2011-11-17 18:25:17 – Grindy bug fixy day, but finally got the async server db thread semantics clear.

2011-11-30 13:09:31 – Okay, I think the server database stuff is all done, games are saved, results are saved, the player records are updated, etc.

2011-12-06 13:59:23 – The ugliest lobby ever?  Maybe.

A team of UI designers worked on this for 5 years.

2012-01-05 11:18:54 – I am calling it for the chat system.  I didn’t get to /ignore and /stats and whatnot, but /ignore is not relevant until there are annoying people playing, which I’m sure won’t be the case for a while, and /stats are all displayed in the lobby chooser, so it’s redundant for now.

2012-01-25 16:41:04 – More red strikethroughs!  Getting really close!

2012-01-26 15:08:46 – Getting the PS3 DualShock controller to work on Windows is so much more work than it should be.  Start by installing the totally unauthorized MotionInJoy drivers! Then mess around with configurations.  Don’t even think about getting it working over BlueTooth unless you have certain USB dongles.  What year is this?  Oh well, it works:

Well, at least now I can say I support 360, PS3, and mouse+keyboard.

2012-02-03 14:24:10 – Finally grinding through the website tasks, which should go pretty quick…*cough*.  Here is a screenshot of the saddest page on the blog:

A page you don't want to see in your browser.

2012-02-07 14:27:18 – It’s the worlds ugliest Personalized Early-Access Beta Homepage™ ever!

Well, at least it goes with the ugly lobby.

2012-02-13 14:37:57 – The loneliest leaderboard…

These guys have beta access already and they're not playing more than one game each?! What's wrong with them?

1. Let’s just agree not to mention my original June 16th estimate, okay? [↩]

As you may remember, I planned to try to emphasize SpyParty‘s gameplay depth at PAX West 2011 by setting up the booth with three playtest stations (six computers total), each with a different rule for what kind of players could sit down and play.  My booth was a tiny 10×10, so I knew it was going to be jam packed…here’s the diagram I used to plan out all the placements and cables:

Welcome to professional tradeshow booth planning!

So, how did the experiment in depth-based-playtesting-at-a-tradeshow go?

I think it went pretty well, and as you can see from the pictures below, it seems like the attendees agreed.  We had people playing at the “unlimited playtest station” for well over an hour without getting up, and a small community of regulars—including a bunch from last year, see if you can spot them—camping the station and helping others learn the advanced game.

I think it really helped people get a feel for the deeper game that is hard to experience in that kind of environment, and I think I’ll do it again, maybe at PAX East.  The one problem I noticed has an online equivalent:  matchmaking is hard!  Often a single player would come up and want to play, but there wouldn’t be anybody else at their skill level available, so they had to wait a bit for another single to show up.  This problem lessened over the course of PAX as more players became experienced and started hanging around the unlimited station, but I think next time I’ll have announced times when players can come and make a date with somebody for later in the day.  So, maybe at 11am and 3pm unlimited players can come and make arrangements with other unlimited players, or something like that.  Manual matchmaking, indie style!

Here were the three signs I made for the different stations:

One other flaw with the depth based testing was I didn’t have any way for players to log in and use a persistent username, so I couldn’t track metrics for each player.  I’ll post the metrics from all the PAX games soon, but they’re just going to be a jumble of every game played, which is fine when everybody just gets to play two matches, but I really wanted to track longer term data on the unlimited station.  I thought about a lot of different ways to do this, but they were all clumsy and error prone.  Then Matt Gilgenbach from 24 Caret Games stopped by to chat and told me about what they did for their Retro/Grade leader boards:  card readers!  Hardcore players were given a magnetic strip card like a credit card, and they could swipe this to log in before they played.  So quick and easy and tradeshow-floor-compatible!  Matt said they got the card readers for cheap on ebay.  I’m totally stealing this idea for next time.

Before I go any farther, I have to thank all the folks who helped me with the booth this year, they were amazing!  First off, Sean Willoughby, the PAX Enforcer assigned to the SpyParty booth, went way above and beyond the call of duty in helping me set up.  You can see him in the pics below helping me make signs!  He was an incredible help.  GDC volunteer Roger Hanna returned and did an awesome job of saving my bacon yet again.  He was joined by some game industry friends from the Spore days, Thomas Vu, Steven Lim, and their friend Tony Lee (who I’d never even met before he showed up out of the blue with Thomas and Slim and then proceeded to rock booth duty like a seasoned pro!) .  Paul Sottosanti, who you may recognize as one of my elite playtesters, and who has recently gone indie himself, also helped a bunch, as did Marc ten Bosch, creator of Miegakure.

Speaking of other indie games and helping each other out, I had planned on having Carbon Games show off their new game AirMech, and Marc show off Miegakure, but the Carbon guys ended up getting a fulltime booth at the last minute through some friends, so we had Miegakure set up both weekend mornings.  You can see it mixed in with the pictures below.  People had a great time playing it for an hour, and I let some real hardcores play until well past lunch since they were having such a great time!  I really want to thank the Carbon folks, and especially James Green and Ken Klopp, for being so cool…they lent me a big screen tv and stand even though they ended up not showing at my booth.  I owe them bigtime.  Indies rock!

Okay, on to the pictures, and thanks a ton to all the people who came and played!