Game telemetry is being used both during development and post release. One of the most exciting applications of this work is the use of game telemetry to support the game design process. Game telemetry analysis can help a designer answer the following questions:
How do players interact with the game?
Which features, modes, and content are players experiencing?
Why do players quit playing the game?
Using game telemetry provides an alternative to conventional playtesting. Rather than bringing in individual participants for playtesting, collecting telemetry enables your entire user base to become playtesters. The advantage of this approach is that data can be collected at an enormous scale, from players in their natural environment playing with friends, across long periods of time. The main disadvantage of this approach is that telemetry provides only quantitative data about players. It enables a designer to identify what players are doing in the game, but not understand why players exhibit specific behavior.
I've put together a project for interacting with the BWAPI library via JNI. Using JNI addresses several performance issues and enables additional BWAPI functions to be called from Java. Here's a video of the system in action:
A lot of interesting bots were submitted to the StarCraft AI Competition. However, even the best could not beat a skilled human player.
But what about amateur players? Kotaku claimed that most casual StarCraft players would be unable to defeat the bots. The goal of this post is to test that claim. To do so, I played against the winner of each of the four tournaments. My crowning achievement in StarCraft is once reaching D+ on ICCup, so my skill is much less than that of DoGo, who previously played against the bots.
Tournament 1: Micromanagement The first tournament focuses on micromanagement in flat-terrain environments. The winner was FreScBot, which uses a multi-agent, finite-state-machine approach. The results are shown in the video below:
Winner: Bot While I was able to win 2 of the games, the bot's micro was too beastly for my skill.
Tournament 2: Small-Scale Combat The second tournament builds on the first tournament by adding interesting terrain to the maps. The winner was again FreScBot and my encounter versus the bot is shown below:
Results: Match 1 - Dragoons: 1-0 man advantage Match 2 - Marine/Medic: 1-0 man advantage Match 3 - Dragoons: 1-0 man advantage
Winner: Man Adding complex terrain into the mix really offset the skill of the bot. While I was unable to defeat the bot's dragoons micro in the first tournament, I was able to use the topology of the map to my advantage in the second tournament.
Tournament 3: Tech-limited Game The tech-limited game tournament evaluates bots in a simplified StarCraft environment. The winner was Mimic Bot, which attempts to mirror your build while also stealing your gas and expanding as necessary:
Results: 2-1 man advantage
Winner: Man This match was surprisingly difficult. The first game I went goons and was simply out microed by the bot. In the second game, I focused on zealots and used the lack of collecting gas to gain a slight advantage in zealot count. In the third game, I had some fun with the bot, and somehow pulled out a win. While I was able to win a majority of the matches, the games were very entertaining and I enjoyed the challenge provided by the bot.
Tournament 4: Full Gameplay The complete StarCraft tournament evaluates bots in the full-blown StarCraft game. The winner was Berkeley's Overmind, a bot too complicated to try to explain:
Results: Terran: 1-0 man advantage Protoss: 1-0 man advantage Zerg: 1-0 man advantage
Winner: Man In the first game, I went for a fantasy style build with a vulture drop. I had several issues with my build, such as a failed wall-off, but I was still able to pull out a victory, eventually. Once the mutalisk get out, it is relentless pounding on your base, so it took me awhile to deal with the opponent's expansions. In game two, the bot simply could not handle the zealot timing. Part of the issue is that the bot went for a 12-pool instead of a 9-pool, even though it scouts quite early in the game. And in the final game, the bot could not handle my zergling speed while trying to tech too quickly.
Conclusion: Despite advances in AI, man still reigns supreme, even at an amateur level. There are a few interesting takeaways from this experiment:
These bots are really fun to play against! Overmind and MimicBot challenged me in ways I never imagined, and it really required adapting my strategy to overcome the opponent.
Building a zerg bot is hard! Berkeley was pretty ballsy in deciding to go with Zerg and I commend them for their effort.
I'd like to encourage people to play against the bots, but the setup is a little challenging. All bots are now available on the tournament website.
I've recorded a few of the more interesting games from the StarCraft AI Competition. All replays from the competition are available and I encourage people to post additional videos and commentaries: http://eis.ucsc.edu/StarCraftAICompetition#Results
Here's one of the matches from the finals between Krasi0 and Overmind (Replay):
This video shows the initial match between =DoGo= and Krasi0, with DoGo playing as Protoss (Replay):
One of the highlights of the competition, Skynet versus Krasi0. This video shows the match-up on Heartbreak Ridge (Replay):
The semi-finals between Chronos and Overmind (Replay):
The AIIDE 2010 StarCraft AI Competition has come to a close. The challenge given to competitors was to build the best performing bot for an immensely popular, commercial game. The competition consisted of four tournaments of varying complexity. This was the first year the competition was held and it turned out to be a success. Even though no prizes were offered, over twenty-eight teams participated in the competition. My presentation on the competition provides an overview of the participants and results.
The showcase game of the competition was a bot versus human match. In the exhibition match, =DoGo=, a World Cyber Games 2001 competitor played against the top ranking bot of the competition. The result was an exciting man versus machine match highlighting the state-of-the-art in RTS AI.
While the expert player was capable of defeating the top performing bots in the competition, the results are quite encouraging.
Tournament 1: Micromanagement
The first tournament evaluated bots in unit micromanagement scenarios.The winner of this tournament was FreSCBot, which uses multi-agent finite-state machines. The runner-up was Sherbrooke, which also uses state machines. Complete tournament details are available here and the full results are posted here. The video below shows the final match between FreSCBot (red) and Sherbrooke (blue).
Tournament 2: Small-Scale Combat The second tournament evaluated bots in small-scale combat scenarios. The results of this competition mirrored the results of the first tournament: FreSCBot won and Sherbrooke was runner-up. Complete tournament details are available here and the full results are posted here. The video below shows the final match between FreSCBot (purple) and Sherbrooke (orange).
Tournament 3: Tech-Limited Game The third tournament tested bots in a tech-limited StarCraft environment which requires reasoning at strategic and tactical levels, but omits much of the complexity of the full version of StarCraft. The winner of this tournament was Mimic Bot, which attempted to mirror the opponent's strategy while also performing a gas steal and applying fallback strategies. The runner up was Botnik, which executes a Zealot rush strategy. Complete tournament details are available here and the full results are posted here. The video below shows the final match between MimicBot (red) andBotnik (teal).
Tournament 4: Full Gameplay The final tournament was a best-of-five, double-elimination tournament simulating a professional gaming competition. The winner of this tournament was Overmind, a Zerg bot that effectively scouted its opponents, interrupted their economy, and performed Mutalisk harassment until victory was ensured. The runner-up was Krasi0, a Terran bot with excellent defense and pushing capabilities. Complete tournament details are available here and the full results are posted here. The video below shows an exert from one of the games between Overmind (yellow) and Krasi0 (brown).
Highlight #1: Krasi0 versus Skynet During this minute of action between bots Krasi0 and Skynet, it was difficult to distinguish whether the game was being played by skilled human players or the StarCraft equivalent of Deep Blue.
Highlight #2: Berkeley's Mutalisk Evolution Contemporary StarCraft wisdom tells us that the best way to use mutalisks is to clump them. In human versus human battles, this makes it difficult to single out the weaker mutalisks, because the units are stacked on top of each other. However, UC Berkeley's team identified a flaw in this tactic; it reduces the damage output of each individual mutalisk, because not all mutalisks will fire when using this tactic. Instead, they employed a model in which mutalisk are always moving, maximizing damage output while simultaneously maximizing movement.
Conclusion The inaugural StarCraft AI Competition was a huge success! I would like to thank Blizzard, BWAPI, AAAI, and all of the participants in making this event a reality. The next competition will be hosted by the University of Alberta at AIIDE 2011.
I recently ported Infinite Mario to the Android platform in order to evaluate the use of procedural content generation in mobile devices. My goal was to develop a casual platform game, where players can jump right into the action without worrying about making it to the next save point. Procedural content generation is used to provide the player with short, unique gameplay sessions. The player's goal in Mario Forever is simply to complete a level. The major challenge in this game is learning to defeat levels you have never encountered before.
Mario Forever
The game is freely available and I highly encourage trying it out, but I have not posted it to the Android market due to Intellectual Property issues. The game can be downloaded for Android 2.1 at:http://users.soe.ucsc.edu/~bweber/MarioForever.apk
Installation is a little tricky, because the game is not on the market. To install the application, point your phone's browser to the link above and download the apk file. In order for the application to be installed, the following setting must be enabled on the phone: Settings -> Applications -> Unknown Sources.
The controls are a little difficult, but two options are provided. If you have the Motorola Droid like me, you can use the directional pad to walk and run (with the center button) and use S to jump. For other models, the touch screen can be used to control Mario's movement: hold your thumb on the left side of the screen to walk or run left and hold your other thumb on the right side of the screen to walk or run right. To jump, gesture your thumb upwards for the duration of the jump.
The game can also be ran without an Android phone using Java web start here.
The sure way to distinguish the computer's messages from the human's is to recognize that the computer has a rather sentimental attachment to the apostrophe and the comma.
This statement reflects Carr's view that the internet is making us stupid. Carr recently appeared on The Colbert Report to promote his new book "The Shallows: What the internet is doing to our brains". He claims that as the internet helps facilitate the ability to multitask, we lose our ability to think deeply and perform reflection and introspection. Carr believes that if EISBot passes the Turing Test in StarCraft, "it won't be because computers have become smarter; it will be because humans have become dumber".
The game is freely available and I highly encourage trying it out, but I have not posted it to the Android market due to Intellectual Property issues. The game can be downloaded for Android 2.1 at:
