For the past few months, I’ve been working on a tool to compile .NET applications into JavaScript. While there’s a lot of work involved, a large chunk of that is caused by static language idioms that cannot easily be expressed in JavaScript. Even when coding by hand, some of these patterns are difficult to turn into JS that performs well, works correctly, and is easy to understand. In this post, I’ll begin describing some of the challenges involved.

Value Type Semantics

Value type semantics are a common feature in static languages. Most static languages allow you to define custom types that inherit the semantics of the language’s built in primitive value types, so that your struct (or whatever your language happens to call it) behaves much like a single int or float would when being manipulated. This feature makes it possible for custom numeric types (like a fixed-point value, for example) or even composite types like vectors and matrices to follow the same rules.

At first glance, the lack of this feature might not seem significant – especially if you’re used to object oriented programming, it would seem like translating code dependent on these semantics is not an intractable problem. While it’s certainly not impossible, it has a number of downsides that surprised me, especially once I started profiling real-world code in modern JavaScript implementations:

Algorithms designed for value types are slower without them

Those of you who’ve worked with Java are probably familiar with this particular problem already: If you take an algorithm designed around the presence of value types and port it to a language without them, while you can easily preserve behavior by inserting copies in the correct places, you may find that your performance has gone through the floor. The reason for this is simple: In many runtime environments, the cost of an instance of a reference type is significantly larger than the cost of an instance of a value type.

The reasons for this cost tend to vary, but one common trait is that these algorithms produce tremendous amounts of garbage when run in a garbage-collected environment. Some of the more sophisticated garbage collectors provide mechanisms to improve performance for this kind of code, like generational collection and escape analysis, but the sad fact is that even the most sophisticated collectors available still suffer from visible pauses when the collector runs. These pauses not only impair the overall performance of your application, but in games, they cause painful framerate hitching and can even impair the playability of your title.

Another common source of higher costs for instances of reference types is inheritance. In particular, virtual inheritance. Virtual inheritance as implemented in most languages requires every object instance to have a few bytes devoted to identifying its type, via vtable pointers or the like. In some cases compilers can optimize this overhead out, but the overhead is usually there regardless of whether you use it, and in some languages, the default is to make everything virtual – so suddenly even your Vector3 object has a vtable pointer.

A third source of overhead is indirection – reference types almost always live in the application heap, which means they are allocated in a chunk of space dedicated to that particular instance. Even if you allocate an array of your Vector3s, reference type semantics mean that it is probably necessary for the heap to contain an allocation for each one, so that if half of them are destroyed, those resources can be freed and reused by some other data types. Not only does this mean that the simple act of creating that array has become more expensive, but it means that an array of values is no longer guaranteed to be contiguous in memory. Sure, your Vector3[] is contiguous, but all it contains is object references. Hm, what’s that cache locality thing people were talking about again?

(Somewhat obvious pre-emptive nitpick: This last bit here doesn’t apply to C++. But you’re using C++, so you know this already, and you should probably go make sure MSVC hasn’t run out of memory while compiling that boost template instantiation. You know the one.)

Algorithms based on value types are fragile using reference types

Despite the above problems, Java developers are able to produce quality software (depending on who you ask, anyway). The same is true for JavaScript: These algorithms can, with effort, be made to work, and even work well. Unfortunately, there is a significant cost involved. You have to rework your code in order to reduce the amount of garbage it creates, and avoid patterns that naturally require the creation of garbage. For example, in a language like C++, your game probably relies on a math library, featuring structured value types with names like Vector3 and Matrix. If you’re one of those people who likes operator overloading, you’ve probably overloaded various operators for those structures, so that you can write things like this:

The above code is simple and self-explanatory to someone familiar with the basics of algebra (and those of you who abstain from operator overloading have probably written similar code using functions instead of operators). As mentioned above, the performance of such algorithms will suffer badly in a garbage collected environment, but that’s not the worst part. First of all, to correctly translate these algorithms, you must carefully locate all the implied copies contained within the code. As written, the above algorithm is correct even without value types. Let’s convert it to valid JavaScript:

Seems fine, right? But, the keen-eyed reader realizes that a copy is implied in the Add and Multiply operations, so for them to be correct, they must create and return new Vector3 instances. Let’s fix it so it doesn’t produce garbage.

Hm, we need a copy here. That sucks. We’ll just store an extra vector in the object so we don’t need to create garbage. All good, right?

A little ugly, perhaps, but not bad, right? The clarity suffers a little, but we could address that by naming variables more carefully, etc. Not horrible. But wait a second. We pulled the values position and velocity out of an object. If we just modified position and velocity, does that mean we’ve modified the state of the object it came from? Yes, yes it does. Well, we just need to emulate the semantics of value types, right?

Okay, there. It’s correct again now. But wait, garbage collection! Uh, crap. Okay, let’s remove the copies we don’t actually need… we’ll assume that the value of NextPosition isn’t being used, so we don’t need to copy it. And we’re not modifying Position so we don’t need that copy either…

Okay, life is good again. But a week later, you’re reading over commits (sneaky interns keep adding features behind your back, the nerve) and you see something that makes you spit up your coffee:

Luckily, as noted in the comment, the game isn’t using ZoomFactor yet, so it’s always 1. Nothing horrible has happened yet. But now you’re worried, and you’re starting to sweat a little: What other bugs has JDoe introduced that are just waiting to break your code once it hits production? Didn’t this guy develop payment processing systems at a bank? Are your savings in danger? Oh man, what if a bug in code he wrote allows Albanian hackers to steal that money you’ve been saving up for your little startup venture? You might be stuck working here for another 10 years!

Quality algorithms designed without value types are hard to find

So, let’s say you’ve calmed down a little bit after that panic attack earlier. JDoe’s not a bad guy, he’s just new at this. You’ll go through and review his code with him, make sure he understands what he did wrong, and then you’ll add another 5 pages to the company’s ever-growing coding standards document, right after you find a new bank. Life moves on, right?

While you’re doing the code review, though, you run across something that makes you pause. You notice that JDoe modified some code that was written by Chris Beardy, a wizardly programmer type who had already been working here for 8 years when you started out. Nobody really knows what happened to Beardy; one day he just stopped showing up. Until now, his code hadn’t been a problem, since most of it worked great, and the bits that stopped working you just threw out and rewrote (with variable names longer than two characters).

JDoe has done his best to modify the code, and while it seems to work, you realize you don’t know whether it’s correct. You could assume that JDoe messed up, but what if you’re wrong? How can you be sure the code is correct? You ask around the office to see if anyone has Beardy’s phone number, but nobody is even certain if he owned a phone. You try to contact his next of kin but all of their phone numbers bear international dialing codes and it’s 3am in Singapore. This is a problem.

After consulting with a respected colleague, you hit upon a solution: It may be true that neither of you know whether the code is correct, but you know what it was supposed to do. It turns out it’s based on some old academic paper, and the paper’s been cited a bunch of times, and other games use this same algorithm. You figure you can just read a couple of those papers and implement the algorithm yourself – hell, maybe you can find a good implementation out there on the internet that’s fit to use!

Soon after you begin reading the papers, panic begins to set in once again. You’re starting to wonder if that literature major was such a good idea, and trying to remember the stuff you learned in the few math classes you did take. You’re not entirely convinced that these symbols used for variable names exist in any alphabet, and not even your coworkers know what it means when the author writes them upside down. You abandon your paper-reading expedition, reminding yourself that code reuse is better anyway, and as a good engineer you should avoid Not Invented Here syndrome by adopting someone else’s battle-tested implementation…

Now the horror begins. You narrow down the few available implementations for your programming language, rule out the ones that depend on libraries you can’t possibly bundle in, and check with Legal to figure out which specific open-source licenses don’t cause them to flee in terror. You scan over some documentation and things don’t seem too bad. This one is enterprise ready, and you can get a support contract! This one is written using Inversion of Control and Dependency Injection, whatever those are – you think you might have heard someone say something good about them once. You’re feeling pretty okay about things until you pop open the source code. Why are half of the comments in Japanese? This class name is 50 characters long and you’re pretty sure all it’s responsible for is adding numbers. The constructor for one of the key objects creates no less than 8 abstract factories, and two of those only exist to create other abstract factories…

While you start updating your resume, you start to wonder whether you should consider another line of work…

(Now, let’s be completely fair: This one applies to pretty much any programming language, value types or not. But I’ve never been the type of person to let reality interrupt a good rant, so the disclaimer sits here at the end.)

In Conclusion

Now, if you’ve read all this you might be thinking, ‘hey, I’m really offended by that comment about Albanians. What’s wrong with this guy?’. And you’re right, that was kind of offensive. I’m sorry.

You also might be thinking: ‘Wow, this is a really negative perspective on things. Can life really be that bad writing JavaScript? I wrote a webpage once, and it was pretty okay. Google writes whole applications in it and they seem to be making tons of money’. You’re also right! Maybe not as right as that other guy, but hey. I said I’m sorry.

These problems are not insurmountable. In my particular case, since I’m writing a compiler, instead of porting by hand or writing JS from scratch, I can solve these problems using verifiable code, instead of trying to get it right in my head. Despite this, fixing some of these problems involves writing some pretty hairy code – and if you’re like me and your Computer Science background is a little – let’s be generous here and call it ‘weak’ – you may find things like performing escape analysis on functions containing goto statements a tiny bit daunting.

When I finally got my translator working on some real-world game demos after a month or two of hacking, I was really excited; the framerate was bad, but I hadn’t done any optimization, so that was to be expected. When I finally ran a demo through a profiler, though, my heart sank a little: Wait, I’m spending 35% of my CPU time in the garbage collector? And another 20% running the constructor for Vector3? I realized that while I had anticipated the challenges in translating working code into another language, I had not anticipated the challenges involved in translating performant code from one language into performant code in another language. So, I hope that these few examples I’ve provided might give you a little insight.

In many graphical applications, a common pattern is the need to remember user preferences or inputs. Modern languages provide useful tools that can make this task simpler – for example, both Java and C# provide ways to automatically serialize an entire data structure into XML, which eliminates the need to manually read/write your preference values. Unfortunately, even with the aid of these features, you still find yourself with a lot of duplication – even if the runtime can automatically serialize your data structure, you still have to fill the structure out with your preferences, or read them back in manually. I’m going to demonstrate how you can use a little-known aspect of the new LINQ feature in C# 3.5 to eliminate duplication with the aid of the compiler.

As an example, we’ll be using the simple application shown above. It’s a Windows Forms dialog box, with a few different controls, set up to remember user inputs between sessions. You can download the source code for this application below:

Preferences Example (v1)

If you run the application, input some values, and click OK, you’ll find that it stores your inputs in a JSON file in the %AppData%\PreferencesExample folder. If you run it again, you’ll see that it pulls your inputs back out of the JSON file to populate the UI.

Let’s take a look at the relevant parts of the source code:

public void LoadPreferences () {
  if (!File.Exists(GetPrefsPath()))
    return;

  var serializer = new JavaScriptSerializer();
  var preferences = serializer.Deserialize<Dictionary<string, object>>(File.ReadAllText(GetPrefsPath()));

  checkBox1.Checked = (bool)preferences["checkBox1"];
  radioButton1.Checked = (bool)preferences["radioButton1"];
  radioButton2.Checked = (bool)preferences["radioButton2"];

  listBox1.SelectedIndices.Clear();
  var indices = preferences["listBox1"] as ArrayList;
  foreach (var index in indices)
    listBox1.SelectedIndices.Add((int)index);

  textBox1.Text = (string)preferences["textBox1"];
  dateTimePicker1.Value = (DateTime)preferences["dateTimePicker1"];
}

public void SavePreferences () {
  var preferences = new Dictionary<string, object>();

  preferences["checkBox1"] = checkBox1.Checked;
  preferences["radioButton1"] = radioButton1.Checked;
  preferences["radioButton2"] = radioButton2.Checked;
  preferences["listBox1"] = listBox1.SelectedIndices.Cast<int>().ToArray();
  preferences["textBox1"] = textBox1.Text;
  preferences["dateTimePicker1"] = dateTimePicker1.Value;

  if (!Directory.Exists(Path.GetDirectoryName(GetPrefsPath())))
    Directory.CreateDirectory(Path.GetDirectoryName(GetPrefsPath()));

  var serializer = new JavaScriptSerializer();
  File.WriteAllText(GetPrefsPath(), serializer.Serialize(preferences));
}

This application is relatively small – in particular, the JavaScriptSerializer handles a lot of the work involved in storing preferences for us. But regardless, we find ourselves having to list out all the control names multiple times, and some of those references to the controls are strings. Since they’re strings, that means that automatic refactoring tools, like the Rename tool in Visual Studio, won’t find them. As a result, this sort of code becomes difficult to maintain, especially in a larger codebase – changes are extremely error-prone and require careful, manual work.

What we want is a way to eliminate this duplication – a way to pull the list of controls and values out and store them in one place, so that we don’t have to specify them multiple times. Unfortunately, there’s no obvious way to do this in either Java or C#. While you can add the controls to a list quite easily, and walk over the list to get control names – eliminating part of the duplication – there’s no clean way to specify the particular fields/properties you wish to store.

Those of you familiar with reflection techniques may realize that you can eliminate the duplication by using reflection. By specifying a control and the name of the field we care about, we can use reflection to find that field and fetch its value. This technique will work, and other than the performance overhead of reflection, it will work well enough to ship in an application. However, it still suffers from the downside of breaking automated refactoring tools, and it’s somewhat error-prone due to the complexity involved in finding fields/properties via reflection.

One of the new features in C# 3.5 is support for ‘Expression Trees‘. Expression trees allow you to convert a C# expression into a tree of objects that describe the expression and allow you to evaluate it. This feature is used to support LINQ – for example, when you write ‘where i < 5′ in a LINQ query, the ‘i < 5′ portion of the query is turned into an expression tree, that LINQ can then use to run the query – for example, by turning it into SQL. As it happens, we can use this feature to solve our problem!

In C# 3.5, virtually any expression we can write can be converted into a delegate using lambda syntax, like so:

Func<int, int> func = (x) => x * 5;

And, as part of the Expression Tree feature, we can convert that lambda into an expression tree, at compile time:

Expression<Func<int, int>> expr = (x) => x * 5;

The compiler provides us all the information we need to evaluate an expression within the expression tree – if the expression references a field, the expression tree will contain the object containing the field, along with the FieldInfo object representing the field. Using this knowledge, we can eliminate the duplication from our sample application: By using a single expression to represent each preference we want to store, we can list all of our preferences in a single place in our application, and use that list of expressions to both load and save preference values. Best of all, because the compiler gives us full information on the expressions, we know the exact data types involved, and we can find out the name of the field being stored, and even the name of the object containing the field – eliminating the need to name each preference manually.

Using this knowledge, we can write a simple helper class that represents a field or property reference, and construct instances of that class by passing it an expression. You can find the source code to the class here:
http://code.google.com/p/fracture/source/browse/trunk/Squared/Util/Bind.cs
And we’ll be using it to eliminate the duplication from our sample application. The code is relatively small – a couple hundred lines – so by reading it you can easily see how to apply this technique for your own purposes, even if you don’t end up using my code.

The first step is to use expressions to build a list of all the values we want to store. We’ll do this by creating an array of bound members in the dialog box’s constructor:

IBoundMember[] BoundMembers;

public MainDialog () {
  InitializeComponent();

  BoundMembers = new IBoundMember[] {
    BoundMember.New(() => checkBox1.Checked),
    BoundMember.New(() => radioButton1.Checked),
    BoundMember.New(() => radioButton2.Checked),
    BoundMember.New(() => listBox1.SelectedIndices),
    BoundMember.New(() => textBox1.Text),
    BoundMember.New(() => dateTimePicker1.Value)
  };
}

Note that all we have to do is create an expression that evaluates to the value we wish to store. The C# compiler will convert it into an expression tree that tells us which object contains the value, and the FieldInfo or PropertyInfo we can use to fetch the value.

To eliminate the need to specify the name of each preference, we can combine the name of the field/property along with the name of the control containing the field/property, like so:

string GetMemberName (IBoundMember member) {
  return String.Format("{0}.{1}", ((Control)member.Target).Name, member.Name);
}

If we specify the Checked property of a RadioButton named radioButton2, this function will evaluate to ‘radioButton2.Checked’.

With these two pieces of functionality, we can now eliminate all the duplication from the old LoadPreferences and SavePreferences functions:

public void LoadPreferences () {
  if (!File.Exists(GetPrefsPath()))
    return;

  var serializer = new JavaScriptSerializer();
  var preferences = serializer.Deserialize<Dictionary<string, object>>(File.ReadAllText(GetPrefsPath()));

  foreach (var bm in BoundMembers)
    bm.Value = preferences[GetMemberName(bm)];
}

public void SavePreferences () {
  var preferences = new Dictionary<string, object>();

  foreach (var bm in BoundMembers)
    preferences[GetMemberName(bm)] = bm.Value;

  if (!Directory.Exists(Path.GetDirectoryName(GetPrefsPath())))
    Directory.CreateDirectory(Path.GetDirectoryName(GetPrefsPath()));

  var serializer = new JavaScriptSerializer();
  File.WriteAllText(GetPrefsPath(), serializer.Serialize(preferences));
}

Another thing you’ll notice is that the type casting from the original version is gone – since we have type information, we can automatically cast preference values to the appropriate type – and if we were using a strongly typed datastore, instead of JSON, we could actually store preferences directly in their correct type, without having to do any boxing or type casting.

If you download the source code for the revised example:
Preferences Example (v2)
You can run it and see that while it works the first time, and correctly saves preferences to JSON, it fails when loading preferences back from JSON. This turns out to be because of the JSON serializer: While listBox1.SelectedIndices is of type SelectedIndexCollection, the list we get back from the JSON serializer is of type ArrayList. The two types aren’t convertible, so the application fails.

Luckily, we can easily solve this problem. Since this technique works for any field or property, we can simply define a property that handles type conversion so that we don’t get an error when trying to load the preference value from JSON, like this:

// This has to be ArrayList since that's what we get back from the JSON serializer
ArrayList ListBox1_SelectedIndices {
  get {
    var result = new ArrayList();
    foreach (var index in listBox1.SelectedIndices)
      result.Add(index);
    return result;
  }
  set {
    listBox1.SelectedIndices.Clear();
    foreach (var index in value)
      listBox1.SelectedIndices.Add((int)index);
  }
}

If we change the list in the dialog box constructor to reference this new property instead of referencing the listbox directly, we can run the application and it will now successfully load and save preferences.
Preferences Example (v3)

With this technique, you can easily handle automatically loading and saving values from almost any datastore – and do so efficiently, without any unnecessary duplication, thanks to the information we’re given by the compiler. As an example, here’s a simple implementation of an object that loads and saves preferences from a database using this technique:
http://code.google.com/p/fracture/source/browse/trunk/Squared/TaskLib/Data/PropertySerializer.cs

I hope this post has shown you something useful! In the future I’ll be showing some other interesting applications of this technique.

I’ve released the source under the BSD license (just like Berkelium) and included two simple examples (one for Windows Forms, one for XNA). It’s not a Chrome competitor but it’s surprisingly easy to get a lot of complicated things working – you can watch Hulu videos if you have Flash installed, and Flash games like Dino Run work in the Windows Forms example since it implements keyboard input.

You can download pre-built demos to test it out, or grab the source code over at Google Code.

Have fun!

P.S. If you try out the examples, be aware that content that opens pop-up windows doesn’t seem to work. I think this is a bug in Berkelium.
Edit: Figured out how to build Chromium and Berkelium myself and fixed the bug. Open source is awesome!

Services

Boy, that’s a generic heading. Anyway, so your game is collecting important data on gameplay events, and it’s reporting that data periodically to a web service on your server. Unfortunately, since you haven’t actually written that service yet, it doesn’t seem to be working. Odd how that goes, isn’t it?

The primary issue you have to deal with when building a service to recieve collected data from your game is storing the data. There are other incidental issues – security, performance, serialization, etc – but none of them are of any significance if you can’t find a way to reliably store and access the data. This isn’t a simple problem, but there are ways to handle it.

Essentially, the biggest issue you have to confront here is that you are going to have a lot of data. You may not have a lot of data now, but you will eventually. On one hand, you could spend weeks and weeks trying to design the perfect solution for storing all of this data – but doing that doesn’t actually get your game any closer to shipping, and doesn’t actually guarantee that you won’t have scalability issues down the road. On the other hand, if you completely ignore the problem, you risk losing some of the data you’ve already gathered by discovering significant design issues after you ship. In practice, you probably want to prioritize shipping over the longevity of your data, since (assuming you ship and your game isn’t terrible) you can always get more data.

As a starting point, I decided to build my services on Google App Engine, since it was relatively easy to get up and running and provides fairly generous quotas for free.

I built my services in Python, using the provided datastore and JSON modules to handle most of the work. For those unfamiliar with how App Engine works, essentially your process is this:

• Modify your app.yaml file to specify the URLs of your services and the python scripts that implement them.
• Create one or more Python classes to be used for storing your data in the GAE datastore.
• Implement your services by creating python WSGI modules to process incoming requests and store the resulting data in the datastore.

For those more familiar with traditional databases, working with the App Engine datastore can be a challenge – its write performance borders on terrible, and the documentation is somewhat sparse. However, the actual API is very easy to use and since you’re using Python, most of the problems you’re dealing with are easy to solve. It helps that most of Django is included with the GAE APIs, since that means you have access to all sorts of useful tools like their HTML template module.

So, for the event format I showed in my previous post, the Python class looks like this:

class Event(db.Expando):
  playerId = db.IntegerProperty()
  sessionId = db.IntegerProperty()
  timeMs = db.IntegerProperty(indexed=False)
  eventType = db.StringProperty()

Fairly straightforward. Note that I’m deriving from Expando and not Model – to get an idea of what this means you’ll need to read the GAE documentation, but essentially this lets me add additional, unindexed fields to my instances as necessary. Since I only plan to do most queries based on event type and player ID, this is perfect.

Given that class definition, in my service, I can simply unpack the JSON I recieve from a game client and convert it into one or more Event instances, ready to send to the datastore:

      playerId = int(body["PlayerId"])
      sessionId = int(body["SessionId"])
      events = body["Events"]

      put_list = []

      for event in events:
        eventData = event["Data"]
        extraArgs = {}

        while eventData:
          if isinstance(eventData, types.DictType):
            d = eventData
            eventData = None
            for k in d.iterkeys():
              value = d[k]
              if k == "Data":
                eventData = value
              elif isinstance(value, types.DictType):
                continue
              else:
                key = ("%s" % (k,)).encode("ascii")
                extraArgs[key] = value
          else:
            extraArgs["eventData"] = eventData
            eventData = None

        evt = Event(
          playerId=playerId,
          sessionId=sessionId,
          timeMs=event["Time"],
          eventType=event["Type"],
          **extraArgs
        )
        put_list.append(evt)

      db.put(put_list)

The resulting data format doesn’t allow you to answer every question you could possibly have, but does leave you with a relatively easy to query volume of data in the google datastore. It’s also very simple, which makes it relatively straightforward to debug failures and make changes.

As mentioned before, the datastore’s write performance borders on terrible. If you attempt to perform a put on multiple objects in sequence, its write performance goes from bad to earth-shatteringly horrible. However, the datastore API allows you to hand it a list of objects, and it will attempt to put them all at once, which significantly improves your performance so that it’s at least tolerable. If I were using a relational database like MySQL this would be somewhat equivalent to trying to insert all my rows in a single multi-line query to reduce round-trip time.

Also of note are the somewhat ridiculous contortions I do here to convert event data into fields for the Event instance. Essentially, if the incoming event’s Data field is just one value, I shove it into a single field inside the Event – but if it’s a dictionary, I dig through it looking for values and store them into corresponding fields on the Event instance. This ensures that all the values attached to the event end up with corresponding columns in the datastore.

This also illustrates some slightly interesting design decisions: Events don’t have any global timestamp, nor do they have a sequence number – all they have is a player ID, session ID, and timestamp relative to the beginning of the session *in game time*. This is mostly a pragmatic decision; I could provide a global timestamp and get some benefits out of it, but the costs associated with that are quite significant, since accurate timestamps interact badly with batching and the general problem of internet latency.

One particularly nasty sticking point is that GAE has fairly rigid limits on how much CPU and ‘API CPU’ time you can consume in a given request, or in a given minute, or in a given hour, or in a given day. These limits are extremely easy to hit if you are using the datastore in the manner I am – so it’s important to make sure that your game isn’t sending large batches, or your requests will time out and google will become very angry with you for wasting precious CPU milliseconds. In practice, I’ve found that batch sizes between 6 and 12 reduce network traffic without hitting the built-in GAE limits. Your mileage may vary based on the nature of your data.

It’s also worth pointing out that the HistoryBlock approach I described in the previous approach is a bit problematic here: You’re basically forced to store all those position values within a single event, because turning them into multiple events would absolutely destroy the datastore. Unfortunately, it appears that storing a long list of values within a single event also absolutely destroys the datastore. Oh well, what can you do? (Other than switch to another hosting provider)

Also note that this doesn’t account for security, which means you probably can’t deploy it in production, because people like breaking your services. Oh well!

Now that we’re all familiar with how frustrating web service development is, let’s venture into the comforting realm of data analysis:

Presenting Your Data (Front-ends)

Mmm, much better.

Now that you’ve got all this data sitting around in the GAE datastore, what are you going to do with it?

Among the many things you can do, here are a few good ideas:

• You can perform high level queries against your event data to generate general ‘gameplay statistics’ for player profiles.
• You can perform statistical analysis of your event data to detect patterns in usage and player behavior.
• You can take the position values attached to your event data, and use it to generate heatmaps representing certain events or behaviors – player movement, player death, creature death, etc.

Since I’m uneducated, I’m going to skip the statistical analysis and describe the other two.

Creating gameplay statistics from your data is actually relatively easy, as long as you’re not worried about performance (and right now, you probably shouldn’t be). Even if performance is an issue, you can often solve this easily by track counters and statistics in a summary table, either by maintaining a count in your service when storing events, or by writing a cron job to periodically update your summaries from the most recently stored events. Doing this will allow you to maintain statistics without having to query your entire event history.

For example, given my event format, I can determine how many times a given player has killed the Drowned Queen by querying the datastore like so:

player_events = db.Query(Event).filter(
  'playerId =', playerId
)
drowned_queen_kill_events = player_events.filter(
  'eventType =', 'PlayerKilledCreature'
).filter(
  'CreatureName =', 'DrownedQueen'
)
drowned_queen_kill_count = drowned_queen_kill_events.count()

Essentially, what I’m doing here is taking my entire event data set, and iteratively filtering it down – first I start with the player ID, then I filter by the event type. Both of these columns are indexed, and each of these filters will eliminate a huge number of rows. After this I can filter by CreatureName; as long as I don’t have thousands of PlayerKilledCreature events this will be relatively cheap to do. After that it’s simple to ask the datastore how many events match my filter.

Given statistics like these, if you choose you can then come up with achievements and other information to display on a player’s profile page. Players tend to love these things, even though they’re not directly integral to a game experience. They give players a way to gauge their own skill at a game versus others, and also provide them with interesting ‘carrots’ to run after, whether it’s finding all the secrets in a game or defeating a boss without taking any damage.

So! Now for heat maps. If you’re not familiar with them, you might want to glance at the linked Wikipedia page, but they’re fairly easy to understand, at least conceptually. You essentially take a large volume of data (in our case, events) and use it to generate a bitmap that you can overlay on an image of a space that you care about. To give a real-world example, if you had a list of recent car accidents, you could generate a heat map from that list and overlay it on a map of your neighborhood, and possibly see clusters around particularly dangerous intersections. You can’t always rely on a heat map for drawing conclusions, but they make it much easier to spot patterns and trends because our brains are great at analyzing data when it’s presented in this format.

In my case, I want to take all that HistoryBlock data and convert it into a heat map, so I can figure out where players are spending most of their time. To do this, first I needed to get images of all my levels – this took some work, since I hadn’t previously done any offline rendering of my game content. In some cases you can do this sort of thing offline during your build process, but in my case since my renderer relies on Direct3D and XNA, I simply start up an instance of the game engine in a ‘batch processing mode’, and it scans through a level, using the 3D card to generate screen-size ’tiles’ of the level, one at a time, and save them out to disk. After that I scan through the tiles and scale them down to create a ‘minimap’ that represents the level at a given magnification.

Once I’ve got images for each of the levels, I can begin to generate a heat map for them. First, I query a service that pulls down all of the HistoryBlock events from the server and returns them as JSON. This sort of thing is likely to have scalability issues, so in practice you might want to only pull down a subset of your events – the most recent ones, or a random sampling – and use them instead. Right now my dataset is small enough that this isn’t a problem.

Once I’ve got the list of positions, I can scan over them and create the heat map. Essentially, what you want to do is create a 2D bitmap that is roughly the same size as your minimap, or some even multiple of it. Then, you can through all your positions and map them to points within that bitmap. Every time you successfully map a position to a point, you increment the value at that point, so that the most frequently reported locations have higher values.

Once you’ve done that, you can compute a minimum, maximum, etc. for the heat map, and use that to translate those ‘heat’ values into colors at each point. Then, all you have to do layer the resulting color bitmap onto your mini map, and you have a heat map. Here’s a few ones I just generated, based on the events I’ve been recording while testing out my services and achievement system. You may need to click on them to view them at full size:

This heat map was generated from a dozen or so play-throughs of the first level in the Level Up 2009 demo.

There are a few interesting conclusions you can draw from this heatmap – for example, you can easily spot the locations of switches/cranks, because those points are orange or red (from the player tending to stand in front of them to manipulate the object). Because of the short number of playthroughs, you can actually see faint traces of me dying by falling into the water.

The data also gives you a general idea of typical paths through the level, and shows you where players spend more time – for example, the ledge on the right side of the water is much brighter, because the player tends to stand there for a while fighting the monsters that are spawned there.

This presents some good opportunities to improve the flow of the level, address difficult platforming challenges, or make it easier for the player to find objectives (keen-eyed observers will note that there’s a secret passage in this level that I did not enter in any of my playthroughs. (: )

This one was generated from the small arena in which the player fights the boss from the demo, the Drowned Queen.

Even a quick glance at this heat map reveals some interesting patterns, despite the fact that it is based on a small amount of data.

I rarely died from falling into the water below, which indicates that the water is not particularly effective as a hazard (or that I’m particularly adept at avoiding it).

The extremely bright clusters to the left side indicate that I rarely moved around the arena while fighting the Queen, tending to instead dart back and forth within the same area. This might indicate problems with the design of the level or the boss, since you would usually expect players to be running around in an arena like this while fighting a well-designed foe.

The big bright red point up in the top left is aligned perfectly with one of the arena’s grip plates. While you’re supposed to use the grip plates to avoid one of the Queen’s special attacks, the huge amount of time spent there suggests that the boss’s AI is leading me to remain on the plate for a long period of time, looking for a safe opportunity to drop down, instead of actively fighting her and having a good time.

Keen-eyed viewers who’ve played the demo will also note that there are yellowish regions at each of the locations where a dialogue sequence occurs. This indicates that I tend to stand still while progressing through the dialogue sequence, despite the fact that the player is allowed to move freely. Interesting!

This final one is generated from the second level in the demo, the aqueducts.

Rather embarassingly, this demonstrates the perils of trying to draw conclusions from player data: Almost all the points in this heatmap are barely visible.

Why? Well, most likely, I left the game running with the player standing in that position for a long time, which resulted in that point being far, far brighter than the rest of the level.

Of course, there are ways to address these sorts of problems with your data – but if you’re not careful, they may influence your conclusions without you even realizing it. In this case I would probably want to discard or ignore any points that have values vastly out of whack compared to the rest of the heat map, so that the heat map will demonstrate general trends instead of only showing outliers.

I hope these posts have given you an idea of how you can go about gathering player data in your games, and how you can put that data to work – both towards improving your game, and giving your players access to data they want.

For a system like this you have a few important pieces:

• Your game needs to collect data during play sessions – in my case, I track important events like player/creature death, and periodically sample the player’s position to get a general idea of where players are in a given level during their session.
• You need a way for your game to periodically report collected data to a remote server. You have to handle lots of edge cases here – for example, it’s likely some people will play without an active internet connection or suffer temporary loss of connection, so you need to batch up events to report later in this case – you definitely don’t want the game to fall over and choke without access to the internet, and if possible you want to avoid losing data too.
• You need to build a set of services to collect data from the game. This typically means you also need services to handle things like uniquely identifying individual play sessions and computers, so that you can track achievements for individual players and perform analysis on individual sessions instead of only on a player’s entire play history.
• You need to build an in-game frontend, to expose collected data to a player. This means turning your event data into user-friendly statistics – like a kill counter, or an achievement for killing a particular boss. One interesting challenge here is that you probably want to expose this information online, too, so that players can share their profiles and achievements.

In this post, I’m going to cover the first two.

Collecting Data

So, first, you need a way to collect important player data. For it to be useful, you need to collect it in as consistent a format as possible; you don’t want every piece of data to have a unique set of parameters attached to it, for example, because that would make it impossible to perform any generic analysis of your data. Likewise, you want to try and keep your data ‘denormalized’, by storing as much relevant data together as possible. If you achieve both of these goals, you end up with a stream of ‘events’ that represent the things you care about in a form that lets you inspect them individually or analyze them in large groups.

In my case, this step was actually quite straightforward. I already had a simple ‘event bus’ integrated into the game, that I was previously using to synchronize animation and sound effects with gameplay. Given this, it was simple to create a data collector that attaches to the event bus and listens for events that it wants to collect.

I did, however, have to make some changes – many of my events didn’t expose the information necessary to be useful; for example, the event representing a player death didn’t contain any information to tell you who (or what) killed the player, so I had to tweak the game code to attach that to the event. Likewise, many other events didn’t include position information, which made it hard to get meaningful information about them – knowing that the player grabbed onto a ledge isn’t particularly useful if you don’t know *where* or *which ledge*. In some cases, the data collector attached important parameters into the events so that game objects wouldn’t have to; for example any event coming from the Player automatically has PlayerX and PlayerY parameters attached to it (since the Player is the source of the event, it is simple to extract a position and attach it to the event).

One thing that required some special treatment as well was recording the player’s position over time – while attaching position information to events gives you a general idea of the player’s location, it’s not very useful for figuring out where players are getting stuck or confused. To address this, I built a simple class that periodically samples the player’s position and adds it to a list. Every time the list reaches a certain size, its contents are batched up into a single event – a ‘HistoryBatch’ – and sent to the data collector. This allows me to get fairly high-resolution position data without having to report hundreds of individual events every minute (which would have contained lots and lots of redundant information).

Reporting Data

Once you have your player data, you need to report it to your server (and probably store some of it locally, too). I honestly haven’t tackled the latter part yet, but here’s how I handled the former:

To handle periodically reporting events, I created a simple ‘Event Reporter’ class that collaborates with the data collector. Essentially, the event reporter maintains a queue of pending events, and runs a thread that is responsible for sending batches of events to the remote server once the queue gets large enough. It automatically waits a certain amount of time between requests, and attempts to batch events into groups instead of reporting them one at a time as they occur. This allows me to get relatively low latency (if I want it) but still remain relatively efficient in my use of the network. Whenever the data collector gets an event, it adds it to the event reporter’s queue.

Once I have a batch of events ready to send to the server, I pack them together into a class that’s laid out optimally for network transmission. I then use .NET 3.5′s built in JSON serializer to convert the batch into a blob of JSON data ready to send to my remote server. A typical blob looks like this:

{
  "PlayerId":xxxx,
  "SessionId":yyyy,
  "Events":[
    {"Type":"ActiveControllerChanged","Time":0,"Data":{
      "IsKeyboard":true,
      "ConnectedGamepads":0,
      "ActiveController":0
    }},
    {"Type":"LevelLoaded","Time":5436,"Data":"troupe"},
    {"Type":"HistoryBlock","Time":7788,"Data":{
      "LevelName":"troupe",
      "PlayerX":[-136,-74,35,192,209,209],
      "PlayerY":[588,875,920,920,920,920]
    }}
  ]
}

As you can see, events are small when possible – they simply store the event type and the timestamp at which the event occurred, along with their associated information. In cases where an event has lots of information attached, I send a dictionary, but in some cases I can just send a single value. In the case of a history block, I make sure to send the X and Y coordinates as individual lists, so that I don’t have to waste time encoding ‘PlayerX’ and ‘PlayerY’ along with every individual coordinate. A batch of events has a player ID and a session ID attached, which allows me to analyze individual play sessions and track achievements. (Note that both of these IDs are randomly generated on the server, so they’re anonymous.)

The reporter starts an HTTP POST to send the JSON to the server, and goes to sleep until it’s finished. If the send fails, the events are left on the queue and the thread sleeps for a while so it can retry sending them later. Once a send completes, the reporter checks to see if the queue is empty – if the queue is now empty, it goes to sleep until the queue contains items again. If the queue still contains items, it sleeps for a short while and prepares to send another batch – this allows the event reporter to spend the majority of its time asleep, but still handle large bursts of events without sending a huge blob of events at the server in one POST (since that would be very likely to fail or time out.)

One important detail is that the reporter needs to be able to correctly report events even if the player quits. In my case, the player can quit at any time, and I can end up with a very large number of events in the queue at the time. To deal with this, once the game has finished tearing down, the event reporter is given around 30 seconds to report any remaining events to the server. This usually allows events to get through, but prevents the game from hanging indefinitely on exit (in the event that there’s some sort of network failure, or the event queue is really full). Since this occurs after teardown, the game window is gone so the player doesn’t get the impression that the game has hung, and since the reporter spends most of its time sleeping, they won’t see any significant CPU usage either.

The reporter also runs in its own isolated thread and uses its own task scheduler, instead of sharing resources with the game. This increases the likelihood that the game will be able to successfully report events in the case of a bug or crash, and also means that if the reporter fails for some reason – probably a network outage – it won’t take the game down with it.

In my next post, I’ll cover the remaining two items and show you some of the things you can do once you’ve gathered player data. In the interim, you can take a look at my player profile!

The solution to this was, at least to me, relatively obvious: My levels were much larger than the camera, so it didn’t really make sense to update the entire level every frame.

The first thing I tried to verify this theory was simply hacking it in: Do a check before updating each object to see if it was onscreen. Interestingly, this didn’t make the game any faster on the 360. Depending on your point of view, this either confirmed or denied my hypothesis: If the problem was simply the cost of all the floating point operations, the cost of doing the onscreen check for each object (since the camera and object bounds were both expressed in floating-point) could have been making the problem worse. Clearly, I didn’t have enough data to be sure about the right choice to make.

So, I spent a day or so rigging up the necessary infrastructure to be able to profile my game on the 360. Since you can’t use tools like CLR Profiler or NProf on the 360, I ended up building a very simple frame timing system, and adding an overlay to the game that would show timing data. This let me get a good idea of how much time each subsystem in the game was using, and then I could compare the costs of individual subsystems, and try making changes and seeing how the profile data changed.

Once I had the profiler up and running on the 360, a clear pattern emerged.

Updates were consuming a huge amount of CPU time on the 360. While on my desktop, updates basically accounted for no more than 1% of CPU time, on the 360 they actually accounted for more CPU time than rendering – this was actually a bit of a surprise to me since rendering was definitely the bottleneck at one point on the 360. It seems that at some point along the way, I solved my rendering performance issues on the 360, but didn’t notice because I had made updates so much more expensive – one mistake I plan not to repeat was that I went a week or two without testing the game on the 360, since my 360 was not hooked up at the time. During that span of time I made a lot of changes that drastically altered the game’s performance characteristics, so it was hard to tell what had caused things to degrade.

Now that I knew updates were expensive, I decided to try and narrow down which object types were the problem. I added profiling markers around specific types of objects, to try and figure out which ones cost the most CPU time to update. After doing that and deploying a few builds to the 360, I found one of my culprits: Water.

While I had suspected that water might be a problem, I was still somewhat surprised by the results; A similar piece of in-game geometry, zones, used similar update code but turned out to have virtually zero update cost at runtime, while water cost a tremendous amount of CPU for very little gameplay impact. The problem turned out to be a subtle difference in how they were implemented.

Zones were inefficient in that every frame, each zone did an obstruction test to see if any entities were inside – in most cases a zone would be empty, so this was wasted effort. This would have been better implemented by having every entity check for nearby zones, since there are typically far less entities than there are zones. However, in practice this didn’t turn out to be the problem. The problem was that water built on this logic, and then used it to perform additional work: It did an obstruction test to determine how far the water should fall, and then did another obstruction test to locate any entities inside the water and apply ‘flow force’ to them so that the flowing water would push them in a given direction. These two obstruction tests each ended up accounting for a significant portion of the time spent updating the level.

To solve this, I took two steps. First, I reworked zones and water so that they both operated the way I described – each entity does a check to locate all the zones it’s within, in a single obstruction test. This reduced the number of obstruction tests I was running every frame by a large amount, and helped reduce CPU usage for water. However, that still wasn’t enough.

The biggest improvement came from reworking things so that only onscreen objects get updated every frame. Instead of performing a test against every object to see if it’s onscreen, I decided to build on the partitioning scheme I use for rendering to get a list of onscreen objects, and use that as my list of objects to update. Once I had that working, my performance improved drastically and the game easily ran at 60fps in every part of my levels with CPU to spare.

Of course, doing this introduced bugs. One of the biggest issues is that often you will have important objects just outside the edge of the screen that need to keep updating, like moving platforms or enemies. To solve this, I added two mechanisms:

First, I added a margin around the screen within which objects still continued to update. This meant that an object just barely offscreen would keep updating, and solved the problem of a moving platform or enemy getting left behind as soon as he dropped offscreen.

Second, I built a simple ‘update manager’ that maintains a list of all the objects that are currently onscreen. When an object leaves the screen, instead of removing it immediately, the update manager instead sets a timeout, which causes the object to become ‘asleep’ within a certain number of frames. As a result, once an object leaves the screen it has a second or two to return to the screen before it falls asleep, which helps with things like moving platforms that are going to move on and off the screen regularly – since they don’t spend very long offscreen, they never have a chance to fall asleep.

The update manager also gives me the ability to exclude some objects from updates entirely, by flagging them as ‘unable to wake’, so the update manager knows to never remove them from sleep status. Likewise, it gives me the option to flag objects as ‘unable to sleep’ so that they always stay active – for example, I do this to the player character and his companion to avoid any unintentional bugs that might result from the player remaining offscreen too long (say, during a cinematic).

The update manager has some other benefits, too. Here’s what it looks like:

    public class UpdateManager<T>
        where T : class, IUpdateable, IHasBounds {

        public struct Entry {
            public readonly T Object;
            public int WakeCounter;

            public Entry (T obj, int wakeCounter) {
                Object = obj;
                WakeCounter = wakeCounter;
            }
        }

        protected Dictionary<T, bool> _VisibleObjects = new Dictionary<T, bool>(new ReferenceComparer<T>());
        protected UnorderedList<Entry> _Entries = new UnorderedList<Entry>();

        public int SleepTimeout = 30;
        public readonly SpatialCollection<T> Collection;

        public UpdateManager (SpatialCollection<T> collection) {
            Collection = collection;
        }

        public void Update (Bounds liveRegion) {
            using (var e = Collection.GetItemsFromBounds(liveRegion, true))
            while (e.MoveNext()) {
                _VisibleObjects[e.Current.Item] = false;
            }

            Entry currentItem;

            using (var e = _Entries.GetEnumerator())
            while (e.GetNext(out currentItem)) {
                if (!_VisibleObjects.Remove(currentItem.Object)) {
                    // Object not visible

                    if (!currentItem.Object.AllowSleep) {
                        // Object cannot fall asleep
                    } else if (--currentItem.WakeCounter == 0) {
                        // Object fell asleep
                        e.RemoveCurrent();
                        continue;
                    } else {
                        // Object still awake
                        e.SetCurrent(ref currentItem);
                    }
                } else {
                    // Object visible

                    currentItem.WakeCounter = SleepTimeout;
                    e.SetCurrent(ref currentItem);
                }

                currentItem.Object.Update();
            }

            foreach (var newObject in _VisibleObjects.Keys) {
                if (newObject.AllowWake) {
                    _Entries.Add(new Entry(newObject, SleepTimeout));

                    newObject.Update();
                }
            }

            _VisibleObjects.Clear();
        }
    }

One of the nice things it does in addition to improving performance is that it simplifies my game code – previously, I had hand-written logic to step through the various object lists (geometry, entities, etc) and update them every frame, and that code differed in subtle ways. The update manager unifies all that, so it kills a lot of duplicated code. Also, by maintaining a unique ‘awake objects’ list, it allows me to remove objects from the geometry/entity lists while performing an update, instead of having to wait until the end of the frame, which simplifies some of my entity code as well.

And despite the amount of problems it solves, it ends up actually being quite simple and easy to use. All I have to do to rig up an update manager is point it at a collection of objects and hand it the current camera boundaries every frame.

A customer downloads and installs your game. He’s running it on a Windows PC. He has his keyboard layout set to DVORAK, because he heard it helps reduce wrist strain. His keyboard still has QWERTY labels printed on it, so he presses ‘S’ to type an ‘O’.

He starts your game up, and let’s say the splash screen says ‘Press S to continue’. He presses S on his keyboard.

Nothing happens.

This is because the OS is remapping the S keystroke into an O. For your XNA game to see an S keystroke, he will have to press the semicolon key (;:) instead.

While users with alternate keyboard layouts are a comparative minority compared to people using the QWERTY keyboard layout, that’s no excuse for ignoring them. As it turns out, the solution to this problem is pretty straightforward.

What you need to do is find a way to map a keystroke in your keyboard layout – let’s say QWERTY – to whatever keyboard layout your customer is running at the time. The Win32 API actually makes this quite simple, so as long as you’re willing to include some P/Invoke code in windows builds of your game, you can accomplish this in a couple dozen lines of code.

To accomplish this, I use the following helper struct:

uusing System;
using Microsoft.Xna.Framework.Input;
using System.Runtime.InteropServices;

namespace Labyrinth.Framework {
    public struct LocalizedKeyboardState {
        internal enum MAPVK : uint {
            VK_TO_VSC = 0,
            VSC_TO_VK = 1,
            VK_TO_CHAR = 2
        }

        [DllImport("user32.dll", CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError = true)]
        internal extern static uint MapVirtualKeyEx (uint key, MAPVK mappingType, IntPtr keyboardLayout);
        [DllImport("user32.dll", CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError = true)]
        internal extern static IntPtr LoadKeyboardLayout (string keyboardLayoutID, uint flags);
        [DllImport("user32.dll", CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError = true)]
        internal extern static bool UnloadKeyboardLayout (IntPtr handle);
        [DllImport("user32.dll", CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError = true)]
        internal extern static IntPtr GetKeyboardLayout (IntPtr threadId);

        internal const uint KLF_NOTELLSHELL = 0x00000080;

        public struct KeyboardLayout : IDisposable {
            public readonly IntPtr Handle;

            public KeyboardLayout (IntPtr handle) : this() {
                Handle = handle;
            }

            public KeyboardLayout (string keyboardLayoutID)
                : this(LoadKeyboardLayout(keyboardLayoutID, KLF_NOTELLSHELL)) {
            }

            public bool IsDisposed {
                get;
                private set;
            }

            public void Dispose () {
                if (IsDisposed)
                    return;

                UnloadKeyboardLayout(Handle);
                IsDisposed = true;
            }

            public static KeyboardLayout US_English = new KeyboardLayout("00000409");

            public static KeyboardLayout Active {
                get {
                    return new KeyboardLayout(GetKeyboardLayout(IntPtr.Zero));
                }
            }
        }

        public readonly KeyboardState Native;

        public LocalizedKeyboardState (KeyboardState keyboardState) {
            Native = keyboardState;
        }

        public bool IsKeyDown (Keys key, bool isLocalKey) {
            if (!isLocalKey)
                key = USEnglishToLocal(key);

            return Native.IsKeyDown(key);
        }

        public bool IsKeyUp (Keys key, bool isLocalKey) {
            if (!isLocalKey)
                key = USEnglishToLocal(key);

            return Native.IsKeyDown(key);
        }

        public bool IsKeyDown (Keys key) {
            return IsKeyDown(key, false);
        }

        public bool IsKeyUp (Keys key) {
            return IsKeyDown(key, false);
        }

        // Maps a localized character like 'S' to the virtual scan code
        //  for that key on the user's keyboard ('O' in dvorak, for example)
        public static Keys USEnglishToLocal (Keys key) {
            var activeScanCode = MapVirtualKeyEx((uint)key, MAPVK.VK_TO_VSC, KeyboardLayout.US_English.Handle);
            var nativeVirtualCode = MapVirtualKeyEx(activeScanCode, MAPVK.VSC_TO_VK, KeyboardLayout.Active.Handle);

            return (Keys)nativeVirtualCode;
        }
    }
}

Here’s how it works: First, at startup, we ask the Win32 API to load up a specific keyboard layout; US English QWERTY. From then on, we can ask the Win32 API to convert a ‘virtual key’ from that keyboard layout into a scan code. Once we have a scan code, we can then ask the Win32 API to convert that scan code into the equivalent virtual key for the end-user’s keyboard layout. Since the XNA Framework uses virtual keys (the Keys enumeration contains virtual key values), this allows us to apply this technique to existing keyboard input code without any significant changes.

So, with this helper struct, you can add support for alternate keyboard layouts like DVORAK with only a couple changes:

• First, add the helper struct to your game code somewhere so you have access to it.
• Replace any uses of the XNA KeyboardState struct with the LocalizedKeyboardState helper struct. If you use some of the more obscure KeyboardState helper methods, you may need to do some work to add them to LocalizedKeyboardState; it only provides IsKeyUp and IsKeyDown.
• Figure out whether any of the keys you’re using should not be remapped based on the current keyboard layout. For example, it’s common practice for some kinds of games to expose a ‘console’ that allows the user to view log messages and enter commands. In a lot of games, you press the tilde (~) key to open the console. This is a convenient key since it’s near the top left corner of a QWERTY keyboard. However, if you allow the keystroke to be remapped to the current layout, non-QWERTY typists will find they have to press some random key on their keyboard to open the console. In this case, you’ll want to pass a value of true for the second, optional parameter to the IsKeyDown/IsKeyUp helper methods:

if (ks.IsKeyDown(Keys.OemTilde, true))
    ShowConsole();

However, doing this made it clear that I had some architectural issues to tackle: All of these mechanics were heavily dependent on the tunable constants for the creature in question, which meant I couldn’t just pull methods and variables out of my entity classes into classes of their own.

To solve the problem of accessing an object’s constants, I came up with a solution based on reflection. I can define a helper object designed to handle an aspect of an entity’s mechanics – for example, a HealthPool object to manage the creature’s health, along with associated aspects like regeneration. The helper object can define instance variables for the constants it needs access to, like so:

public class HealthPool {
    public Constant<float> HealthMax = null;
    public Constant<float> HealthPassiveRegen = null;
    public Constant<float> HealthRegenDelayTime = null;
    public Constant<float> HealthRegenRampTime = null;
    public Constant<float> FlinchThreshold = null;
    public Constant<float> FlinchThresholdDecay = null;

    public readonly RuntimeEntity Entity;
    public readonly ITimeProvider TimeProvider;
    public float Health = 0.0f;

Note that these variables are the same name and type as the actual tunable constants – the difference is that instead of being static, they’re instance variables. Doing this allows me to pull a function out of an entity’s source code without needing to change the way it references particular constants, since the constants have the exact same names as before.

Of course, since these variables default to null, we need some way to fill them in with references to the actual tunable constants we want to use. To do that, we apply reflection:

public static void BindConstants (object destination, params Type[] sourceTypes) {
    var genericConstant = typeof(Constant<>);
    var destinationType = destination.GetType();
    var destinationFields = new Dictionary<string, FieldInfo>();

    foreach (var field in
        destinationType.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.FlattenHierarchy)
    ) {
        var fieldName = field.Name;
        var fieldType = field.FieldType;

        if (!fieldType.IsGenericType || fieldType.GetGenericTypeDefinition() != genericConstant)
            continue;

        destinationFields[fieldName] = field;
    }

    foreach (var sourceType in sourceTypes) {
        foreach (var field in
            sourceType.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.FlattenHierarchy)
        ) {
            var fieldName = field.Name;
            var fieldType = field.FieldType;

            if (!fieldType.IsGenericType || fieldType.GetGenericTypeDefinition() != genericConstant)
                continue;

            FieldInfo destinationField = null;
            if (destinationFields.TryGetValue(fieldName, out destinationField)) {
                destinationField.SetValue(destination, field.GetValue(null));
                destinationFields.Remove(fieldName);
            }
        }
    }

    if (destinationFields.Count > 0) {
        var constants = String.Join(", ", (from key in destinationFields.Keys select key).ToArray());
        var types = String.Join(", ", (from type in sourceTypes select type.Name).ToArray());

        throw new InvalidDataException(String.Format("Type(s) {0} do not declare the following constants:\n{1}", types, constants));
    }
}

What we’re doing here is pretty simple: We accept a reference to an object that has constants requiring binding, and a list of source types to retrieve constants from. The function operates in two stages: First, we enumerate all the instance variables defined in the target object, and build a list of all the tunable constants it has that need to be bound. After that, we enumerate all the static fields of the provided source types, looking for constants that have names matching those of the instance variables on the target object, binding them where appropriate. After this, we can simply check to see if our list of constants is empty or not – if it’s empty, we successfully bound all our constants, and if it’s not, we know that one or more of the desired constants was missing.

We get a few useful things out of this: First, accepting a list of types allows us to do simple inheritance of constants. If we first check the most-derived type and then the base type of an entity, that allows us to define a ‘default value’ for a particular constant, like ‘Maximum Health’, in a base class, and then define a new constant with the same name in the derived type. This also allows us to create ‘global defaults’ for a given constant – for example, if we always put Game at the end of the type list, we can have global game-wide constants for things like physics parameters, and only override them in specific classes if necessary.

Finally, to wire things up, we just need to do a little work in the constructor for our helper object:

    public HealthPool (RuntimeEntity entity, ITimeProvider timeProvider) {
        Entity = entity;
        TimeProvider = timeProvider;

        ConstantManager.BindConstants(this, entity.GetType(), entity.Game.GetType());

        Health = HealthMax;
    }

In this case, we’re initializing the HealthPool using the constants defined in the entity, and falling back to any constants defined in the Game when the entity doesn’t specify them. If a necessary constant is missing, we’ll get an exception thrown when constructing our helper object. Once we’ve bound the constants, we can just use them like we would otherwise – in this case, the HealthPool automatically initializes itself based on the HealthMax constant.

This is definitely preferable to the approach I used to use for exposing an entity’s constants – previously, for important constants like an entity’s bounding box size, I’d define an abstract property in a base class, and override it in each derived type to return the value of the constant. Now, I don’t need to use any abstract members or interfaces; I can just bind to the constants once when I initialize my helper objects.

One thing of note is that since this technique uses reflection, you might run into performance issues if you’re binding constants repeatedly. This is pretty trivial to solve, however; you can just cache the results of a constant binding operation based on the destination and source types, since those aren’t going to change at runtime.

• Combo System & Attack Chains
• Instance Limiting Overhaul
• Active Controller Detection

Combo System & Attack Chains

Previously, combat basically consisted of hitting the punch button over and over to kill monsters. This sucked.

I took a few major steps to address this:

First, I added a second attack type bound to the button I was previously going to use for the grappling hook. This attack is slower and hits heavier, with a wider attack arc, and gives you a nice alternative to the fast, lighter-hitting punch attack.

After the addition of the second attack type, I built on that to add ‘combo’ variations of each attack (punch combo, slash combo). The player can combo these attacks onto a previous attack by properly timing another button press near the end of the previous attack animation. Missing the timing (either by pressing too early or too late) ‘botches’ the combo and causes the delay before they can perform another attack to be longer than it would be otherwise. The size of the time window in which you can successfully combo an attack decreases every time you combo, so over time it gets harder. This means that simply attacking once will be slow and inefficient, but you are also prevented from comboing attacks indefinitely, which strikes a good balance. The fact that a failed combo has a longer delay than a normal attack means that a player won’t be punished for choosing to simply combo once or twice and then attack again, since the effectiveness ends up being nearly the same.

Finally, I added a ‘attack chain’ system that tracks the number of hits you land on a foe in rapid succession. This allows me to delay the ‘flinching’ animation normally played when a creature recieves damage, so that the creature stays within reach of your attacks, allowing you to combo. Each successive hit extends the chain for a short period of time, allowing you to continue landing blows, and when the chain ‘breaks’, all the hits you landed deal their damage to the creature at once, knocking it back and possibly killing it. Chains can span across multiple creatures as well, allowing you to keep multiple enemies ‘locked’ by your chain at once. Right now it’s a bit overpowered, but I think some careful tuning will maintain most of the positive aspects without making the game too easy.

Instance Limiting Overhaul

Recently, while working on audio improvements, Troupe ran across a bug in XACT. PC builds of the game ran perfectly without any significant CPU usage problems, and the audio sounded great – but on the 360, as soon as enough channels of audio started playing, the game’s framerate tanked to around 15FPS and stayed there indefinitely. This was despite the fact that the game already issues most of its audio calls on a background thread to work around the prohibitively high cost of XACT’s API calls.

The problem turned out to be that despite the fact I was using XACT’s built in instance limiting support to control the number of instances playing at once, XACT was struggling to handle the number of cues I was asking it to start at once. Essentially, I was starting all the ambient loop cues for my level at once – around 30 – and expecting it to pick the 8 loudest ones to play at any given time based on the instance limit. On PC this worked perfectly without any performance issues, but for whatever reason, not so on the 360.

As a result I basically tore out all the existing code that relied on XACT instance limiting, and reimplemented limiting inside my engine. Luckily this only required changes to about 500 lines of code, but it was still rather frustrating to have to reimplement it when it worked perfectly on PC. On the bright side, now I have more control over how instance limiting behaves, so I at least got something out of it.

Active Controller Detection

While doing some testing on my 360 I realized that my approach to handling the 360 controller was incorrect. While I assumed that the player might want to play with any of their four connected controllers, I neglected to notice that most of the XNA Guide APIs (storage device selection, etc) are designed to only respond to input from a single controller. This meant that I needed to detect which controller the player was currently using and make sure to only show XNA dialog boxes using that controller, and that I also needed to detect if that controller became unplugged so that I wouldn’t attempt to show a dialog box the player was unable to close.

I ended up spending a while changing my input framework so that it would automatically detect disconnected controllers, and inform the game code when a controller had been reconnected. I also did some work to automatically track the active controller, while still handling keyboard input correctly on the PC.

Below, you can see a short annotated video walkthrough where I demonstrate the technique and show how it integrates with XACT.

There are a few key pieces necessary for this to work.

First, I need a way to get updated audio into the game. It turns out this is pretty simple – if you change the settings in your XACT project, you can get it to build output files into a folder of your choice. At that point, all you need to do is change your game’s XACT code to be able to load from that location.

Second, I need a way to pull information out of the XACT datafiles that I can use to attach cues to events. Since the XACT API provided by the XNA Framework is basically useless for this purpose, I ended up solving this problem by loading up the raw datafiles and pulling the names of my cues out of the datafiles. Yes, it’s disgusting. But it works! Luckily, the cue names are right at the end of the data files, and they’re null-terminated, so it’s not difficult to read them. It’s beyond my understanding why the framework developers opted not to provide the information.

Third, I needed a way to broadcast events from objects in my game world. The solution I ended up building for this was essentially an improved version of the event framework that I previously used for handling input events, like button presses. The framework allows me to ‘subscribe’ to various types of events, either for a specific object, or for all objects that can broadcast that event. The ability to subscribe to all objects inexpensively gives me a straightforward way to say things like ‘whenever any creature gets hit by a rocket, play an explosion sound’, which is important.

Fourth, I needed a way to expose event information in an understandable manner, so that it would be easy to figure out what events are occurring in-game and how to attach sounds to them. I solved this by creating a simple ‘event overlay’ that shows a list of the most recently fired events, and highlights objects when they broadcast events. This allows you to simply play the game with the overlay open and look for things that are missing sound effects – once you find something, just look at the log to find out the name of the event.

If you’re interested, you can check out the source code and automated tests for the event system here, on Google Code. In the future I will be releasing the rest of my audio framework as open-source for people to use. Please don’t hesitate to leave a comment if you have any questions; I’d be glad to help explain more about how this technique works.