My basic approach is primarily inspired by Christer Ericson’s post about the approach he uses for ordering draw calls. Some other useful sources of information were Tom Forsyth’s post about the cost of renderstate changes and Guerilla’s presentation on Killzone 2′s renderer from DEVELOP 2007. The Killzone 2 presentation is especially worthwhile since it describes the way they take advantage of concurrency techniques in their renderer.

Until now, the rendering code in my game has been almost entirely immediate-mode; various objects in the game world like tiles and entities had Draw methods that would utilize the game’s GraphicsDevice and SpriteBatch to render themselves, changing render states as needed and generating dynamic geometry where necessary (like for water).

Of course, if every object changes render states, you quickly run into a situation where you’re changing states tens of thousands of times per frame, which doesn’t perform very well. As a result, I ended up having to add some basic support for grouping together identical render states. In the end, I often had to perform multiple passes over the scene in order to minimize state changes, and the addition of multiple onscreen ‘layers’ meant that I often had to make half a dozen passes over the same collections (like Level.Entities and Level.Geometry) in order to render things in the correct order without too many state changes. My new renderer clearly needed to provide an inexpensive way to reduce state changes without complicating my code.

In addition to the issue of state changes, however, was the simple fact that rendering was too expensive. In particular, rendering all the onscreen tiles for the 6+ layers in the level could often add up to 5%+ of my CPU time, due to the relative inefficiency of the XNA SpriteBatch class when trying to draw thousands of bitmaps at various locations on the screen. The use of SpriteBatch also meant that I couldn’t effectively sort tiles by texture, because using a custom blending configuration with SpriteBatch requires you to disable its texture sorting support. (My tiles and sprites have to be premultiplied so they look correct when scaled, unfortunately.) As a result, my new renderer needed to minimize the cost of individual drawing operations, and it also needed to give me a way to utilize material/texture sorting with custom blending configurations.

One final detail that factored into the design was concurrency: Between the cost of actually rendering the game world, and the cost of waiting for vertical sync, the main thread was spending as much as 40% of its time performing rendering, which made it much harder to maintain a smooth, consistent 60fps framerate. In order to minimize the amount of time the main thread spent blocked during rendering, my new design needed to make it possible to move rendering work off the main thread without introducing thread safety issues – putting a lock around all my game state wasn’t going to cut it.

Every frame, at the beginning of my Draw method I construct a new Render.Frame object to represent the frame that’s being rendered.

The Render.Frame object contains a list of Render.Batch objects; each Batch has an associated Layer and Material. Layers are simple integers that allow me to explicitly order batches, so that I can create batches in any given order, and even create multiple batches at once. This also allows me to perform a single sort of the Batches array before sending the batches off to the video card, in order to minimize state changes.

Any given Batch contains a list of structures representing ‘Draw Calls’. In some cases, a draw call maps directly to a hardware drawing operation – like DrawUserPrimitives – but in other cases, it maps to something more granular. For example, a BitmapBatch contains BitmapDrawCalls, where each draw call represents a single bitmap, much like the arguments you pass to SpriteBatch’s Draw method. This allows me to sort individual draw calls based on their parameters to minimize state changes within a batch.

Finally, the Material objects associated with a given Batch are a superset of the XNA’s Effect class – they include a VertexDeclaration, Effect, and optional delegates for configuring other rendering state like the current blending function or stencil state. Grouping all these parameters together in one object allows me to sort batches cheaply by comparing material instances, but still gives me some level of granularity since I can change the shader parameters of an active Effect within a batch, for example to support rendering multiple textures inside a single BitmapBatch.

In addition to the more obvious performance advantages of this approach, like the ability to sort by material, one other advantage is less obvious: Since a Frame contains information on all the drawing operations that need to be performed, but doesn’t depend on any of my game state, I can safely hand that object to another thread and have that thread perform the drawing operations. This lets me move a significant portion of my rendering off the main thread, and begin performing my next Update while the previous Draw completes, without needing to add any locking or complex synchronization.

First, I basically have to reimplement everything from scratch – SpriteBatch and SpriteFont are both completely impossible to extend, so I have to reimplement them in order to render text and bitmaps with this approach, and the same goes for any other rendering code based on directly manipulating a GraphicsDevice.

Second, this approach is inherently more dependent on the garbage collector, since most of the types must be classes by necessity. If I want to run well on the 360, this means I need to make use of pooling and other techniques to avoid frequent allocations during frames. I’m also no longer able to reuse a single scratch buffer when generating geometry, so every piece of geometry I render needs to have its own buffer – more allocations.

So far I’m pretty pleased with this approach. There’s more work to be done – for example, I don’t have pooling implemented so my game collects extremely often on the 360 – but a large portion of my game is now running on this new rendering architecture, and my average framerate has already improved slightly from moving work off the main thread, despite the fact that I haven’t spent any time on optimizations.

        public void RenderTileLayer (int index) {
            var layer = RuntimeLevel.Layers[index];
            RuntimeLayer.ItemInfo itemInfo = null;

            BeginSpriteBatch(BlendModes.AlphaPremultiplied);

            using (var e = layer.GetItemsFromBounds(Camera.Bounds))
            while (e.GetNext(out itemInfo)) {
                RenderTile(itemInfo.Item, SpriteBatch, Camera.ViewportPosition, Camera.Zoom, AnimationTimeProvider.Ticks);
            }
        }

        private void RenderTile (RuntimeTile tile, SpriteBatch spriteBatch, Vector2 viewportPosition, float zoom, long time) {
            var info = tile.TileInfo;
            var pos = (tile.Bounds.TopLeft - viewportPosition) * zoom;
            spriteBatch.Draw(info.Texture, pos, info.Rectangle, Colors.White, 0.0f, new Vector2(0, 0), zoom, strip.GetSpriteEffect(), 0.0f);
        }

You can see here that my approach for rendering tiles is pretty simple: Get all the tiles within the screen’s boundaries, and render them one by one using a SpriteBatch. I’m able to reduce the number of state changes since I know that every tile within a given layer shares the same state, but I still need to change state once per layer (using the BeginSpriteBatch function, which calls SpriteBatch.Begin and sets up my render state). I also have to go out of my way to render each layer in the right order, which means calling RenderLayer in multiple places so that certain tiles appear below entities while other tiles appear above entities.

The new implementation looks like this:

        public void RenderTileLayer (int index) {
            var layer = RuntimeLevel.Layers[index];
            RuntimeLayer.ItemInfo itemInfo = null;

            int drawLayer = (index <= 2) ? DrawLayers.Background : DrawLayers.Foreground;

            using (var bitmapBatch = new BitmapBatch(PendingFrame, drawLayer + index, Materials.Bitmap[BlendModes.AlphaPremultiplied]))
            using (var e = layer.GetItemsFromBounds(Camera.Bounds))
            while (e.GetNext(out itemInfo)) {
                RenderTile(itemInfo.Item, bitmapBatch);
            }
        }

        public void RenderTile (RuntimeTile tile, Render.BitmapBatch batch) {
            var info = tile.TileInfo;

            var drawCall = new Render.BitmapDrawCall(info.Texture, tile.Bounds.TopLeft, info.Bounds);
            drawCall.Mirror(info.Strip.Mirroring.X, info.Strip.Mirroring.Y);

            batch.Add(drawCall);
        }

One thing you’ll notice is that RenderTile now has considerably fewer arguments. Since I had to reimplement SpriteBatch from scratch, this gave me the opportunity to integrate some of my rendering calculations, like zooming and viewport positioning, directly into the shader. As a result, I don’t need to pass those values around as parameters anymore; I simply set them as a shader parameter every time they change. Using an EffectPool for all my shaders also means that I only need to set those parameters once, instead of having to update all my individual shaders with the correct values.

One other difference here is the need to explicitly choose a layer for the tiles to render on when I create a BitmapBatch. I have a small set of constants (named DrawLayers) that I use to roughly organize my layers, but I also add values to those constants so that I can organize batches within those layers. That way, I can be certain that if I draw three sets of tiles on the same layer, they are always drawn in the same order relative to each other.

I also have to explicitly pass in a Material object for the BitmapBatch, instead of just passing a blend mode to the BeginSpriteBatch function. This isn’t a significant change, but it does mean that I now have to explicitly manage my materials – as a result, my game now has a LoadMaterials function that runs at startup and creates all the various permutations of parameters I need and stores them so that I can grab them at runtime.

You may also notice that there are no explicit drawing operations in here anywhere; I’m just creating a BitmapBatch and adding draw calls to it. Essentially, what’s going on here is that creating a batch automatically attaches it to the Frame being built. When the batch is Disposed (by the using block, in this case), the Frame is notified that the contents of the Batch (its draw calls) are ready and it stores them for later. The use of IDisposable to represent ‘readiness’ instead of disposal is a little weird, but it’s convenient in that it gives me relatively automatic batch management. This also means that if I wanted to, I could create lots of batches at once and fill them with draw calls on multiple threads, since the Frame has a straightforward way to determine whether all of its batches are ready yet.

One of the problems with using vertical sync in a video game is that it eats into the available CPU time for performing game updates. The way vsync is implemented in most graphics APIs, it causes your Present/EndDraw/SwapBuffers call to block until the card enters vertical blank and the frame is shown to the user. While this is ideal from a correctness perspective, it’s a tremendous waste since it means you can end up sitting there for up to 16 milliseconds, waiting for vertical blank. If your game spends lots of time doing both updating and drawing, all that time could be spent performing updates instead. Ouch.

Currently, my game spends about as much time drawing as it does updating. A significant portion of the time spent drawing (20-30%) is within the EndDraw function. Turning off vertical sync drops the amount of time spent in EndDraw considerably, but introduces tearing. So, as a potential solution, why not call EndDraw on a background thread? While the thread waits for vertical blank, I can begin performing the next frame’s Update, and in the event that I finish updating before the previous frame is visible, I simply wait for that previous EndDraw call before beginning to paint the next frame. In the optimal case, this means I can come much closer to the best possible framerate without introducing tearing, and in the worst case, the cost of rendering an individual frame is only *slightly* increased by the use of thread synchronization. The fact that I’m only doing EndDraw on another thread means that I don’t have to worry about protecting my game data with locks and other synchronization techniques, since the GraphicsDevice doesn’t use any of my game data when performing the EndDraw operation.

So, to test this out, I overrode my Game class’s BeginDraw and EndDraw methods. This turns out to be all we have to do to change the way drawing is performed, because the XNA Framework developers were kind enough to make both of these methods virtual.

        protected override bool BeginDraw () {
            _DrawCompleteEvent.WaitOne();
            _DrawCompleteEvent.Reset();
            return base.BeginDraw();
        }

        protected override void EndDraw () {
            _DrawRequiredEvent.Set();
        }

Of course, at this point, the two events used here are never set, so this code won’t work. Thus, we add a background thread to perform our painting:

        AutoResetEvent _DrawRequiredEvent = new AutoResetEvent(false);
        ManualResetEvent _DrawCompleteEvent = new ManualResetEvent(true);

        public Game () {
            ...

            _DrawThread = new Thread(DrawThreadFunc);
            _DrawThread.IsBackground = true;
            _DrawThread.Start();
        }

        protected void DrawThreadFunc () {
            while (true) {
                _DrawRequiredEvent.WaitOne();
                base.EndDraw();
                _DrawCompleteEvent.Set();
            }
        }

Fairly simple thread programming here: We create a thread, and set IsBackground to true so that it will stop as soon as the main thread exits. The thread spends all of its time waiting for a ‘required draw’ signal, and then performs an EndDraw call. Once the call is complete, it sets another signal to inform the Game class that the previous draw has finished and it’s safe to perform a BeginDraw call (this lets us make sure that we never use the GraphicsDevice on the main thread while the background thread is performing an EndDraw).

Once this is all done, I start up my game, and… the framerate isn’t any different. Huh? What’s more, my frame profiler indicates that Update is now taking ten times as long as it used to, while Draw isn’t any faster. Huh???

In situations like this, it’s always good to consult a profiler to see if you’re missing something important:

So, we can see that the DrawThread is definitely doing its job – it calls EndDraw, then waits for a signal asking it to draw again. Both are taking about as much time as we’d expect. But why is Update taking so long…?

… oh. Oops.

So it turns out that I was using GraphicsDevice.Viewport.Width and GraphicsDevice.Viewport.Height in my camera code. Accessing the Viewport property caused the XNA framework to call into Direct3D to retrieve the viewport, which acquired the exact same lock being used by EndDraw, causing my main thread to stall until the draw completed. WHOOPS.

This is especially embarassing since the viewport size never changes anyway, so I could have just stored the width/height into constants. After doing just that and starting the game again, the profile looks more like you’d expect:

What’s more, this is actually an improvement: With vertical sync enabled, this results in a significant reduction in the amount of time spent inside the BeginDraw/Draw/EndDraw functions on the main thread, which means there’s more time left to perform Updates. This means that I can maintain a solid, smooth 60fps easier on dual-core/hyperthreaded machines.

Even with vertical sync disabled, this is still an improvement, though not as significant – apparently other things are happening inside EndDraw (not a big surprise), so by shifting that work off onto a second thread, I’m still gaining some time to spend performing the next update. When I disable the built in framerate balancer, this brings my framerate from ~350fps up to ~380fps. Not bad for a couple dozen lines of code!

Of course, it’s worth pointing out that the XNA Framework documentation doesn’t make any promises here, so it’s possible that this technique is unsafe. When it comes to concurrency, it’s very easy to do the wrong thing and get away with it – as you might have noticed here, I was doing something utterly stupid and unsafe in my Update function, and I got away with it because the DirectX developers had the foresight to put a lock in the right place. If they hadn’t, my game might have corrupted state from accessing the GraphicsDevice on two threads, and crashed intermittently.

Regardless, this is a handy technique – once I’ve had the chance to do lots of testing on various PC configurations (and the XBox 360), I’ll probably be using it in my game when I ship.

The basic goal here is to be able to write game code in an imperative, sequential manner, without having to deal with locks, callbacks, polling, or race conditions. You rarely achieve perfection when dealing with concurrency, but even getting within sight of perfection can be worth the effort if it means you don’t have to spend time pulling your hair out trying to reproduce a rare threading bug.

In my game, any operation that can be performed off the main thread is done without blocking the main thread. Loading content, loading files, saving games, loading games, interacting with the guide, etc. This means that I can seamlessly load things in the background while the game is running without any noticeable stuttering. Normally, you’d need to do lots of juggling to handle this correctly – locking, threads, explicit synchronization – but thanks to cooperative threading, I don’t have to deal with any of that. The techniques I describe in this post are the same basic techniques I’m using for my game. Hopefully, that means they should work for yours too (though unfortunately that means if there are bugs in them, you’ve probably just found a bug in my game… whoops.)

Unless you’re working on an XNA game that uses networking (which sadly, I am not, so I can’t speak to the difficulties there), the biggest concurrency related issues you’re likely to deal with are twofold:

• Interacting with the XNA Guide APIs
• Performing I/O

By the end of this article I’ll have shown you how you can tackle these problems in your XNA games without having to deal directly with threads or synchronization, in a simple, imperative manner that works equally well on the PC and XBox 360.

The Guide APIs are almost all designed in a manner that requires use of either threads, callbacks/continuations, or polling, because they come as Begin/End pairs that require management of ordering and state beyond a single frame. This is for the most part a necessity, as detailed in the blog post I mentioned above. So instead of trying to find a workaround, the best course of action is to simplify the process of working with those APIs – if possible, without introducing new issues or restricting functionality.

I/O is a more interesting problem; even when writing desktop applications, you often perform large I/O operations that have the potential to block, and in those cases, a well-written application uses concurrency techniques – threading, callbacks, etc – to avoid ‘hanging’ and frustrating the user. However, on almost any modern desktop machine, you can typically get away with tiny I/O operations if done correctly – a good example is Firefox 3, which still regularly performs short disk operations on its UI thread, which can cause it to hang if your machine is under extreme I/O load, but works fine in almost every other situation.

The XBox 360 makes I/O particularly challenging because you can’t rely on latency and throughput characteristics that you might be used to on a desktop machine: Not only do you have to deal with the downsides inherent in hard disks, but you also have to deal with the possibility of a player using a memory card, and even worse, the possiblity of the available storage devices changing while your game is running (due to memory card insertion/removal, etc). This means that doing I/O in the main thread is pretty much a non-starter. You’re stuck: It’s concurrency time.

Luckily, cooperative threading provides a great solution for tackling both of these issues: It lets you reason about your problems in a manner you’re familiar with, and solve them by writing imperative code that still behaves correctly in tough situations, at the expense of some slight overhead and minor changes to your game code. You don’t have to build a tremendously complex state machine (since the C# compiler can be convinced to do the heavy lifting for us), and you don’t have to worry about locking and synchronization since all your code is guaranteed to run sequentially in the same thread, at the appropriate time.

So, for the initial scaffolding. As you might have guessed, we’ll be trying out the cooperative threading framework from my open source library, Squared.Task. This library is inspired by the techniques used in Twisted and imvu.task. The library is designed to perform well under tight constraints, so you can use it just fine on the XBox 360 without any garbage creation* or thread contention issues.

The first thing we’re going to need is a way to manage our cooperative threads. We’ll do this by creating a TaskScheduler owned by our Game object, that will run our ‘Tasks’ (cooperative threads) on the same thread as the rest of our game logic, and asking it to Step all pending tasks every time the Game’s Update method runs:

    public class TaskExampleGame : Microsoft.Xna.Framework.Game {
        Squared.Task.TaskScheduler taskScheduler;
        ...

        public TaskExampleGame () {
            ...
            taskScheduler = new Squared.Task.TaskScheduler();
        }

        ...

        protected override void Update (GameTime gameTime) {
            ...

            taskScheduler.Step();
        }

Pretty trivial so far. Creating a TaskScheduler is extremely cheap and quick, so you can do it right in your Game’s constructor; any particular expensive resources it requires are created on-demand. Calling the Step method every Update ensures that any waiting tasks are processed on a regular basis, as long as your game is updating at a solid framerate – which you always want it to be. If you don’t have any tasks pending, Step is effectively free, so when you’re not using the scheduler you don’t have to think about it.

Of course, this is still boilerplate. What we really want to do is actually get something done! So, let’s come up with a contrived example: We’ll make the A button increment an onscreen counter, the B button read the counter from a file on the player’s storage device, and the X button write the counter to that same file. This means we have to handle storage device selection, error conditions, and disk I/O – and we want to do it without causing any stuttering or hanging!

To begin with, we need the counter, and some basic scaffolding so that our example at least slightly resembles a well-behaved game – we’ll make sure to display the counter on screen so you can actually tell the example is working, and suspend our handling of user input if the Guide is open so that things don’t happen mysteriously in the background while you’re reading a message somebody sent you on MSN. I’m going to omit this scaffolding since you can see it in the source code (included at the bottom), and it’s tremendously boring.

So, now we can hold the A button to increase the counter, and it just keeps going up! Great fun, really – just add a rope to burn and we’re ready for an IGF submission. Anyway, let’s get to the meat of it: We want to write the counter out to a file when the player hits the X button. To begin with, let’s just spike a basic implementation:

        void SaveCounterToDiskViaCallbacks () {
            Guide.BeginShowStorageDeviceSelector((asyncResult) => {
                var storageDevice = Guide.EndShowStorageDeviceSelector(asyncResult);

                System.Threading.ThreadPool.QueueUserWorkItem((_) => {
                    var container = storageDevice.OpenContainer("TaskExample");

                    var file = System.IO.File.Open(
                        System.IO.Path.Combine(container.Path, "counter.bin"),
                        System.IO.FileMode.OpenOrCreate,
                        System.IO.FileAccess.Write,
                        System.IO.FileShare.None
                    );

                    byte[] bytes;
                    lock (CounterLock)
                        bytes = BitConverter.GetBytes(Counter);

                    file.Write(bytes, 0, bytes.Length);

                    file.Close();
                    container.Dispose();
                });
            }, null);
        }

Even this trivial example has lots of problems – not only am I completely ignoring error handling, but there are plenty of other issues that make this approach fail to scale up to large projects. At least it works, right?

Next, let’s spike a callback-based implementation of reading the counter:

        void LoadCounterFromDiskViaCallbacks () {
            Guide.BeginShowStorageDeviceSelector((asyncResult) => {
                var storageDevice = Guide.EndShowStorageDeviceSelector(asyncResult);

                System.Threading.ThreadPool.QueueUserWorkItem((_) => {
                    var container = storageDevice.OpenContainer("TaskExample");

                    var filePath = System.IO.Path.Combine(container.Path, "counter.bin");

                    if (!System.IO.File.Exists(filePath))
                        return;

                    var file = System.IO.File.Open(
                        filePath,
                        System.IO.FileMode.Open,
                        System.IO.FileAccess.Read,
                        System.IO.FileShare.None
                    );

                    var bytes = new byte[4];
                    file.Read(bytes, 0, 4);

                    int newCounter = BitConverter.ToInt32(bytes, 0);
                    lock (CounterLock)
                        Counter = newCounter;

                    file.Close();
                    container.Dispose();
                });
            }, null);
        }

If you look carefully, you might even see a bit of error handling in there. Now that we all know what a trivial stall-free implementation looks like, we can try out our sample and see that it’s fully functional. In most cases on a PC, it will work more or less perfectly, since I/O is lightning fast, despite the many bugs and errors experienced .NET programmers can find in it. One obvious problem is the fact that it basically does everything in one threadpool work item, because anything other than that is painful. This means that you have to put locks around basically any data and objects you use within the implementation, which gets complicated quickly. Even worse, being in the thread pool means that if we need to call a function that can only be run from the main thread, we have to marshal back and forth manually.

If the above spikes confused you, you can find explanatory comments in the downloadable source code. Let’s continue.

Now, to flex our mental muscles, let’s try and build a relatively ideal implementation of this example using our task scheduler. We’ll even handle errors and inform the user about them. As a result, you’ll see some code that mysteriously resembles scaffolding… Let’s get started.

To begin, we want to create tasks for our load and save operations. Let’s just write them, and worry about the functions they call later.

        IEnumerator<object> SaveCounterToDisk (int counterValue) {
                var storageDeviceFuture = SelectStorageDevice();
                yield return storageDeviceFuture;

                var storageContainerFuture = OpenStorageContainer(
                    storageDeviceFuture.Result,
                    "TaskExample"
                );
                yield return storageContainerFuture;

                using (var container = storageContainerFuture.Result) {

                    var filePath = System.IO.Path.Combine(container.Path, "counter.bin");

                    var fileFuture = GetFileAdapter(
                        filePath,
                        System.IO.FileMode.OpenOrCreate,
                        System.IO.FileAccess.Write
                    );

                    yield return fileFuture;

                    using (var file = fileFuture.Result) {

                        var bytes = BitConverter.GetBytes(counterValue);
                        var writeFuture = file.Write(bytes, 0, bytes.Length);

                        yield return writeFuture;
                    }
                }
            }
        }

So, let’s address the most confusing parts of this first before we continue on to the load function:

The function has to return an IEnumerator<object> so that the task system will recognize it as a task, and it can yield any waitable object out as a result. Unfortunately, we can’t use a strong type here because the compiler will only allow iterator functions to return IEnumerator or IEnumerable.

One of the patterns you see is retrieving a future from a function that begins an operation, and then yielding the future to wait for it to acquire a result. We don’t have to do these two operations in sequence, so if we wished, we could acquire futures for multiple operations in sequence before yielding on all of them at once – easy parallelization.

Once a future has a result, we’re free to retrieve it (using the Result property) or simply discard it. A future’s result will either be a value of the type you expect (the result of the operation), or an exception (caused by the operation failing). If the future’s result is an exception, attempting to retrieve it automatically causes it to be rethrown within a FutureException, preserving the original stack trace. This means that code using futures doesn’t ignore unexpected exceptions – it bubbles them up, just like single-threaded imperative code.

Another bonus here is that a discarded future is still monitored by the task system as a result of waiting on it – if you discard a future you’ve waited on, and the future contained an unhandled exception, it is automatically bubbled up through your task, causing resources to be cleaned up and allowing a higher up function or task to handle it. This means that futures with out a result we care about can simply be discarded without sacrificing correctness.

You may also wish to note that we pass the original value of the counter into the task to preserve it. This means that if the counter continues going up after the save begins, the original value will still be saved. No locking or state consistency issues to worry about; the compiler handles storing that argument for us while our task is running.

Most of the logic you’re familiar with from the original callback-based implementations has been replaced with simple functions that return futures. I’ll show those after the load function, which follows:

        IEnumerator<object> LoadCounterFromDisk () {
            var storageDeviceFuture = SelectStorageDevice();
            yield return storageDeviceFuture;

            var storageContainerFuture = OpenStorageContainer(
                storageDeviceFuture.Result,
                "TaskExample"
            );
            yield return storageContainerFuture;

            using (var container = storageContainerFuture.Result) {

                var filePath = System.IO.Path.Combine(container.Path, "counter.bin");

                var fileFuture = GetFileAdapter(
                    filePath,
                    System.IO.FileMode.Open,
                    System.IO.FileAccess.Read
                );

                yield return fileFuture;

                if (fileFuture.Failed)
                    yield break;

                using (var file = fileFuture.Result) {

                    var bytes = new byte[4];
                    var readFuture = file.Read(bytes, 0, bytes.Length);

                    yield return readFuture;

                    Counter = BitConverter.ToInt32(bytes, 0);
                }
            }
        }

Not much new here. One thing of note: If the GetFileAdapter call failed, we just abort the task immediately. The using blocks clean up all the resources we were using for us automatically, which means we don’t attempt to read from a nonexistent file. If we wanted, we could show an error dialog to the player here. In contrast to this, we don’t bother checking the number of bytes that were read, even though readFuture contains them. If I were a serious game developer instead of a blogger writing an example, I’d check the result and show an error if it’s anything other than four bytes (or perhaps just throw an exception).

Hopefully you’re all following me so far. For completeness, I’ll now show the wrapper functions used above:

        Squared.Task.Future<StorageDevice> SelectStorageDevice () {
            var f = new Squared.Task.Future<StorageDevice>();

            Guide.BeginShowStorageDeviceSelector((asyncResult) => {
                try {
                    var result = Guide.EndShowStorageDeviceSelector(asyncResult);
                    f.SetResult(result, null);
                } catch (Exception ex) {
                    f.SetResult(null, ex);
                }
            }, null);

            return f;
        }

        Squared.Task.Future<StorageContainer> OpenStorageContainer (StorageDevice device, string titleName) {
            return Squared.Task.Future.RunInThread(
                () => device.OpenContainer(titleName)
            );
        }

        Squared.Task.Future<Squared.Task.IO.FileDataAdapter> GetFileAdapter (string fileName, System.IO.FileMode mode, System.IO.FileAccess access) {
            return Squared.Task.Future.RunInThread(
                () => new Squared.Task.IO.FileDataAdapter(
                    fileName,
                    mode,
                    access,
                    System.IO.FileShare.None
                )
            );
        }

These wrappers are no more complicated than the original imperative code, with the exception of the try block in the Guide wrapper (necessary to propagate any errors out through the Future).

One advantage you might not notice is that since the Guide wrapper issues the original Begin call directly, it will bubble right up into the caller without any delay, allowing you to handle it immediately without waiting, while errors that occur later on will be stored into the Future and bubbled up into the caller.

We invoke OpenContainer on a thread to avoid it blocking the main thread. Those of you who are sticklers for correctness may have noticed that this isn’t 100% correct, since there’s no guarantee that this function is thread-safe. We’re relatively okay since we’re certain it won’t be used on any other threads at the same time, thanks to the ordering guarantees provided by the task system, but there’s still a slight potential for badness here. Caveat emptor. If you feel nervous, just call it directly from your task instead – no thread safety issues there.

One somewhat major difference is that I’m now constructing a FileDataAdapter object instead of opening a file using the standard System API. A FileDataAdapter is a utility class provided by my framework that simplifies the task of performing asynchronous I/O at a low level. It implements a generic interface that allows manipulating other data sources, like sockets and streams, in the same manner. For those seeking a higher-level interface, there are utility classes for performing WriteLine, ReadLine, and similar operations on an adapter – without needing a thread! Just because you want to write concurrent code doesn’t mean you have to suffer.

To run these tasks, the code you write isn’t much more complicated than calling the original functions:

    taskScheduler.Start(
        SaveCounterToDisk(Counter),
        Squared.Task.TaskExecutionPolicy.RunAsBackgroundTask
    );

The reason for the ‘RunAsBackgroundTask’ argument is non-obvious, so I’ll explain: One of the features of the task system is lifetime management. Starting a task returns a Future that will be completed when the given task is done running. If you wish, you can Dispose the future, and the task will be cancelled. RunAsBackgroundTask is a special execution policy that tells the TaskScheduler that not only do you have no desire to manage the lifetime of the task, but you want it to automatically handle any errors that occur in the task.

Of course, the default error handler for background tasks knows nothing about XNA. So let’s write one that does, and use the Guide to show the error message to the user:

        public TaskExampleGame () {
            ...
            taskScheduler.ErrorHandler = (error) => {
                taskScheduler.Start(HandleTaskError(error));
                return true;
            };
        }

        IEnumerator<object> HandleTaskError (Exception error) {
            while (error.InnerException != null)
                error = error.InnerException;

            string message = error.Message;

            if (message.Length > 255)
                message = message.Substring(0, 255);

            while (Guide.IsVisible)
                yield return new Squared.Task.WaitForNextStep();

            var f = new Squared.Task.SignalFuture();
            Guide.BeginShowMessageBox(
                "An unexpected error occurred",
                message,
                new string[] { "Okay  " },
                0, MessageBoxIcon.Error,
                (asyncResult) => {
                    f.SetResult(null, null);
                }, null
            );

            yield return f;
        }

A fairly simple error handler that schedules a new task to show a message box to alert the user about the error. You may notice I apply a simple polling technique here to avoid the need for any complex queueing or locking – if the error handler task discovers the guide is running, it simply suspends itself until the next step, allowing the guide to close. Obviously, you don’t want to be doing this in thousands of tasks every step if you want to scale, but for a task like this, it’s more than adequate.

You’ll notice that most of the trivial error cases work perfectly in this case. I even left one obvious case unhandled so you could easily trigger a failure – just cancel the storage device selection screen (though you’ll only see it if you have more than one), and you’ll see ‘NullReferenceException’ pop up. What about opening the guide while selecting a storage device? Handled just fine.

Now that we’ve got our example up and running, let’s add a little polish. We’ll add a helpful message to explain how the example works, and put a bouncing smiley face on the screen that moves smoothly based on the current time, so we can see if the game hangs by watching the smiley. We’ll also add a simple status display so that we can see what the example is doing, and avoid queueing up dozens of copies of an operation all stalled waiting for the same resource (like a storage device). I’ll omit the code for these changes since it’s mostly uninteresting, but as before, you can see it in the downloadable source code.

Now that we’ve walked through the process of building this example (and gotten tremendously bored), here’s the source code and binaries so you can try it out on your Windows PC or XBox 360:

http://luminance.org/xna/TaskExampleBinaries.zip

http://luminance.org/xna/TaskExampleSource.zip

You’ll need XNA Game Studio 3.1 installed, of course. You’ll need the latest version of my TaskLib and Util libraries, along with the XBox 360 versions of them (mostly identical), available on Google Code:

http://code.google.com/p/fracture/source/browse/#svn/trunk/Squared

The XBox 360 versions live in the ’360′ folder while the Windows versions live in their own named folders. You can either build them and add them as references, or add the projects themselves to the example as references. For my game, I’ve simply checked out the project as a subversion external so that it’s easy to keep it up to date, but my game doesn’t suddenly break if it changes.

I hope you’ve found this useful. This example, like all the source code for my libraries, is open source. I encourage you to share it, modify it, experiment, and maybe even use it in your own games. Please don’t hesitate to email me or leave a comment if you have questions, suggestions, or think you’ve found a mistake.

The main change I made was to one of the core classes I use for concurrency: Future. The future class is a container that can store a single value, and can only be written to once. You can attach one or more listeners to it that will get invoked once it has a value. The primary advantage of using this class is that it lets you easily perform asynchronous operations without having to deal with locking and most types of race conditions, due to the fact that it’s write-once.

Having a type like Future is extremely useful when dealing with things like disk IO or network operations. But it also turns out to be useful in games – you can use Futures to represent long-lived operations like getting a user’s selection from a menu, finding a path through a level, or loading a texture from disk. Future is also the foundation of the Task primitive in my cooperative multithreading library.

The original interface for the Future class didn’t make use of .NET generics. This meant that it was easy to write general code atop futures – you could write a simple function that would take a list of futures as inputs and return a single future as a result that would become completed once all the inputs were completed, for example. The downside is that you basically didn’t get any type safety, and values were always boxed – a future that contained an integer had to store it as an object, and any time you wanted to retrieve a value from a future, you had to cast it.

This week I decided to try and refactor my code such that Future could utilize .NET generics but still be usable with simple utility functions.

The solution was to create a baseline interface that all Futures implement, and write all the utility functions and libraries using that baseline interface, but expose fully typed generic Futures as return values from functions. Here’s a before and after example:

        public static Future WaitForAll (params Future[] futures) {
            return WaitForX(futures, 1);
        }

        public static Future RunInThread (Func<object> workItem) {
            var thunk = new FuncRunInThreadThunk {
                WorkItem = workItem,
            };
            ThreadPool.QueueUserWorkItem(FutureHelpers.RunInThreadHelper, thunk);
            return thunk.Future;
        }

As you can see, the original code lacks type information, so you have to infer it from usage and do casting everywhere. The upside is that it’s general, but you end up paying for it when you make mistakes about types and have to add explicit casts everywhere.

        public static IFuture WaitForAll (params IFuture[] futures) {
            return WaitForX(futures, 1);
        }

        public static Future<U> RunInThread<U> (Func<U> workItem) {
            var thunk = new FuncRunInThreadThunk<U> {
                WorkItem = workItem,
            };
            ThreadPool.QueueUserWorkItem(FutureHelpers.RunInThreadHelper, thunk);
            return thunk.Future;
        }

In the refactored code, the type information is preserved when going through a utility function like RunInThread – if the function being run in a thread returns a float, the resulting future will also contain a float. This eliminates casting but still preserves generality – since a Future<float> implements IFuture just like a Future<int>, you can use that underlying type when writing utility functions, like WaitForAll (though one downside is that WaitForAll is forced to return an IFuture instead of a future of a specific type, since there’s no easy way to guarantee that all the futures are of a given type.)

Performing this refactoring took lots of individual steps – first, I had to change various code to use IFuture instead of Future, and make minor tweaks in order to keep things functioning and keep tests passing. In some cases I still needed to use Future (one obvious example – you can’t construct an interface, so in that case the real type needed to be used). The largest chunk of work there was updating the handlers for futures – previously, a future listener looked like this:

var handler = (OnComplete) (future, result, error) => { Console.WriteLine("Future {0} completed with result {1}/error {2}", future, result, error); };

The three arguments were mostly an artifact from previous refactorings – originally the Future argument wasn’t there at all. The existence of the result argument was a problem, because it had to be the same type as the future – a Future<float> needed an OnComplete listener that took a float result, which made it difficult to construct generic listeners.

In order to make things simpler, I removed the result and error parameters. Including the future still made it possible to get at the result and error, but allowed the listener to be general by accepting an IFuture instead of a strictly-typed Future<T>. This was important for being able to wire futures up to each other (The WaitForAll function is a good example of this, in that it connects multiple futures to a single result future.)

Once I had updated all the existing listener code, the final step was to add a type argument to Future, and add type information to various library functions.

Here’s an example of the results – if you were writing a Task to do asynchronous reads from a socket or a file, one line at a time, the code to do the read might look like this:

Future f = input.ReadLine();
yield return f;
string line = (f.Result as string);

The ReadLine method of the input stream returns a Future, which will be completed with a string when a line has been successfully read from the stream. The Task yields on the Future to suspend execution until it’s been completed with a result. Once it’s completed, execution resumes, and we pull the resulting line out of the future. It’s obvious that there are some problems with this approach. With the new generic type, the cast is eliminated and ReadLine has a clear return type:

Future<string> f = input.ReadLine();
yield return f;
string line = f.Result;

And if you put C# 3.0′s language features to work, the end result is almost free of duplication:

var f = input.ReadLine();
yield return f;
var line = f.Result;

This is one example where having bidirectional iterators (like in Python) would be beneficial. Python’s yield expression has a result, so you can ‘send’ an input into an iterator every time you advance it. As a result, this code written using a library like imvu.task would look like so:

line = yield input.readLine()

The advantage is pretty obvious – the entire operation is in one line and the meaning is more clear.

Still, this solution is pretty good for C#. You get most of the advantages you could want: Simple code, type safety, thread safety, rock-solid error handling, and scalability. (One of the advantages of using the Future type is that it ensures that you always get either a result or an error. This means that any code that uses a Future to retrieve a result will have exceptions rethrown when an asynchronous operation fails – no need to manage error flags and add lots of try/catch blocks.)

While doing all this refactoring, my large set of unit tests came in handy – they caught a number of bugs I introduced early, and revealed an existing bug that was months old. As a result, I ended up with better error handling, better type safety, and a more robust set of unit tests. On the other hand, I still don’t have any unit tests for my platformer, so maybe I should take another shot at figuring out how to write some for it. (:

If you’re interested in seeing how Future and Task work, you can see the source code and some working examples here:

http://code.google.com/p/fracture/source/browse/#svn/trunk/Squared/TaskLib

I encourage you to check out the repository and try the MUD Server and Telnet Chat examples. They’re not real-world applications, but they’re easy to understand and demonstrate how using Futures and Tasks simplifies concurrency. The MUD even has gameplay!