During this past week, one of the problems I tackled was finding a way to deploy builds of my game. When dealing with deployment, an important principle is that the process should be as close to a single click as possible – a complex deployment process discourages you from getting customer feedback, and increases the likelihood that you’ll make a mistake and end up with a failed deployment, which wastes valuable time.

For most of my previous projects, I’ve tended to take either a hands-on approach to deployment, or a completely hands-off one, either building an installer by hand using tools like NSIS, or deploying the project as a ZIP file full of binaries and expecting the end user to install the necessary dependencies and figure out how to get the program running. Neither extreme is ideal, really (though hands-off deployment can be amazing if you manage to set things up such that all the end user has to do is run your game – no install necessary).

I spent a little while considering options like WiX for building my own installer and deployment process, but quickly realized I could spend entire weeks trying to solve that problem and not be done. With my schedule I can’t really afford to waste time like that, so I had to go for something simpler. On the other hand, hands-off deployment isn’t an option here either – XNA games have enough dependencies that it’s nontrivial for a casual gamer or newbie computer user to get them working on a machine.

As it turns out, Microsoft’s ClickOnce deployment tool is extremely easy to use with modern versions of the XNA framework. For a simple game, you really don’t need to do anything other than build your project and click Publish – it spits out all the files necessary to install your game, and you can upload them somewhere on the internet. It even has decent support for automatic updates.

Unfortunately, ‘magic’ deployment systems like this only work well if they have all the information they need to deploy your game. This means that your project file needs to accurately represent everything your game needs to run – libraries it uses, content it loads at runtime, etc. This ended up presenting the biggest challenge for me as far as getting ClickOnce deployment to work – I had to convince it to deploy the files my game needed, which meant including all my game assets into my project.

At first I considered some simple solutions to the problem – using post build steps to copy all my assets, or adding all of them to the content project by hand – But eventually I realized the right choice was to integrate my game with the content pipeline so that ClickOnce would know exactly where to find all my assets. As it happens, XNA 3.1 added some useful features that made this process considerably easier.

I’ve previously shared some basic details about my approach to serialization. Essentially, all the custom content in my game is described as simple graphs of objects, where each object in the graph is serialized by Microsoft’s built in XMLSerializer and the graph as a whole is serialized by my storage framework. This gives me a nice balance between the ridiculous simplicity of XMLSerializer (for simple data types, you basically just point it at them and you’re done writing your serialization code) and the flexibility provided by doing your own serialization.

Before XNA 3.1, to integrate with the content pipeline, I would have had to write four classes – an importer to read a tileset object from my XML graph format, a processor to convert it into a ‘processed’ tileset blob, containing all the appropriate tile textures and information, a writer to convert that blob into the XNB format, and a reader to convert the XNB format back into a TileSet object at runtime. A lot of work, to be sure, and typically a lot of duplication as well. XNA 3.1 simplifies things by letting you omit the Reader and Writer classes – assuming you design your data types correctly, you can get extremely efficient reading and writing for free with the addition of a few attributes. Knowing that, the only classes I now need to write are the importer and the processor.

Some aggressive refactoring lets us simplify things even further: Since I know all my custom files are in my XML graph format, I can use the same importer for all of them, and implement all the unique code inside the processors. This means that I write a simple ‘raw XML’ importer once, and then write a processor for each unique data type. Now the problem is extremely manageable: A little duplication (data type + processor), but hardly anything to get upset about.

However, one problem still remains: I’ve already got code for reading and writing these graphs from XML, and code for converting those graphs into the data structures I need at runtime. If I end up having to reimplement all that behavior for the content pipeline, I’m potentially wasting a lot of effort. Once again, however, refactoring comes to the rescue: By carefully moving around classes and dependencies, I was able to reuse the same serialization and initialization code in the content pipeline and in the game engine.

Moving things around in this way meant that I could make these changes incrementally, as well – instead of spending days with my game engine in a broken state, trying to bend my data to fit the content pipeline, I could swap out one bit of code at a time, since everything was built on the same fundamental infrastructure. So, I started by getting my Tileset format to properly build in the content pipeline, then changed the game engine to load the content pipeline’s version of the tileset instead of the source XML files. Once that was working, I did the same thing with the Sprite format. After that was done, I was free to make more aggressive changes, like enhancing my content pipeline processors to build all the necessary textures for a tileset or sprite into a single file for more efficient loading, or doing things like premultiplying my tiles during the tileset build process.

Since I’ve rambled on for a while now, I’ll just share some code so you can see how various pieces of the pipeline look now. If you have any questions, feel free to ask and I can elaborate more on how pieces of this work.

First, the raw file importer. This is a really trivial content pipeline class that handles turning one of my data files into a content object that can be handed to one of my content processors. The key here is that this makes things general so that I can reuse as much serialization/processing code as possible between different formats.

    [ContentImporter(
        ".*",
        DisplayName="Raw File Importer"
    )]
    public class RawFileImporter : ContentImporter<RawFile> {
        public override RawFile Import (string filename, ContentImporterContext context) {
            filename = Path.GetFullPath(filename);
            using (var f = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) {
                var result = new byte[f.Length];
                f.Read(result, 0, (int)f.Length);
                return new RawFile { Filename = filename, Data = result };
            }
        }
    }

As you can see, it’s really simple: Read the contents of the file being built by the content pipeline and turn that into an object that can be manipulated by a content processor. If I wanted to, I could even do XML parsing and graph loading here to take advantage of the similarities between my file formats.

The next piece of the puzzle is the content processor. For example, here’s the TileSet processor. It takes a raw file as input and spits out a processed version of my TileSet object that can be loaded at runtime.

    [ContentProcessor(DisplayName = "TileSet Processor")]
    public class TileSetProcessor : ContentProcessor<RawFile, TileSet> {
        public override TileSet Process (RawFile input, ContentProcessorContext context) {
            var contentRoot = context.GetContentRoot();
            var resolver = new AssemblyTypeResolver(__LabyrinthModel__.Assembly);
            var textureParameters = new OpaqueDataDictionary {
                { "ColorKeyEnabled", false },
                { "GenerateMipmaps", false },
                { "ResizeToPowerOfTwo", false },
                { "TextureFormat", TextureProcessorOutputFormat.Color }
            };

            TileSet tileset;
            using (var stream = new MemoryStream(input.Data)) {
                var serializer = new XmlNodeSerializer { StreamOpener = (fn) => stream, TypeResolver = resolver };
                var f = serializer.LoadNode<TileSet>(null);
                f.GetCompletionEvent().WaitOne();
                tileset = f.Result;
            }

            var textures = tileset.TileStrips.Select((ts) => ts.Filename).Distinct().ToArray();

            foreach (var texture in textures) {
                var texturePath = Path.Combine(tileset.TilePath, texture);

                var itemPath = Path.Combine(contentRoot, texturePath);

                var sourceRef = new ExternalReference<TextureContent>(itemPath);
                var builtRef = context.BuildAndLoadAsset<TextureContent, TextureContent>(
                    sourceRef, "PremultipliedTextureProcessor", textureParameters, null
                );

                foreach (var ts in tileset.TileStrips) {
                    if (ts.Filename != texture)
                        continue;

                    ts.Filename = null;
                    ts.Texture = builtRef;
                }
            }

            tileset.TilePath = null;

            return tileset;
        }
    }

There’s a lot going on in this class, so I’ll go over the important bits one at a time:

First, you’ll see I do some basic initialization before I start processing the content – setting up parameters for the texture processor and creating objects needed by the graph serialization code. Next, I take the raw file data and use the graph loader to convert it into the type I care about – in this case, TileSet. At this point, the TileSet object only represents the contents of the XML file; not the complete asset. Since this is the same serialization code I use at runtime, it runs in a background thread and I have to wait for it to complete before continuing.

Next, I walk over all the TileStrips defined in the file to build a list of unique texture files that this TileSet depends on. LINQ is handy for this sort of thing, and since our code is running in the content pipeline, we don’t have to worry about garbage creation like we do on the 360.

For each texture, we determine its fully qualified filename and use that to create an ExternalReference to it. Once we have that reference, we can pass it back to the content pipeline and request that the texture be built and loaded for us so that we can embed it into the result TileSet object. We pass in the parameters we configured earlier, along with the name of the content processor to use for the texture (in this case, it’s “PremultipliedTextureProcessor”, a special texture processor I wrote that premultiplies textures after running them through the standard content processor).

Once we have a built version of a given texture, we walk through all the TileStrips defined in the TileSet, and if they depend on that texture, we null out their Filename field and store our built texture into their Texture field. The important detail here is that while the TileStrip object has both of these fields, each specific file format only uses one of them – this is done using the appropriate attributes, like XmlIgnore and ContentSerializerIgnore. This sort of technique can be dangerous, since it makes your code harder to understand, but in this case, it allows us to get away with one type where we’d normally need two similar, but distinct types.

Finally, we null out the TileSet’s TilePath field, as a way of indicating that the tileset’s tiles do not need to be loaded from disk – it’s fully constructed. At this point, we return our TileSet object to the content pipeline, because it’s ready to be stored out to disk. Now the content pipeline takes over, and its automatic serialization code converts our TileSet to XNB and then back to TileSet at runtime.

The next step to making this work is pulling out the important parts of my object model into their own library, so that both the game itself and the content pipeline plugin can access them. As described in some of Shawn Hargreaves’ blog posts, getting the content pipeline right can be difficult. Moving classes like TileSet and Sprite into their own assembly means that both the Game and the Content Pipeline can access those types without needing to depend directly on each other – the game depending on the content pipeline would mean no XBox deployment, since the content pipeline doesn’t work on the XBox 360, and the content pipeline depending on the game would make building the game impossible, since there’s now a circular dependency between the game’s assets and the game itself.

Making those types loadable from both XML and XNB formats with the same code required altering them to implement a common interface with a few important methods:

        public void Initialize (string contentRoot) {
            if (TilePath != null) {
                foreach (var ts in TileStrips)
                    ts.Filename = Path.Combine(TilePath, ts.Filename);
            }
        }

        public IEnumerator<object> LoadContent (IContentLoader contentLoader) {
            var tiles = new Dictionary<string, TileInfo>();
            Texture2D texture;

            foreach (var tilestrip in TileStrips) {
                texture = tilestrip.Texture as Texture2D;

                if (texture == null) {
                    var f = contentLoader.LoadContent<Texture2D>(tilestrip.Filename);
                    yield return f;
                    texture = f.Result;
                }

                tilestrip.GenerateTiles(tiles, texture);
            }

            Tiles = tiles;
        }

In this example, you can see that the Initialize method takes the location of the game content as an argument, and uses it to convert any filenames from the tileset into fully-qualified paths. When loading from XNB, these filenames have already been cleared out by the content processor, so this method does nothing. When loading from XML, the filenames are populated from the raw XML, so converting them into fully-qualified paths means that we know the exact location of each texture on disk.

In comparison, the LoadContent method walks over all the TileStrips in the TileSet, and if they don’t already have a texture, it uses the provided ContentLoader to read their texture from disk. Once they have a texture, it generates the tile info from the TileStrip and the texture so that the game’s renderer has a list of individual tiles to use. Since the content processor initializes the Texture field, this function doesn’t have any content loading to do when working from XNB. When loading from XML, the Texture fields are all null, but (thanks to the Initialize function), each TileStrip has a fully-qualified path pointing to a texture. We can use the provided ContentLoader to read the texture from disk and populate the fields, so everything works as it would have if we were loading from XNB.

Using a custom interface for reading content here gives us the advantage of being able to reproduce the content pipeline behavior we care about (like premultiplying textures) without needing to drag in the entire content pipeline and build XNBs at runtime. We can provide a simple wrapper over the standard ContentManager class that knows how to premultiply textures after loading them and then hand them back to the LoadContent method.

The final step is to put this all together and use it to load both XNB and XML versions of our content at runtime. There’s a little duplication involved, but it’s not bad:

        protected IEnumerator<object> LoadContentFromDirectory<T> (string directory, string filter)
            where T : IContentNode {

            var result = new Dictionary<string, T>();
            var filenames = Directory.GetFiles(directory, filter);
            var loader = new XmlNodeSerializer {
                StreamOpener = (fn) => File.OpenRead(fn),
                TypeResolver = Game.TypeResolver
            };

            foreach (var filename in filenames) {
                var f = loader.LoadNode<T>(filename);
                yield return f;

                var item = f.Result;
                item.Initialize(GetContentPath());

                yield return item.LoadContent(TextureAndContentLoader);

                result.Add(item.Name, item);
            }

            yield return new Result(result);
        }

        protected IEnumerator<object> LoadCompiledContentFromDirectory<T> (string directory)
            where T : IContentNode {

            var result = new Dictionary<string, T>();
            var filenames = Directory.GetFiles(directory, "*.xnb");

            foreach (var filename in filenames) {
                var f = ContentLoader.LoadContent<T>(
                    Path.Combine(Path.GetDirectoryName(filename), Path.GetFileNameWithoutExtension(filename))
                );
                yield return f;

                var item = f.Result;
                item.Initialize(Content.RootDirectory);

                yield return item.LoadContent(ContentLoader);

                result.Add(item.Name, item);
            }

            yield return new Result(result);
        }

        protected IEnumerator<object> LoadTilesets (bool compiled) {
            Future<Dictionary<string, TileSet>> f;

            if (compiled)
                yield return LoadCompiledContentFromDirectory<TileSet>(Path.GetFullPath(Content.RootDirectory) + @"\tilesets\").Run(out f);
            else
                yield return LoadContentFromDirectory<TileSet>(GetContentPath() + @"\tilesets\", "*.tileset").Run(out f);

            Tilesets = f.Result;
        }

The IContentNode interface is the simple interface we defined that contains the Initialize and LoadContent methods, so that we can read and initialize our data without needing to know its specific type. The two different Load…ContentFromDirectory functions contain the necessary logic for finding all the content of a given type and loading it using the appropriate serialization code (for compiled content, Content.Load<T>(“foo”), for XML content, XmlNodeSerializer.LoadNode<T>(“foo.xml”).) Once I’ve loaded all the objects, I can walk over the collection and initialize them, and then wait for them to load any required content. You may notice that the loader for XML content uses a special ‘TextureAndContentLoader’ object that knows how to read raw PNG textures in addition to XNB textures, where the XNB loader only passes in a ContentLoader that can read XNBs.

Given those two functions, I can write a simple function like LoadTilesets that works correctly for loading both compiled content pipeline files and raw XML content files. So when first loading my game, I can load my content like this:

                yield return LoadTilesets(true);
                yield return LoadSprites(true);

And then at runtime if the user presses the hotkey to reload content, I just call the functions again, passing false to load the content from XML:

                yield return LoadTilesets(false);
                yield return LoadSprites(false);