Let’s compare two automation tools for Unity game projects: Game.ci and Codemagic!
This article was first published on Codemagic’s blog.
I’ve already had the chance to talk quite a lot about CI/CD and why it’s interesting for all kinds of software projects, including video games. As I also mentioned earlier this year, these tools are sadly not that widely adopted among game developers…
… but this trend is catching on, and we are now starting to see some video game-dedicated CI/CD tools!
In particular, for developers who, like me, want to automate their Unity project, there’s a special solution called Game.ci. Ever heard of it before? I decided to explore it, and wanted to share my findings.
So, let’s learn more! In this article I’ll:
- Talk about what is Game.ci
- Detail the shared features and differences between Game.ci and Codemagic
- Run through Game.ci’s Gitlab example project and compare this process with a comparable Codemagic setup
- Try to run Game.ci on top of Codemagic and see if the two solutions can (and should) be combined
What is Game.ci?
An overview of Game.ci
Game.ci is an open-source collaborative project that started back in 2020, and aims at preparing a toolbox to ease the build and testing of your game projects. For now, it only supports Unity, but it could eventually grow into something more.
The big thing is that it’s fully community-based, meaning that the original authors of the project, Gabriel Le Breton and Dick Webbink, aren’t the only one to code and contribute features to Game.ci. Rather, everyone can enter the Discord, join the “team” and offer improvements, fixes or new functionalities for the tool. This quickly translates to the fact that the tool is completely free, but also fairly subject to changes in the future…
That is what you can find out from Game.ci’s website. However, it’s a bit unclear what exactly is Game.ci? It sounds like it’s a continuous integration platform, but in fact it’s not.
To put it short, Game.ci is a collection of dockerized versions of Unity. You can spin up a Game.ci Docker container on pretty much any CI/CD platform that supports Docker, and build your Unity project with it. Obviously, Game.ci has documentation on how to do that, and so far it documents the process for the two popular CI/CD tools – GitLab CI and GitHub Actions.
How do Game.ci and Codemagic compare?
Given that Game.ci is not actually a continuous integration and delivery platform, it’s quite hard to compare it with Codemagic. But we’ll try either way. First things first, it’s important to understand that Game.ci doesn’t really cover the same steps of an automation pipeline as Codemagic. As stated in their docs, Game.ci is a CI (Continuous Integration) tool; but it doesn’t handle the deployment part!
In other words, you can use Game.ci to build your game, and optionally run some tests on it, but it is by no means supposed to (or capable of, at the moment) deploying your game to Google Play, the App Store, a Firebase App, etc.
In short: the sole goal of Game.ci is the building and testing part – there is no CD (Continuous Delivery) here.
However, Game.ci provides some documentation on using fastlane, a known publishing tool, together with it to compensate for the lack of continuous development functionality.
Codemagic, on the other hand, can handle both the building/testing phase and the deployment to various well-known and commonly used market places or distribution services: it is a full CI/CD automation tool.
Ok – so what about the build phase? Now, because Game.ci is solely focused on Unity game projects, it can “afford” to dedicate more resources to them – in particular, the tool offers all current (modern) versions of Unity via Docker images. So you can simply get one of their Docker images, and you’ll instantly have a computer with a given OS and a given Unity version on it.
Codemagic is more diverse and it can be used for non-Unity projects, too. So, it doesn’t have that many Unity versions readily available. (At the time of writing Codemagic build machines have either a 2020 LTS or a 2021 LTS version). However, you can install a different version of Unity using Unity Hub CLI, and Codemagic has this process fully documented.
From that point of view, we could say that Game.ci has a slight advantage since it offers more versions out-of-the-box.
And the testing? Here, let’s be honest, the two tools are very similar: it’s up to you to prepare your unit tests beforehand in the Unity project and run them from the shell with the right command! Game.ci’s example Gitlab project shows us how to run a
test.sh script during the CI, which is about the same as the technique I used to run unit tests in my Unity project with Codemagic (just with a bit more options).
What about the ease-of-use? To me, that’s the real tie-breaker. Automation is supposed to make your life easier – I’m OK with spending a bit of time learning the basics, but I prefer the docs and overall usage would be clear and centralised… which I feel is not yet perfect for Game.ci!
When you work with Codemagic, all you need to provide is your code versioned somewhere on a Git provider (Github, Gitlab, Bitbucket), with a script to build the project from the command line and the
codemagic.yaml config file. Yes, you have to link this repo to your Codemagic app, but then everything happens in the Codemagic dashboard – there, you’ll even be able to manage your team permissions and global environment variables.
Game.ci, on the other hand, is more of a DIY solution. To be clear: their Docker images are great, it’s very cool to know that you can easily download a machine config that’s all set to run your Unity version 🙂
But once you have this image, it’s still up to you to setup all your GitLab or GitHub CI/CD (using the basic
.gitlab-ci.yml or GitHub Actions features from your Git provider), you have to add the same type of build/config files as with Codemagic, and if you want the deploy part, you’ll have to handle the install of the necessary tools in your instance of the Docker during your pipeline.
This is interesting to improve your DevOps skills and learn more about the various CI/CD tools that exist nowadays, but it means that Game.ci is essentially a (Docker image + sample projects) provider. Unlike Codemagic, Game.ci doesn’t have it’s own runners, it requires some hardware to spin those Docker images on.
That’s a very valuable thing, but it’s clearly not as “all-inclusive” as Codemagic, and it doesn’t work “out of the box”!
Note: incidentally, that’s also why Game.ci is free – apart from building Docker images and hosting them on the public Docker hub (which is free for small projects), they don’t need to manage any actual hardware or API connections, they leverage the ready-built features like GitHub Actions or Gitlab CI runners.
Just to wrap up this quick comparison, I’d like to point a few things:
- by default, because Game.ci relies on the GitLab CI/GitHub Actions free runners, it has the same limitations as they do – meaning that if everyone is using them at the same time as you, you might need to wait for a while before you get your build!
- on the other hand, since Game.ci is based on the Git providers, it also shares some of their advantages… such as the on-premise/personal instances, that allow you to keep your code inside your enterprise! This can be important to avoid security issues, and some companies might prefer this “customisable” solution instead of a cloud-based one like Codemagic.
- if you look at Codemagic’s docs on building/deploying a Unity app, you’ll see that it requires a Pro or Plus license, whereas Game.ci also allows for a Personal Unity license. Again, that’s because Game.ci is more of a “CI kit”, and so it simply lets you handle the registration step with Unity, and then expects a pre-activated Personal license in its pipeline – this is not actually “included” in the CI features.
But still, if you want to automatically request a Personal license activation, you can get a nice script from Game.ci to do it over here! 😉
A quick example with Game.ci
To begin with, let’s test Game.ci and see how to set up a basic CI pipeline with this tool. In this section and in the rest of the tutorial, I’ll work with a basic sample Unity project I prepared and published publicly on GitLab 🚀
As we can see in Game.ci’s GitLab demo project, to use this CI, we need to add three things:
.gitlab-ci.ymlthat describes the CI stages, environment vars and other config data (placed at the root of the repository)
- a C# build script so that the project can be built via the command line (in a
- various shell scripts to get the Unity’s license activation file, prepare a CI stage environment or build the project (in a
Check out the GitLab project to see what these files contain – they’re mostly stuff I copy-pasted from the Game.ci demo project, except for the
.gitlab-ci.yml that I re-adapted and simplified to suit my sample 🙂
With this CI setup, each commit will trigger a new GitLab CI pipeline in my repo. However, since I’m using my personal Unity license, I first need to activate my license and add it as an environment variable in my Gitlab project, as explained in the Game.ci docs.
This can be done by running a first pipeline that will stop after producing a license request file (this file can be downloaded in the artifacts, with the 3-dots menu on the right):
Then, once the
UNITY_LICENSE var is defined (follow the Game.ci instructions to get this value), the rest of the pipeline will run to build the project and produce a Mac app.
We see that the first pipeline (to activate the license) takes about five minutes, while the second one (to actually build the project) takes six minutes.
If you take a detailed look at the pipeline steps length, you’ll notice that the Docker initialisation is quicker after the first time. That’s because, thankfully, GitLab has a caching system that cleverly “stores” the Docker image for later re-use.
This means that, as long as I use the same Docker image (i.e., in my case, the same OS and Unity version), I’ll be able to “skip” some of the setup time and directly go to the build step, which is fairly quick for this small project.
Of course, the same thing happens for the build target-specific Docker during the build phase, so there might be some initial overhead if you change the build target between two pipeline executions (because Gitlab will need to pull another Docker that is specifically for this build target and is not yet cached).
This small experiment shows us that:
- setting up a CI with GitLab and Game.ci is fairly straight-forward…
- … as long as you’re already familiar with the GitLab CI tools, and you have some ready-made shell scripts to run Unity from the command-line!
Mixing Game.ci and Codemagic?
Ok – we just saw that Game.ci is interesting because it can instantly provide you with the right Unity version, in a given OS, using Docker. On the other hand, at the time of this writing, Codemagic only has two available Unity versions: Unity 2020.3.31f1 and 2021.3.0f1.
So, what if we tried to take the best from worlds, and use these Docker images on a Codemagic machine? Can we gain some performance boost by mixing the two, or maybe just use Codemagic instead of GitLab CI/GitHub Actions?
Foreword: Installing a specific Unity version on a Codemagic instance (without Game.ci)
Let’s first prepare a little “comparison sample” where we stick to the Codemagic tool on its own and simply install a specific version of Unity on it. This way, we’ll be able to compare the build time with a direct Unity install, and a Game.ci prepared Docker image.
The Unity project I’ll use for these Codemagic tests is still the same sample demo from before.
My Codemagic pipeline is very simple: I’ll build my project for a Mac but, before I run any Unity-related command, I’ll first install my custom Unity 2020.3.18f1 version on the machine.
To do this, all I have to do is:
- go to Unity’s download archive page
- select my Unity major version in the top tab:
- then, find my Unity version (here, 2020.3.18f1) and open the “Release Notes”:
- and finally, scroll down to the bottom of the page to find the changeset of this specific version:
And once I have this changeset, I’ll just run the following workflow (in my
codemagic.yaml config file:
Note: as explained in my other Codemagic articles, I also need to setup three environment variables in my Codemagic app (in the “unity” group):
If I run this workflow, I eventually get my Mac app as an artifact… after 16 minutes, 12 of which are just for the Unity install!
This clearly shows the drawback of Codemagic compared to Game.ci for using a specific Unity version – downloading and installing a custom editor can take a while; and because we are “borrowing” a different machine each time, we have to re-run these setups every time and we don’t have the same caching tricks as before, with the Gitlab CI.
Note however that, conversely, if you’re using the Unity version that is pre-installed on the Codemagic machine, then there won’t really be a setup time before activating the license and building the project (whereas the GitLab-based CI always requires some docker pulls and execution, even with caching) 🙂
Running Game.ci on top of Codemagic
Now that we have a rough idea of how long it takes to setup the environment to build the Unity project with the 2020.18.3f1 version on a Codemagic instance, let’s see if Game.ci can help.
Mixing Game.ci and Codemagic the wrong way
We’re going to start with a naive approach… and see why it’s clearly sub-optimal.
Suppose we keep the same overall configuration, and in particular we keep our Mac Codemagic instance for the build. This sounds pretty logical since we want to compare two scenarios, right?
Then, rather than downloading a Unity editor version from Unity’s archive downloads, I instead want to use the
docker CLI tool that is pre-installed on Codemagic machines to pull the matching Game.ci Docker image (that contains the right Unity version).
The build phase is more complex than in our initial Gitlab CI-based experiments because, this time, we can’t run the Game.ci Docker images as intended: since we’re on remote Codemagic machines, we’re can’t use the interactive
bash but we have to cheat by “keeping the docker alive” and then sending it commands with
All in all, we get the following
codemagic.yaml configuration file (with both the previous “pure” Codemagic workflow, and our new “Codemagic/Game.ci mix” workflow):
Note: here, I’m using some anchors and references to avoid re-defining my workflow steps, and therefore avoid inconsistencies. For more info, check out Codemagic docs 😉
But at that point, we’re going to hit two issues:
- pulling the Docker still takes several minutes and there is no caching (we do gain about 5 minutes, which is interesting, but overall it depends on the available bandwidth at the moment)
- building the Unity project inside the Docker is actually not working that well!
If you try and run this workflow on a Mac instance, you’ll see that it dramatically slows down the Unity command-line build (either locally or on the Codemagic servers)… and brings our total CI time to a total of roughly one hour!
Now, why is that? Well, if you take a look at the build log file (you can download it by clicking the small download icon at the right of the “Build” step), you’ll quickly see something strange but crucial:
Unity Editor version: 2020.3.18f1 (a7d1c678663c) Branch: 2020.3/staging Build type: Release Batch mode: YES System name: Linux Distro version: #1 SMP Tue Dec 1 17:50:32 UTC 2020 Kernel version: 4.19.121-linuxkit Architecture: x86_64 Available memory: 1989 MB
Take a look at the last line: despite the fact that Codemagic’s Mac Pro machines have a total of 32GB of available memory, the Unity editor is only using 2GB. This in turn means that the Unity editor is severely capped and can’t function at its real speed… hence the hour long build!
This is a well-known problem with Dockers running on Macs: because of architectural limitations, the Docker container has a limited amount of memory available. This amount is set machine-wide, i.e. it’s the same for all Dockers (sadly the
--memory-swap options from the official Docker’s docs don’t work on Mac!), and it’s of 2GB by default.
In other words, the only way to bypass this limit would be to set a higher default memory usage for all Docker containers on this machine… which is not the way to go. Codemagic is a platform that lends you a machine: in addition to it being potentially dangerous, our workflow should not require us to make admin-level modifications to the computer’s configuration!
Mixing Game.ci and Codemagic the right way!
The solution is actually pretty simple: we need to use something else than a Mac as a host machine. Remember that, since we’re building in a Game.ci Unity Mac Docker image, we’ll be able to produce a Mac app anyway – no matter which type of system the Docker container is running on.
Whenever you work with Docker, truth is: Linux is your friend. This OS works great with the tool because they share a sort of parenting. In particular, when you run a Docker on a Linux machine, the computer doesn’t distinguish between the host and the container memory; therefore, we’ll be able to take advantage of the full 32GB memory!
So let’s update our
codemagic.yaml to specify we want a Linux instance:
Just by switching our Codemagic instance type (and not changing anything else in the workflow – since Mac is Unix-based, all shell commands are still valid), we immediately get these results:
Pretty great, right? The CI now completes in just under five minutes and we can see that the Docker pull and Unity project build phases are way quicker than before. We still get a Mac app, because the Docker itself doesn’t care if it’s run on a Mac or a Linux, but we now use way more memory, which allows Unity to build faster:
Unity Editor version: 2020.3.18f1 (a7d1c678663c) Branch: 2020.3/staging Build type: Release Batch mode: YES System name: Linux Distro version: #33~20.04.3-Ubuntu SMP Tue Jan 18 12:03:29 UTC 2022 Kernel version: 5.11.0-1029-gcp Architecture: x86_64 Available memory: 32104 MB
With this alternative technique, we even beat the original GitLab-based Game.ci pipeline (which took roughly 10 minutes to complete, considering the license activation time)!
Quick final notes
Is this workflow always the best?
Despite being a bit cumbersome in terms of set up (the
codemagic.yaml file is honestly a bit more intricate than usual), this quick experiment shows that mixing up Codemagic and Game.ci can be pretty interesting in terms of performance! We can benefit from the power of a Linux computer while having just any Unity version we want, and actually producing a Mac app.
However, keep in mind that the initial Docker image download time actually depends on the current state of the network, but also on the platform you want to build for: if you take a look at all the Docker images Game.ci offers, you see that their size is different for each build platform. And images for Mac builds are about the smallest ones, so chances are that others would take longer to download.
And you should obviously try the various alternatives shown here in your project to see which one is the best in your case!
Security and legal important gotchas
Did you know that, as of this day, Apple forbids you from running macOS (software) on a non-Apple hardware? Doing so results in a breach of their EULA (which you a priori agreed to when you first turned on an Apple device…)!
That’s why Mac Game.ci Docker images don’t actually use macOS but run on Linux 😉
However, this is only possible because we are building a Standalone OS X version of our project, i.e. an app meant to be used by Mac computers (laptops or iMacs). If your goal is to build for iOS smartphones, then you’ll need your pipeline to be run on Apple hardware (in other words, a Mac machine) with XCode installed. So you won’t be able to follow the steps shown here.
You’ll need to either:
- stick with Game.ci on its own and, in your Github or Gitlab CI workflow, make sure that the signing part is done on a Mac, as shown in their docs
- or stick with a “pure” Codemagic workflow done on a Mac instance, in which case, as we’ve seen before, you should probably directly download the proper Unity version if you’re not using the one installed by default on the machine.
In this post, we’ve discussed Game.ci. We’ve seen that the tool is actually not exactly a CI (even though they are working on that, and will soon introduce a CLI tool), but rather a collection of Dockerised Unity versions with some documentation and a community around it. These docs provide you with quick getting started tips and samples to run Game.ci using the Gitlab CI or the GitHub Actions, but in truth, we can use the Dockers on any CI we want, including Codemagic.
Game.ci is interesting because it allows you to quickly get the specific version of Unity you want, along with the OS you want to build for. This can be particularly relevant if you want to take advantage of Codemagic’s computation power, or of its direct features to publish to the App Store or the Google Play Store (that Game.ci doesn’t have)… but that you also want/need to build your Unity project with a given Unity version that is not available by default on the Codemagic machines (because remember, Codemagic already has the latest Unity version installed). By running one of Game.ci’s Dockers on Codemagic’s build machines, you can speed up builds and still get the exact setup you required!
But nothing is a silver-bullet: so, the best is to try it out for yourself and see if, for your own project, it’s better to use Codemagic on its own or mix it with Game.ci 😉
I hope you enjoyed this tutorial — and of course, don’t hesitate to share your ideas for other DevOps topics you’d like me to make Unity tutorials on! You can find me on Codemagic’s Slack, on Medium, or on Twitter.
Don’t forget that you can check out the sample project on Gitlab over here. 🚀