Some background info: Appium and Mobile Games Automation
Before going into details about AltUnityTester, I want to tell you a bit about my own context and the struggles I’ve had in mobile games automation.
A few years ago, I started working on a few projects with Bitbar developing automated scripts for mobile game companies. Most of these companies were looking to get fast feedback on their games by running basic checks on many different devices using Testdroid Cloud. Most of them were also interested in running checks on both iOS and Android and they mostly wanted to see that the game starts, loads correctly and in a reasonable amount of time on as many devices as possible. Some companies were also looking into adding tests to verify the first steps of the game, finishing the first few levels, navigating through the main menus or trying out the most important features.
Appium seemed like the best choice for this, since it offers great interaction with the device on both iOS and Android, supports many programming languages (so each company could choose their favorite) and is a favorite of many testers that are familiar with UI Automation in general, as it’s similar to Selenium WebDriver.
But, for games developed using game engines like Unity or Unreal, that don’t use the native UI frameworks, Appium couldn’t detect any of the objects on the game. Clicking at specific coordinates was not an option considering the device fragmentation problem that most companies were trying to solve and the diversity of resolutions available out there (Here is a presentation I did at ETC 2016 about our approach). We decided to use image recognition to detect “objects” (pictures) on the screen. We had a lot of success with that, but, as expected, image recognition is slow and this approach was not suitable for games like endless runners or games where reaction time is essential.
Unity for App Development
A couple of yeas after that, I started working with a different company as a tester/app developer in their New Concepts team. We were trying out a lot of new ideas, developing small apps and prototyping a lot of solutions in a very fast paced environment. Along with new ideas, we also needed to try new graphical designs and we needed to create apps that were fast to develop and could be deployed to a multitude of platforms if needed. We focused mostly on iOS and Android, but we needed to know that the apps could run on Windows, Mac, etc. After trying out a few options, including some web frameworks, we decided to develop all our apps in Unity. It has support for any platform that we could ever need and it’s excellent for fast prototyping, with incredible graphics on every platform and with an abundance of ready-made artworks, scripts, plugins and solution available on their Asset Store.
While Unity allowed us to be very productive on the development side, I was again struggling on the testing side. I had to deal with a lot of apps that still required a lot of testing. Most of these apps also had to work together as part of a system, and we often found ourselves trying to figure out ways to deploy and run some automated checks on our environment automatically. While the apps were relatively simple and would have been the perfect candidate for simple automated checks with Appium (which works on Android, iOS and even Windows nowadays), Unity didn’t have a solution for running these types of system level / UI driven tests on real devices.
Looking for a solution
While searching for a solution for these two problems, Rasmus Selsmark, a fellow tester who works at Unity in Helsinki, told me about this hack-day project that provides one possible option for running Appium tests against Unity games and getting feedback from Unity about the objects on the screen via the Debug.Log() functionality – so essentially by injecting information about specific objects in the device logs (logcat on Android or syslog on iOS), from where they can then be parsed and used inside the Appium tests.
I tried it out and then decided to try to implement something similar using a TCP socket running from the Unity game that can listen to commands sent from the tests. And so AltUnityTester came to be.
AltUnityTester – now also available on the Unity Asset Store – will let you find objects in your game and get their coordinates on the screen so you can interact with them from Appium.
Currently, AltUnityTester has bindings implemented for Python, C# and Java allowing you to:
- create an AltUnityDriver
- connect to the AltUnityTester component running on the device
- ask for information on objects from the Unity Game
- find elements and get their coordinate
- simulate any kind of device input
Once you get the coordinates for an element (like “Bed” in the example below) you can use Appium to tap() on screen at those coordinates.
Source code and documentation
We decided to have AltUnityTester open sourced and we have posted it on our GitLab account here:
Getting Started Video
To help you setup your game and get started with AltUnityTester, we’ve created a video tutorial:
Running in the cloud
There’s an older tutorial video we used to show how Unity’s Adventure example could be tested using AltUnityTester. I wanted to give it a try in Testdroid Cloud as well, since you can run Appium tests in their cloud. I used their Python server-side Appium example scripts as a starting point and made the following modifications:
- I added the python binding installation in the requirements.txt script:altunityrunner
- I added the adb forward command in “run-scripts.sh”, just before starting Appium:
adb forward tcp:13001 tcp:13000
I then used the following Python script to run the tests using AltUnityTester in the cloud (click on the photo for the link to the entire code snippet):
Here is the result of these tests on a Nexus 5 from Testdroid Cloud:
If you’d like to know more about AltUnityTester or about any of the aspects covered in this blog post, leave a comment, we’d love to hear from you!