AltUnityTester – Testing Unity games and apps using Appium

Some background info: Appium and Mobile Games Automation

Before going into details about AltUnityTester, I want to tell you a bit about my own context and the struggles I’ve had in mobile games automation.

A few years ago, I started working on a few projects with Bitbar developing automated scripts for mobile game companies. Most of these companies were looking to get fast feedback on their games by running basic checks on many different devices using Testdroid Cloud. Most of them were also interested in running checks on both iOS and Android and they mostly wanted to see that the game starts, loads correctly and in a reasonable amount of time on as many devices as possible. Some companies were also looking into adding tests to verify the first steps of the game, finishing the first few levels, navigating through the main menus or trying out the most important features.

Appium seemed like the best choice for this, since it offers great interaction with the device on both iOS and Android, supports many programming languages (so each company could choose their favorite) and is a favorite of many testers that are familiar with UI Automation in general, as it’s similar to Selenium WebDriver.

But, for games developed using game engines like Unity or Unreal, that don’t use the native UI frameworks, Appium couldn’t detect any of the objects on the game. Clicking at specific coordinates was not an option considering the device fragmentation problem that most companies were trying to solve and the diversity of resolutions available out there (Here is a presentation I did at ETC 2016 about our approach). We decided to use image recognition to detect “objects” (pictures) on the screen.  We had a lot of success with that, but, as expected, image recognition is slow and this approach was not suitable for games like endless runners or games where reaction time is essential.

Unity for App Development

A couple of yeas after that, I started working with a different company as a tester/app developer in their New Concepts team. We were trying out a lot of new ideas, developing small apps and prototyping a lot of solutions in a very fast paced environment. Along with new ideas, we also needed to try new graphical designs and we needed to create apps that were fast to develop and could be deployed to a multitude of platforms if needed. We focused mostly on iOS and Android, but we needed to know that the apps could run on Windows, Mac, etc. After trying out a few options, including some web frameworks, we decided to develop all our apps in Unity. It has support for any platform that we could ever need and it’s excellent for fast prototyping, with incredible graphics on every platform and with an abundance of ready-made artworks, scripts, plugins and solution available on their Asset Store.

While Unity allowed us to be very productive on the development side, I was again struggling on the testing side. I had to deal with a lot of apps that still required a lot of testing. Most of these apps also had to work together as part of a system, and we often found ourselves trying to figure out ways to deploy and run some automated checks on our environment automatically. While the apps were relatively simple and would have been the perfect candidate for simple automated checks with Appium (which works on Android, iOS and even Windows nowadays), Unity didn’t have a solution for running these types of system level / UI driven tests on real devices.

Looking for a solution

While searching for a solution for these two problems, Rasmus Selsmark, a fellow tester who works at Unity in Helsinki, told me about this hack-day project that provides one possible option for running Appium tests against Unity games and getting feedback from Unity about the objects on the screen via the Debug.Log() functionality – so essentially by injecting information about specific objects in the device logs (logcat on Android or syslog on iOS), from where they can then be parsed and used inside the Appium tests.

I tried it out and then decided to try to implement something similar using a TCP socket running from the Unity game that can listen to commands sent from the tests. And so AltUnityTester came to be.

AltUnityTester

AltUnityTester – now also available on the Unity Asset Store – will let you find objects in your game and get their coordinates on the screen so you can interact with them from Appium.

Currently, AltUnityTester has bindings implemented for Python, allowing you to

  • create an AltUnityDriver,
  • connect to the AltUnityTester component running on the device,
  • ask for information on objects from the Unity Game
  • find elements and get their coordinate

Once you get the coordinates for an element (like “Bed” in the example below) you can use Appium to tap() on screen at those coordinates.

AltUnityTester used with Appium

Source code and documentation

We decided to have AltUnityTester open sourced and we have posted it on our GitLab account here:

Getting Started Video

To help you setup your game and get started with AltUnityTester, we’ve created a video tutorial:

https://www.youtube-nocookie.com/watch?v=uTO-Uzt3AEQ&t=6s

Running in the cloud

The Getting Started tutorial uses Unity’s Adventure example to show how a game like that could be tested using AltUnityTester. I wanted to give it a try in Testdroid Cloud as well, since you can run Appium tests in their cloud. I used their Python server-side Appium example scripts as a starting point and made the following modifications:

  • I added the python binding installation in the requirements.txt script:altunityrunner==0.1.3
  • I added the adb forward command in “run-scripts.sh”, just before starting Appium:
    adb forward tcp:13001 tcp:13000

I then used the following Python script to run the tests using AltUnityTester in the cloud (click on the photo for the link to the entire code snippet):

Sample Appium scripts used to run AltUnityTester tests in Testdroid cloud

Here is the result of these tests on a Nexus 5 from Testdroid Cloud:

Screenshot of test results using AltUnityTester in Testdroid cloud

If you’d like to know more about AltUnityTester or about any of the aspects covered in this blog post, leave a comment, we’d love to hear from you!

 

Go back


20 responses to “AltUnityTester – Testing Unity games and apps using Appium”

  1. James says:

    Interesting blog. I have looked at using BDD for automating games in the past. Even thought of coming up with a new development approach GDD Gaming driven development somehow using gherkin syntax.

    • Ru Cindrea says:

      Thanks! GDD – interesting idea. Did you already have some thoughts on how to make it specific to games?

  2. Juan Pablo says:

    This is exactly what i needed. I have been working in automating games test developed in Unity through image recognition. I started to do some stuff with the logs to find elements positions, and thinking of other ways to get unity elements. This will help me so much!

  3. daniel says:

    Hi,
    Is Java/C# binding is in the roadmap?

    Thank you.

    • Ru Cindrea says:

      Yes, both are on the roadmap.

      Java will be straightforward to implement with Appium, so we are thinking of doing that next.

      The C# + Appium combination is not the easiest, so for C# we are actually thinking of just implementing something using the Unity Test framework and without Appium. That will mean we could run the tests on any platform, from the Unity Editor.

      We’ll keep you posted!

  4. Ricky says:

    Hi,
    This is looking really great!

    I’ve been trying out the sample which works great on Android and now have a few questions.
    Should it work or are there plans to get this to work with Windows (ideally both classic and UWP)? I’ve tried using the sample with windows classic and the only issue I find is it dislikes the “tap” method.

    The c# solution you mention will use the Unity Test Framework, I assume that limits it to editor time testing only?

    Thanks!

    • Ru Cindrea says:

      Thanks for trying it out!

      I haven’t tested this on Windows myself as I haven’t tested the Windows Appium driver at all (I’m primarily a Mac user) but I do think it should work just fine with Windows as well. The tap method now uses Appium to tap at coordinates, so I’m not sure how the Windows Appium driver performs a click, but I think it should be doable. I’ve now added this as an issue in the Gitlab repository, I’ll try to work on it as soon as possible: https://gitlab.com/altom/altunitytester/issues/6

      About the C# solution: yes, the idea would be to get it to work with Unity Test Framework, using the editor time testing. However, this would mean that the Editor tests would be running against the game/app that’s running on a real device somewhere on the network (on a mobile device, or maybe even on the same host as the Unity Editor, but as a standalone app) so we would be able to run end-to-end tests against it. Hopefully! 🙂

  5. Vijay Jain says:

    Would like to know how to access Game Objects when we have canvas based screens/games.

    The only option using Appium is doing screen comparison and that to has drawbacks with different resolutions.

    We want to access game objects via Appium

    • Ru Cindrea says:

      Hi! AltUnityTester helps with exactly that! You can access game objects via Appium because we expose those using a socket connection. You need to instrument the game to allow the socket to be opened, but after that you can search for objects via AltUnityTester, get their coordinates and click/interact with them using Appium.

  6. Anne says:

    Hi, I’m new to automated testing (and by no means an engineer), but I have got this solution working now on Windows 10 🙂 This looks very promising!

    I have two quick questions.

    Does every game object need to have a unique name or can I select objects through their parents or hierarchy? Buttons usually do have unique names, but the text objects beneath them do not. And Ok or Cancel buttons have generic names, but are unique within a screen or panel.

    Secondly, can I combine altunitydriver with other drivers? My game uses native keyboard to enter player name for instance. So I would like to use the send_keys command for this.

    Thanks!

  7. Pragya says:

    Hi, I’m looking for AltUnity for Windows sing Selenium Java.. Is it available?

    • Ru Cindrea says:

      At the moment, we only have bindings written in Python. We are thinking of also adding Java, we just haven’t had the time to do that yet.. The Python bindings do work on Windows, so maybe that would be an option?

  8. Kalib says:

    Hi, I’m super new to Appium and just found AltUnity Tester through this blog. I’ve been following the exact steps from the video and have come across this error when running the first simple tests:

    ModuleNotFoundError: No module named ‘altunityrunner’

    https://i.imgur.com/VS0K5So.png

    Maybe I did not set up Appium correctly.

    Any suggestions? Thanks so much!

    • Ru Cindrea says:

      It sounds like the command to install the python module didn’t work properly. Can you try re-running it?

      This is the command:

      pip install altunityrunner

      and if you are not using python virtual environments and you want to install it globally, you have to run it with sudo, like this:

      sudo pip install altunityrunner

      I hope this helps!

Leave a Reply

Your email address will not be published. Required fields are marked *