How to run your first automated tests using Unity Test Framework & AltUnity Tester

[Update 2022] AltUnity Tester becomes AltTester Unity SDK starting with v. 1.8.0 and similarly, AltUnity Inspector becomes AltTester Desktop starting with v. 1.5.0 and they are both available here. 

This comes with a series of major changes:

1. AltTester Unity SDK is now available only through our website and is no longer available on the Unity Asset Store

2. We moved the AltTester Unity SDK project from GitLab to GitHub.

Read more about all the major changes in this article and join our Discord Server, if you have any questions or issues with AltTester!

---

Testing regularly allows you to ensure the quality of development so you might wonder what solution is the best. When it comes to testing your game in Unity, there are a few frameworks that serve this purpose.

In this article we are proposing a testing scenario using Unity Test Framework and AltUnity Tester for a Unity example project: Endless Runner. We will briefly describe these two solutions and show you some steps that you can take if you are looking for a solution to test your product. You can check the example project that we are using here.

Unity Test Runner

Unity Test Runner is a testing feature provided by Unity that uses a Unity integration of the NUnit library (open-source unit testing library for .Net languages). This tool is primarily useful for unit testing and it provides all the information you need about the status of your tests. 

The UTF(Unity Test Framework) has two testing modes: 

• Edit Mode – With these tests you can check anything that doesn’t require PlayMode. For example, you can check if a certain Scene contains a Camera or you can unit test your classes. 
• Play Mode – The Play-mode tests, on the other hand, check scripts that require runtime or physics.

Below are some steps that you can follow if you’re new to this technology and want to understand more about writing unit tests for your product.

Setting up the Test Runner

To start testing your game using Unity Test Runner there are some steps that you need to take:

• Open Unity Test Runner by clicking on Window -> General -> Test Runner
• Select PlayMode, make sure that the Tests folder is selected
• Click on Create PlayMode Test Assembly Folder (this button won’t be visible if you already have tests in your project)

• Now that you have created a play-mode folder, name it “PlayMode” and enter the folder to check if the “PlayMode.asmdef” file was created. This is an assembly definition file that Unity is using to point where the test file dependencies are. 
• You now have to set a reference to the scripts where you have your game classes. For doing so, look for the folder where your scripts are located.
  • Right-click on that folder and select Create->Assembly Definition. 
  • Name the created file “Scripts.asmdef”.
• The last step in this set-up is to link the two assemblies together.
  • Open the PlayMode.asmdef in the Inspector window and look for the Assembly Definition References section. 
  • Click on the plus sign and add the Scripts.asmdef file. Click on the Apply button from the end of the page, otherwise your changes will not be saved.

Now that all the set-up is done, you can start writing your first tests, run them and check their status in the Test Runner Editor window. To write edit-mode tests, the steps above should be followed again, but make sure that when you create the Assembly Folder you have selected the EditMode tab in the Test Runner. Also, don’t forget to set the Scripts assembly definition for the EditMode.asmdef file as well. You can see examples of edit-mode tests in our project in which we check if the directions left and right work properly.

In our example project we have written unit tests for the CharacterInputController and CharacterCollider classes. Each method from these classes such as Slide(), Jump(), ChangeLane() has been tested in the PlayerControllerTests and PlayerColliderTests files. 

The tests were kept simple, testing each functionality separately, following the WhiteBox testing technique. The main purpose of these tests is to examine every single function to satisfy the proper code quality. Even if all the unit tests passed, it does not mean that there might not be defects in some use-case scenarios that might be caught during automation testing.

Unit testing your game is great for refactoring code at a later date or for checking parts of the project without waiting for others to be completed. However, unit testing can’t be expected to catch every error in a program and since its very nature focuses on a unit of code it can’t catch integration errors. It may be hard to define the faulty piece of code if the issues are discovered once the whole script is ready. This is why it is recommended to use unit testing in conjunction with other testing activities.

AltUnity Tester

AltUnity Tester is an open source asset able to identify the objects present in a Unity scene, access their properties and create interactions with them using written tests in C#, Python or Java. Other things you can do with AltUnity Tester are: 

• simulation of any kind of device input
• get screenshots from your Unity Game
• visualize input actions during test execution
• in addition, this Unity plugin is compatible with Appium

In order to use this asset, you have to import it from the Asset store. Open the window from Unity Editor -> AltUnity Tools -> AltUnity Tester Editor and place it next to the Test Runner. If you are using our example project, once you open the AltUnity Tester Editor window, the tests list will contain our test classes.

The AltUnity tests in our project mainly focused on end-to-end testing, checking that the whole product behaves correctly when used the way it was developed. The StartPage tests check if the Start page is functional while the GamePlay tests go through some user scenarios such as avoiding obstacles, stopping the game or resuming it. The testing is focused on the integrity of the flow. The automation testing that has been done using AltUnity Tester helped to develop a broader picture of the project’s quality, which has not been possible with unit testing.

Unity Test Runner & AltUnity Tester

Unity’s testing framework comes with the advantage of running the tests in two different ways: running in the editor or running in the game.

The Edit Mode tests run significantly faster than the Play Mode tests and you can run them anytime when you don’t need the game to be running. Some Edit Mode tests that you can write are standard unit tests, checking that single units work as they should or checking the Scenes and their Components. On the other hand, Play Mode tests are slower because scenes need to be loaded.

One thing that is hard to test using Unity Test Framework is player input. Unity uses the Input class to provide information about any events happening when the scene runs as updated by the Input System. All methods are “get” methods so they can’t generate events.

With AltUnity Tester, however, you can simulate any type of device input such as mouse movements, key press actions, click, tap and even swipe. You can use the AltUnity Tester Editor not only to check your test results but also for instrumenting your game if you want to write C#, Java or Python tests using your favourite IDE. Tests run only in play mode in Unity Editor but you can also run them on real devices(mobile, PC). Another plus of this tool is its integration with Appium, giving the user the ability to interact with native elements.

Each of the two frameworks comes with its strengths and weaknesses and using these tools together can lead to a more complex testing strategy and better testing results! 

Let us know if you tried our example and if you found it helpful. For more information about end-to-end automation for Unity you can also check our colleague’s talk at the Nordic Game 2020. If you have any questions or feedback, please leave us a comment or join us on Discord.