A robot useful in scenarios involving test automation on push-button and touchscreen devices.

Built in the Altom test lab, AltTap is a robot designed to help with and mitigate some of the critical challenges of test automation. It is especially useful in scenarios involving push-button and touchscreen devices. AltTap’s special power is that of performing automated tests on any type of touchscreen device, even where it’s not possible to interact programmatically with the application.

How does it do that exactly? It simulates a human tester by the using a more complex stylus to perform click or swipe actions on a touchscreen and push actions on physical buttons. The process also uses an image-recognition algorithm for detecting elements or performing asserts.

The components of AltTap

  1. a work platform upon which the device is placed
  2. an external camera used for taking screenshots of the device screen
  3. an end effector which is moved into position using two stepper motors
  4. a haptic stylus driven by a servo-motor that has a feedback system which allows it to control the pressing force of the stylus on the screens and buttons. Purpose: protect the devices. It can perform other actions like pushing a card in a POS device
  5. a microcontroller which helps manage all the components and ensures communication with the PC from which the tests are actually run
  6. a power supply

Some of these components are 3D printed.

It is important to note that AltTap works together with a PC, which:

  • provides the support for creating tests.
  • processes the images.
  • sends instructions to microcontrollers.
  • checks if a test has passed or failed.

How does it work?

Tests are set up on a PC AltTap is connected to. AltTap will play the role of the tester by performing various click actions on the screen of the device. It will locate elements by using an image recognition algorithm.

Thus, as a prerequisite, the user needs to create a database with the images of the elements to be interacted with during the tests.

The OpenCV image algorithm used utilizes feature detection to create a match between the searched image and the screenshot of the device. If the match is successful, the algorithm will return the coordinates of the searched element. These coordinates are expressed in pixels. The PC then converts these coordinates to millimeters and sends them to the microcontroller. Once identified, elements are interacted with according to instructions.

If you prefer to think in diagrams and arrows, here is a visualization of the process:

Why work with AltTap?

We have put it to the test and are happy to declare that AltTap is a valuable tester’s helper. Here is a list of things we appreciate about working with AltTap:

  • you create the script once and run it multiple times on multiple devices with different resolutions.
  • you can repeat the tests for card payment as if it would be a real person
  • AltTap never gets overwhelmed, tired or bored: it can repeat a test for as many times as you wish, taking identical steps each time – and it can do this for a looong time!
  • it records everything, taking up an important part of the documentation activity.
  • it can test on screens small and large.
  • it can interact with device features that are not accessible programmatically: e.g. on iOS,  it can enable/disable the wi-fi connection or bluetooth, POS devices
  • it can also be used when you cannot interact with an application using software testing tools or frameworks: e.g. testing embedded software on industrial touch panels; or, testing an application that does not expose elements – the element cannot be called and you use a different method such as image recognition.   
  • It can test integrated modules of IOT systems

The limitations of AltTap

Just as its enthusiastic builders, AltTap has its limitations. We are constantly looking for ways to improve and help AltTap grow, but so far:

  • it cannot perform multi-touch gestures such as pinch, zoom, tilt, rotate.

Made with enthusiasm and ingenuity, open about its limitations, AltTap is a reliable and competent helper. It aims to bring value and improve test automation experiences for both testers and project stakeholders. Continuously supporting AltTap’s development, we are committed to seeing it achieve that goal. This is AltTap for now – we are terribly excited about what AltTap will be next!