We recently had a meetup in Cluj focused on testing tours, with the occasion of Eurostar’s contest Teamstar. As part of their contest entry some of our colleagues decided to create a workshop in which we would practice with tours. They prepared intensely for a few weeks: searched materials on this technique, picked a list of tours, practiced with them, and created an exercise for the meetup.
The first thing I realized from this experience is that there is a lot out there about testing tours. The organizers kindly provided some materials to read before the meetup, from diverse sources. Going through them, I found references to even more materials on tours. (You can find them at the end of the article). So it seems to be quite a known technique.
But why all this focus on testing tours? In what ways are they valuable?
I especially liked how Cem Kaner characterizes tours and their value. He starts by saying “A tour is a directed search through the program.”[1] He also mentions that tours help testers to: create an inventory of attributes of the product under test, familiarize with aspects of the product, and explain to someone else the progress and status.
Having all these materials about tours already, what is there more to add?
At the workshop, me and my other 3 team members chose to do a feature tour for the Slack application. The organizers provided us with a nice card that had a guiding phrase on it:
So we took the card, sat down at a table in the garden where the meetup was held, and started exploring. Three of us had used slack before. One person really liked the app and knew a lot of neat features. For one of the team members it was the first contact with the app. This dynamic was very interesting, as the person who had no experience came with very useful questions and a fresh look, while the person who was a power user led our exploration and talked about the features he uses. During the exploration, we would all contribute by asking about a feature we noticed or when we were reminded by a feature we knew about.
When we began the tour, we systematically went through the first steps that someone takes when starting to use the application: creating the account, logging in for the first time, doing the introduction tour. Afterwards we jumped from feature to feature, as we found an area interesting or feature-rich. At some point, after discussing a few possible features, we questioned our understanding of the “feature” concept and searched it online. We concluded that security aspects, like register link expiration, were also features, and the portability on multiple platforms as well, although that was not obvious for me at first.
I started taking notes in a text editor, then realized it was hard to organize them and moved on to Xmind, creating a mindmap.
Here is the outcome of our 1 hour feature tour:
If I were testing the app, I would spend some time to further organize the feature notes in a way that connects them to one another consistently, and I would probably do some additional feature tours to gain more depth in my knowledge of the application.
During the debrief with the other teams, who chose different types of tours, an idea was to pick a few different kinds of tours and do them in a sequence. For example, we might start with a feature tour, then do a users tour, and after we have some knowledge of the app and the users, do a scenarios tour. The combination you might do depends a lot on the context, on the tester’s purpose, but we concluded that some orders were more logical than others. For example if you do not understand what kinds of users the application can have, and what features it has, in depth, it will be very difficult to create good scenarios.
Touring might be useful more often than you think
Another idea which came up during the meetup was figuring out when tours are useful.
Some people at the meetup mentioned that at the start of a project, or when they’re new on a project, the tours would provide value. But later on, as they would learn the system they’re testing, they think they would find less and less value from tours.
However, during the workshop we practiced with a small subset of tours. And we learned about the application under test from very different perspectives, very different things.
As Cem Kaner says: “Testing is an infinite task.” Given the complexity of the systems we work with, as testers we need to decide which techniques to use and when in order to extract a valuable subset of tests from the infinite set of possible ones. And tours can help with that. We gain an understanding of the system we test from multiple perspectives and we can use that experience later on to choose what to do next. Based on our testing mission, we might choose different tours and tours combinations. The feature tour we did helped me learn new features of a software I was already using. Based on the list we have, I can look more deeply into each item and find test ideas, if I want to test this product. I can also follow testing coverage.
If we’re not learning new things about the software from tours maybe it means we are looking at tours in a shallow way. Or we haven’t chosen the tour which can help us learn. But it seems improbable that we reach a point where there is nothing more to learn about the system we are testing. So if you find you don’t learn anything new from tours, you can try to:
- Do a new type of tour from the large list of tours – one you never tried before. For example, you might have done plenty of feature tours, and you might try a complexity tour, to find complex flows and data.
- Invent a type of tour – what can you focus on in your application that is not in the consecrated list of tours? For example, slack is a communication tool. You could make a list of all the situations you imagine in which someone would like to communicate something to other people and then see how the app can accommodate each of those situations. This could be a communication types tour.
- Pair with someone – this can help to get a fresh perspective and useful questions which allow you to look deeper into how to use this technique.
- Question your understanding of concepts you use – if you do a feature tour, how do you define a “feature”? Is there a different way you could define it to broaden your perspective and help you learn new things?
- Revisit the outcomes of old tours and see if there is something you would add, or how you can re-organize the information.
All these sound like fun activities. However, an objection I anticipate in regards to practicing with the technique or doing the things above is “I don’t have time for all this. I’d rather do actual testing in that time, since there’s so much to test!”.
As testers, we sometimes make decisions about tradeoffs between effectiveness now or improvement for later. Those are reasonable to make, but if your decision is always about effectiveness now, some questions can be raised:
- Since tours outcomes assist further testing, would you not do better testing if you use those outcomes?
- Does being too busy to make tours mean you’re too busy to improve the testing you do?
Here are some things you can do in busy times:
- Time-box the time you spend on tours; you might do 15 minutes tour once every 2 days. Working in small chunks frequently, helps you improve by sustained practice and at the same time it does not take too much time away from actual testing.
- List your priorities and think if a tour can help you to achieve faster one of the important results/pieces of information
- Try pairing in a tour, so that you make it faster; one person takes notes, while the other explores the application.
An essential aspect we can still add to testing tours is our own experience with them. It’s not about establishing a certain way of doing tours for everyone, or promoting some tours over others, but about encouraging testers everywhere to try out this technique and get more skilled at it.
Inspired by the meetup, we(at Altom) discussed during lunch today about other ideas of what we can do to improve with this technique and make it more popular:
- Make videos of you making tours and share them.
- Share a collection of videos with your team/colleagues and give (constructive) feedback on them.
- Write about your experience with using tours.
- Make a list of tour ideas and keep it around, so it is visible when you test, to remind you that you can apply this technique.
- Create a poster with tours and share it.
- Create an app which presents multiple tours(guides users through different types of tours) and allows users to create their own.
I’m ticking one of the elements in the list now. I hope others will get ticked off. Maybe by you?
Here are some resources about testing tours:
- http://www.testingeducation.org/BBST/testdesign/testdesign1a.mp4 (also part 1b and 1c)