I’ve been thinking about how to approach the subject of “Bad habits during testing activities”.
This concept of bad habits was very abstract to me and no bad or good habit came to my mind no matter how much I struggled. Then an idea hit me. How about I put myself in the context of testing something and observe my habits while I test? I may not pass through all the testing activities with this exercise, but it is a good starting point.
I’ll think about the test activities I had yesterday. Let’s see. Yesterday I did pair testing with my colleague Raluca. We were supposed to test a pretty straightforward feature, but the setup to get there was a bit tricky.
Two days before we jumped into testing, I asked the developer to confirm the sources of documentation she had for that feature. She confirmed the sources and also emphasized a new one that otherwise I wouldn’t have paid attention to. Then the following day the three of us had a meeting where we clarified our understanding and how to set up the tests.
The testing story in brief
My colleague Raluca started to explore the new feature, trying to understand how it works. We had a very detailed stepbystep guide of how to setup the tests. We tried two out of three setups and they didn’t seem to work. After a short time I joined Raluca and paired. We were confused about what we did wrong and started varying several parameters. We couldn’t ask for help right away as the developer was in a 7h time zone difference.
After about an hour, we started to document the areas we wanted to cover and which were blocked, to get the bigger picture of where we were and if there was anything that was worth testing next. So, we managed to put aside our blockers and continued testing what was left. As soon as the developer was online, we told her about our setup issues.
After one hour of chatting, the problem was identified and we got unblocked. Good habits in this case I’ll pass briefly through the good habits, as they are not the topic of the article. Getting information before the discussion with the developer. This way we were more efficient and more focused during the conversation. Pair testing. We were two: two minds to generate ideas and detect each other’s slips, especially since things didn’t work and we had no experience on how they should work. Creating the coverage list. After many failed attempts to get unblocked, we stepped back and analysed where we were. This helped in coming back from the rabbit hole and using what was left of our time in a productive manner. Shared our blocking issue with the developer as soon as possible. This way she could unblock us sooner.
Shared the steps we took providing many details. This helped a lot in debugging and identifying the issue.
Bad habits in this case, and maybe not only:
Assumptions. I didn’t start the conversation with the developer with the steps we did. Instead I started with an assumption for why it might not work, then continued with another assumption, and only after some time I listed the steps. Writing down the steps in detail was the key to success. I was surprised I didn’t do this from the start, especially because I am very familiar with bug reporting which starts with steps to reproduce. The brain unconsciously chose to make assumptions, prioritizing that over my six years of bug reporting. How much practice is needed to be aware of assumptions and make conscious choices all along?
Words only in my head. We were unblocked for one setup. As the other blocked setup was using the same script with only one different parameter, we assumed it would work. But we didn’t actually check. I thought about it but I didn’t verbalise it.
Can you guess what setup was blocked the following day? And it was not blocked because the script didn’t work, but because meanwhile some bug fixes broke the functionality. It took a great deal of time to determine if it was because we were doing something wrong in the setup again or if it was a new bug, all because we hadn’t seen that case running successfully before. I felt regretful for not verbalizing my thought which could have saved us a lot of time, and remembered something similar happened two weeks before. This seems to haunt me as a bad habit.
Lazinessplus. The non-working setup had two directions, incorrect steps, or a bug. So when doing pair testing, Raluca and I checked our assumptions with different scenarios. We did some checks for the bug assumption. I was running scenarios on live to see if we were not dealing with older bugs. Meanwhile, she was doing tests on staging where the new code updates were. As staging was under our control, she took advantage of this manually triggering the necessary scripts, instead of waiting for half an hour / one hour to get them triggered automatically. I did not see bugs on live, so when I finished I switched to running tests on staging. That could have been a good point to infirm the validity of the steps we were doing, the issue being running the scripts from the wrong folder. But… it didn’t happen, and reflecting on the events that took place next, it seems that even if I had the chance to do it, I was avoiding it. To identify the setup issue I only needed to do the regular staging setup on my own laptop and run those scripts myself. Instead of doing that, when I needed the scripts to run I asked Raluca to do it for me. Analyzing the situation I think that what stopped me from running the scripts myself was laziness, caused by the fact that there were too many steps to get there. I would have had to
- Step 1. Fin the document where the IP of the server that had the necessary information.
- Step 2. SSH to that server, find another file with the information, open it and look for the required server IP.
- Step 3. Finally SSH to the server I needed.
- Step 4. Go to our issue management tool, find the task on which the setup details were written.
- Step 5. Go back to the server and run the scripts from the setup.
My laziness was not only related to the number of steps. Last time I looked for our staging server on the server from step 2, I remember it was not in that list, so I was unsure I would find it there. So the laziness also involved some unknown as I wasn’t sure that after doing all those steps I would surely find the information I needed. This may have led to searching through other documents, who knows which.
Added up to the frustration of a nonworking setup, I gave up and let Raluca do the magic. My brain stopped there. Never did the thought cross my mind to ask Raluca for the connection details, as she was already connected, she should have known. But my mind was blocked to think that then.
Conclusion and an action item
Writing this article triggered a retrospective thinking process on how I approached testing yesterday. Before I started to write the article, I was somewhat aware of a predisposition to bad habits like not saying things out loud or lazinessplus. It was an AHA moment for me to realize that the assumptions I made delayed spotting the issue! Lost time just because of an assumption!
I also observed that even if I am aware of some bad habits in general, I don’t always observe them in action. I would definitely like to keep track of that and reflect more often on my work. Last year I used to keep a Testing Diary in which I wrote a few lines every day. The idea originally comes from one of Jerry Weinberg’s books, “Becoming a Technical Leader”, if I remember correctly. It used to work well. It felt good identifying my bad habits and alternatives to my choices. It felt smart as I could act quickly to prevent small problems becoming bigger. I should start doing that again. I will start doing it again :).