Sketching Lessons Learned in Software Testing

The other days I was reading a very interesting book “Lessons Learned in Software Testing: A Context-Driven Approach – by Cem Kaner, James Bach and Bret Pettichord”.

I sketched some lessons that I found very interesting, mainly from the first two chapters “The role of the tester” and “Thinking like a tester”. Enjoy!

Lesson 1. You are the headlights of the project

“A project is like a road trip. Some projects are simple and routine, like driving to the store in broad daylight. But most of the projects worth doing are more like driving a truck off-road, in the mountains, at night”

And this is how I imagined that people would see the mountain landscape during the night.

“Those projects (that are more like driving a truck off-road, in the mountains, at night) need headlights.”

“The tester lights the way”… therefore the poor moon can take a vacation once in a while!

Lesson 23. A tester is more than a tourist

This sketch is from Chapter 2, but I think it goes very well here. After understanding that testing is done to provide information, aka to light the way, it’s important to acknowledge that the tester does not only witness, but also evaluates.

Coming back to our landscape, how does a tester evaluate? I’ve chosen two methods from the book which I find very important: questions and bugs.

Lesson 7. Question everything, but not necessarily out loud

“[…] the value of questions is not limited to those that are asked out loud. Any question that occurs to you may help provoke your own thoughts in directions that lead to vital insights”.

Preface. A Few Notes on Vocabulary

I love the way the book is structured! It has 11 chapters of several learned lessons. Before going into the chapters, the authors share some notes on vocabulary, so that readers understand the definitions that the authors use. So how do they define a bug?

One of my favorite lessons is Lesson 3: You serve many clients. And “testing has many clients. They all have their own needs, and their collective needs don’t necessarily align.”

We know that “quality is value to some person” by Jerry Weinberg’s definition. I see these clients as being those person/s Jerry was talking about. The authors of the book list as clients the following:

• the project manager
• the programmer
• the technical writer
• technical support
• marketing
• top management and stakeholders
• the user, of course

Lesson 16. Testing is applied epistemology

“Epistemology is a branch of philosophy that helps you test better.”

Lesson 18. Testing is grounded in cognitive psychology

“If epistemology tells us about how we should think, cognitive psychology tells us about how we do think.”

One of my fellow testers asked me to give an example on how I would apply what I learned from the book. Therefore I’ve chosen Trello, a free web-based project management application. After we finish with the example, we’ll come back to the sketches. Let’s start testing Trello’s registration. You can test it yourself as well by following this link: https://trello.com/signup. You’ll find a registration form like in the below picture.

The first test I would run is creating a regular account with valid data:

• Name: Adina Moldovan
• Email: testadina1@altom.com
• Password: ******
• Click Create New Account… and I’m in

Did it work? It looks like. I’m on a dashboard with a Welcome Board, I see my name in the top right corner. Is it enough to say that the test passed? Some people would say yes, some people would say no. In any case, asking myself this question (not necessarily out loud), provokes my mind to think what else I could check. Well, I guess I could check my inbox for any welcome emails from Trello. Then I could further plan to run some tests without confirming my email address and some after confirming it. How about now, are we done? Hmm, I could switch my thinking to a user’s perspective. I believe that a user wouldn’t normally stop after creating her account, and most probably continue with adding some boards and tasks. Then check if they are linked to the account we’ve just created. Can we stop now with checking if the registration worked? I’m out of ideas, but I still have some questions:

1. How can I get to speak with the people who may know what else could I look at? Maybe I’m missing some internal logs, api calls, related functionality or others that I’m not aware of at this point.
   
2. Should I consider the test as a pass? Or should I scale some observations that I made? I refer to the fact that there is only one Password field for signing up into Trello, and most of the applications I’ve seen had a Confirm Password field as well. How is Trello’s context different than the other applications I know of, such as to justify the existence of only one password field? How do the other applications explain having a confirmation for the password field?
   

These are some of the questions I have for my first registration test, which keep me away from my tea break 🙂 [Hint to the Questions-related sketch]. We learned in lesson 7 to question everything, as questions are the foundation of testing. We also learned that the questioning is not necessarily done out loud.

Let’s go on with that hard to pronounce word, epistemology. Epistemology is all over this example. Some of the related questions previously presented in the sketch are:

1. How do you know the software is good enough?
   
2. How would you know if it wasn’t good enough?
   
3. How do you know you’ve tested enough?
   

When testing, we asked ourselves ‘did it work?’, ‘is it enough to say that the test passed?’. These two questions are related to how we know if the software is good enough. Then the questions ‘how about now, are we done?’ and ‘can we stop now with checking…?’ relate to how we know we tested enough. The how we know the software wasn’t good enough is somewhat covered by having further questions after our reasoning, but it’s not straight forward. Many times when I test I forget to think about this question. I focus too much on how I know the software is good enough. In theory, these questions are complementary, they should lead to the same result… right? Well, cognitive psychology wouldn’t agree with this. What we focus on influences our perception: what we see and what we ignore lead to how we evaluate and lesson 23 emphasizes on the importance of evaluation in a tester’s efforts. What we focus on and the errors that can result out of that can be labeled as any of the framing effect, selective perception, attentional bias, confirmation bias or others. More information about cognitive biases can be found here http://en.wikipedia.org/wiki/List_of_cognitive_biases .

Cognitive psychology teaches me about how I think. It reminds me that I am human and I have biases, that I use shortcuts especially when thinking about complex systems and many other additional stuff. Cognitive psychology pokes me to be aware. This leads to Lesson 19.

Lesson 19. Testing is in your head

Lesson 21. Good testers think technically, creatively, critically, and practically

“All kinds of thinking figure into the practice of testing. But we believe four major categories of thinking are worth highlighting.”

“Overall, thinking like a tester leads you to believe that things may not be as they seem. However things are, they could be different.”

Lesson 38. Use heuristics to quickly generate ideas for tests

“A heuristic is a rule of thumb; a way of making an educated guess. The word comes from Greek, meaning “serving to discover”. Heuristics are not served to lead to the right answer or the best answer, but they are useful nonetheless.” 

A powerful aggregated list of testing heuristics can be found here. “To use heuristics wisely, remember one thing: There is no wisdom in heuristics. The wisdom is with you.”

I recommend this book to anyone working in software, from testers to managers to product owners. It speaks about many other interesting subjects like testing techniques, planning the test strategy, bug advocacy, automating and documenting testing, interacting with programmers, managing the testing project and the testing group, the career in software testing. For me, reading about the tester’s role and thinking like a tester has been very insightful! Don’t wait no more and grab the book! Oh, and when you read it, sketch!

Just in case you think you can’t draw, I thought to link Ru Cindrea’s presentation about sketchnoting. Check it out here.