Introduction
This November I attended the EuroSTAR Conference. The conference takes place once a year in different cities of Europe, and gathers around a thousand people interested in testing. I was involved in running the Test Lab. Further on I will write about the Test Lab, in general and this year’s EuroSTAR Test Lab in particular, and then in the second part I’ll share how I felt at the conference and my takeaways. I will also insert the names and twitter handles of testers connected to the topics of this blog post, because I think that being aware of the ideas shared outside our workplace bubble, plays an important role in our career development.
The Software Test Lab
The Test Lab is the place in a testing conference where delegates and speakers can actually… test. Depending on the conference, it can run in parallel with the talks or after the talks. In either case, the purpose of the lab is to create a context for learning by doing. The test lab crew creates exercises with learning objectives that I like to call experiments – we’re in a lab, aren’t we? Besides these ongoing experiments that attendees can do when they feel like it, there are also practical speaker sessions. So if you missed some speakers by not catching their talk, or you’d like to know more about how they think, you may encounter them in the lab doing a practical session, or trying out experiments. Some test labs have sponsors. The nice thing is that even the sponsor sessions are guided by the hands-on approach, thus they are practical and with learning objectives. There are many things that happen in the lab even besides the planned activities. People meet, share their approaches, pair test, share their own exercises and much more that happens on the spot.
But how has all this begun? The first Test Lab was set up in 2009 at the EuroSTAR Conference, by James Lyndsay and Bart Knaack, with the advice of many, including Stuart Noakes and Mieke Gievers. Ever since, the Test Lab has spread across conferences and the crew has expanded considerably. EuroSTAR, Agile Testing Days, Let’s test (and Tasting Let’s Test Benelux), BTD (Belgium Testing Days, now under the name Business, Testing, Development), and about half of the STAREast–STARWest conferences, organize ongoing Test Labs. Besides the initial promoters of the Test Lab concept, there are Martin Jansson, Ru Cindrea, Kristoffer Ankarberg, Ruud Cox, myself, Maaret Pyhäjärvi, Paul Carvalho, Wade Wachs, Guna Petrova, and the team expands as new people join each year.
This year is the 4th year when some of the Altoms were in the organization team of the EuroSTAR Test Lab. Ru was involved in 2012, 2013 and 2014, and I was involved in 2013, 2014 and 2015. Ru and I together with Maaret have been involved in the BTD Test Lab for two years now. BTD’s program chair is Mieke, one important advisor for the first Test Lab. Also Ru is doing the Let’s Test Lab for 2 years now, together with Martin or with James so far.
EuroSTAR Test Lab 2015
This year, Guna and I worked on thinking the experiments, finding session masters, sponsors, and dealing with logistics. We had James as mentor for challenges that we faced along our preparations journey which started 2 months before the conference. Jyothi, Carly and Susan joined us one week before the conference and helped us a great deal with the busy lab.
The experiments, practical sessions and sponsor challenges that we prepared focused on four main test topics: note taking, testing and thinking, mobile testing, and tools.
Note taking
We had experiments with different ways of taking notes: from sketchnoting, to mind mapping or regular pen and paper. We also used note taking as a reflection tool while in the context of solving a difficult problem, a black box, or in the context of observing and thinking about what our emotions can tell us.
The black boxes were again super stars this year. For 3 years since we’ve created them, thanks to Alex’s idea, wherever they go they’re in the spotlights. We built the first black boxes at the beginning of 2013, starting from the design of James Lynday’s ET 4 and 5 puzzles. In 2015 we organized a hackathon where we brainstormed new designs and implemented them in one night with people from Altom and outside Altom, who were interested in patterns, programming and microelectronics. So what are these black boxes besides a bunch of buttons and LEDs? The concept is the same as their name tells, black box: you have some inputs and some outputs. There is a relationship between them, but you don’t know what’s inside. The task is to find out by performing tests, what is that relationship, how the black boxes behave. We use them as contexts to different learning objectives. At EuroSTAR and not only, we used them to learn about problem solving and reflect upon tools that can help us solve problems. Oana came with this idea some time ago, and I liked it so much that I stuck with it. Ale uses them in her workshop of examining testing skills. This year we came up with a new context that Ale thought of and I put into practice: analyzing what we know about test strategies and what else could help us know even more.
The session master in this focus area was Raji Bhamidipati. Her session format was as a discussion and it was very useful to have her close to answer our questions related to note taking.
Testing and Thinking
Testing is a highly intensive thinking process. Unfortunately not all people share this vision yet, but I’m optimistic that with an open attitude they will learn about the true nature of testing. I hope that having dedicated experiments in the lab will facilitate this learning. This focus name, Testing and Thinking, is redundant because testing automatically implies thinking, just as testing automatically implies an exploratory nature. Therefore normally, there’d be no need to explicitly say and Thinking. As I couldn’t find a good wording to emphasize the thinking aspect of testing, I kept this name for now, but I’m open for suggestions to use in the future.
The experiments in this focus area were related to using models to aid the test strategy, sources of inspiration to use when stuck or out of test ideas, exploring the Session Based Test Management method to measure our testing while not having test cases – yes it is possible to measure testing without test cases 🙂
I loved this last experiment very much and the delegates seemed to have liked it very much as well. I saw a lot of enthusiasm on their faces, and some Mozillians who took part in the experiment are seriously thinking to introduce this method into their daily testing. I mainly used as session based test management tools the Google Scripts that Ru wrote some years ago. Here is the session template and here the dashboard template with measurements. They have a few bugs, but we plan to bring these scripts to a next level soon. You can find some instructions on how to use the dashboard template here.
James Lyndsay was the session master in this focus area. James was multi-functional this year. He played roles from session master, to owning several ongoing exercises, not to mention his input during the preparation phase.
Mobile testing
The experiments in this area included exploring accessibility on iOS, Susan’s flavor, and getting familiar with XCode and adb. I was not involved much in this during the conference, but I did the experiments afterwards. The Monday following the conference I participated in the Software Testing World Cup competition with my colleagues in Altom. The application under test was on mobile. Guess what type of non-functional testing we ran? Guess what tools we used to get the device logs?
Stephen Janaway was a double session master this year. He had two hands on sessions during which he made very good use of both the devices and the sticky notes present in the lab.
Tools
Experiments focused much on tools to use when testing, especially on those that support security testing. We were lucky to have Bill Mathews in the lab, who conducted his security session with a very nice setup: each person had their own virtual machine running an e-commerce application, that was about to be flooded by Zed Attack Injections.
Other experiments were related to using developer browser tools or mobile tools. To get a full overview on the experiments, see our Test Lab Report.
Altom as sponsor
This year, besides offering support to people involved in the Test Lab, Altom also sponsored the lab. For sponsors, EuroSTAR offers a 30 minutes time slot for them to organize a challenge. The Altom challenge was related to the BBST Foundations course. The main task was to analyze a Black Box and develop a strategy about how to approach it. After the people who took the challenge finished their strategy, we had debriefings, guided by the information from BBST Foundations Lesson 2 which is about Test Strategy. All EuroSTAR attendees had the chance to benefit of 10% discount to any Foundations course in 2016 organized by Altom, by using the discount code from the sticky cleaners that were available in the Test Lab. Altom also rewarded one person, who proved to be the most perseverent in her struggle with the Black Boxes, with a free pass to one of the next year’s courses.
How I felt at EuroSTAR
We slowly move towards the second part of the blog post. What I liked a lot was meeting again with great testers and sharing our stories. At my 4th EuroSTAR, I started to feel like home at this conference: the EuroSTAR crew with familiar organizers, the speakers and delegates who return each year, the conference format that changes a bit but not too much such as to become unfamiliar. To be more specific, it felt similar to going home for Christmas holidays: the cosy atmosphere and meeting with all my relatives that we usually see only once a year on this kind of occasions. And EuroSTAR, like Christmas, takes place once a year :). I was happy to meet again Guna, Zeger Van Hese, Julian Harty, Rene Tuinhout, Kalle Huttunen, James Lyndsay, Bart Knaack, Claire Goss, Raji Bhamdipati, Rikard Edgren, Ruud Cox, Shmuel Gershon, Richard Bradshaw, Huib Schoots, Stephen Janaway, Iain McCowatt, Michael Bolton, Kristoffer Nordström, the EuroSTAR Crew: Emma, Paul, Nicola and to meet with people from Tabara de Testare Cluj: Ioana Chiorean and Timisoara: Alin Groza.
I also met new people. It was a real pleasure to work with the Test Lab Team, with each one in particular. I’m also happy I had the chance to meet Bill Matthews, James Thomas, Dan Billing, and the many delegates with whom I had debriefings during and after the experiments.
In case you wonder why the names of some people are highlighted, it’s because I linked their twitter, linkedIn or blog profiles when I mentioned them for the first time in this blog post. In case you don’t have a twitter account yet, I strongly recommend you create one – a lot in terms of testing gets spread out on twitter, that you may not want to miss.
My takeaways
First, I consider that the model we chose to organize the Lab this year – the four main focuses, each corresponding to a half-day slot – was very good. On Tuesday evening the focus was on note taking, Wednesday morning on testing and thinking, Wednesday afternoon on mobile and Thursday morning on tools (and security). Having these in place, it was easier for the Lab Crew to assume roles and organize ourselves in terms of time. This model also allowed me to try out some of the experiments that my fellow testers had prepared. The previous years it was more difficult to focus on other learning objectives than the ones from the experiments I had prepared myself. I used to be so busy debriefing my experiments with the delegates, that I had no time left to experience the other interesting things going on in the lab. This year though, I managed to do one of Guna’s experiments with sketchnoting, I attended for more time than I expected Raji’s session on note taking, I tried to sort out the pattern in one of James’ knob box machines, I sent a tweet by pressing a button of a smart device – that was cool, and one morning I even participated to Lean Coffee in the Test Huddle, our Test Lab neighbors in the Expo area. It was fun! It felt good to experience more of the Test Lab atmosphere outside of my area of expertise.
My second takeaway was on Wednesday, when EuroSTAR usually organizes a dinner with all the conference attendees. This year and last year, among other people, my dinner companion was Rene. He is very skilled in giving talks and last year we talked a great deal about his passion, improvisational theatre. This year he was very encouraging that I submit a talk next year at EuroSTAR and we brainstormed on the topic. What we came up with was to talk about how testing changed my life. Rene has very good humour that he skilfully introduces in his talks, and gave me a few tips on how I could do that too. We had lots of fun letting our minds wander away, while finding analogies between work and life situations. I think that submitting a talk is an interesting and challenging idea, but I know it will be difficult for me to stay away, even for a little while, from the practical hands on way of sharing an experience.
A third takeaway would be awards session. The EuroSTAR Wednesday dinner is usually accompanied by awards. This year, the winner of the testing excellence award was James Lyndsay. I knew in advance that he was nominated, and still, when the moment on stage came, it felt very special. Huib’s speech, touching among the many things James did for the community, the Test Lab – the much beloved Test Lab whose concept I totally advocate for. Then, he also said about the black boxes and mentioned us, Altom, who created hardware versions for two of them. Hearing this, my mind was flooded with great memories while building the prototypes, back then some flaky skinny wires hanging out from a cardboard box; then the workshops in Altom on soldering, making custom wires, and drilling. I was touched that James got this award, and acknowledged this time more than ever, that the direction I support in testing, with the Black Boxes and the Test Lab, is highly appreciated among testers.
Speaking about appreciation, Bill Matthews and Richard Bradshaw, insisted on paying the Test Lab Crew dinners, for our efforts running the lab. They were very kind with this and gave us very good feedback about how the Test Lab felt. We also got credits for the Test Lab Report, that we presented on stage on Thursday after the last keynote. It was good to hear the report was good, we had quite some adventures with it 🙂
Also, one of the nicest things about conferences is bumping accidentally into testers. While heading back to our rooms from the Thursday dinner, Jyothi suddenly asked Is that Julian Harty?. Oh, but it was Julian Harty, who was there for the Friday Mobile Deep Dive Event organized by EuroSTAR as a separate event, after the 4 days conference ended. So, we started to talk and happily delayed our sleep with interesting conversations about Julian’s recent work. He’s writing a book and keeps all his versions in git. Not Word, Git :). So geeky and useful! The same accidental way we bumped into Zeger Van Hesse earlier that day. Jyothi and I were just about to go in town to buy some goodies for the office, when in front of the hotel we meet with Zeger. What a pleasant surprise! Zeger’s work is very interesting, one of my favorite talks of his is Testing in the Age of Distraction.
The last takeaway on my list is facilitating the SBTM experiment. I feel I’m doing the same like some people do when they eat: they leave at the end what they like the most. I talked about the experiment earlier in the blog post, but I had such a good experience with it that I wanted to mention it again among my takeaways. People loved the concept – thank you James and Jon Bach for popularizing it!, they loved the tools – thank you Ru and Shmuel for building them!, and I loved to see their excitement and openness to move away from the sphere of test cases towards a more thoughtful testing approach.
Being part of all this great conference atmosphere filled me with a lot of energy and gave valuable meaning to what I’m doing. My conclusion is that participating to conferences, and the community especially, is not only a buzzword, it is truly a great asset to learn, share, experience and fill up with energy.