Automation is not always the best way

After years of being a Software Developer in Test who primarily focuses on automated tests being the one defending why automated tests are worth the investment; it feels really strange to be the one convincing stakeholders that, right now, automated tests aren’t the best solution we have to raise quality with the given situation. That’s why I want to talk with you about why I came with this, how I dealt with stakeholders and what was the outcome.

Let’s start with some background. We’ve been recently forced to restructure the company, and ending as the one QA is one of the results. That’s one of the reasons I had to start spending more time learning about methodologies, testing techniques and managing expectations. It’s not the first time I deal with these things, but I’ve always had someone to give my advice before committing to the decisions. That’s why it was my responsibility to understand when to apply other techniques (outside my expertise with automated tools to support testing), as well as good ways to implement them.

Another deciding factor is the tight deadline we’ve committed to, as it forces us to deliver a huge amount of work, setting up a situation where raising quality concerns is seen as an annoyance. We had some processes that were rolling out which stopped because of this change, as well as some of the teams decided that they don’t have any time to deal with automated tests; although others focused on them from the beginning, doing TDD while pairing so it helps them working with a half-defined technical design. After some tries trying to convince them like explaining how automated tests will help us preventing regressions while iterating, handing some working frameworks inside their to them so the transition would be easier (I don’t have expertise on JavaScript, so it wasn’t probably the cleanest code) or giving them written scenarios to leverage the work; I decided that maybe it wasn’t the moment to tackle this problem. Also, I got a message from the managers that we should keep noise to the minimum, so developers don’t lose their momentum. I raised my concern to them (now I feel free to say something when I think things aren’t working), and started weighting the different approaches we could take.

I spent the first sprints gathering possible processes and sharing them with our product manager. I also spent a fair amount of time working with the designers, learning from the user testing and reviewing the tests and scenarios from the teams which decided to pair (realising how much they were already caring about Quality allowed me to narrow my focus). When I started seeing and playing with the deliverables, it was obvious that we needed to tackle somehow the Quality of the product.

I started reporting bugs and odd behaviours to a spreadsheet shared with our product owner so the noise to the developers was way lower (as he could discuss with me what were our priorities). I did short exploratory testing sessions per feature and iterate through them. Afterwards, I paired with the designers and product owners to run more exploratory sessions, allowing me to centralise the reports reducing related errors. I wanted to do it with the developers so the feedback loop was shorter (this is a common practice and, in my opinion, one of the highest value testing activities), but I was not able to do so. As the release date was approaching, I involved and encourage anyone in the company to test the current state of the product to identify things like usability issues, user experience feedback, etc. I asked them to use low noise channels to give me the feedback so I could again act as a hub gathering and cleaning it.

I won’t talk about all the challenges, failures and problems we ran through. I wanted to write this because of the lesson I learnt: no matter how much expertise you have in one field, there are some situations when you need to jump out of your comfort zone because it is a clearly better solution, even if you make a poor implementation due to the lack of experience. Personally, I feel lucky because I’ve been changing industry and methodologies during my short career, making me feel that I’m not biased yet. But I feel particularly proud of myself because I didn’t try to find a solution using my automation knowledge. Have you been in a similar situation?