On 100% Coverage

Yes, we need to write tests because it is something that we think will help us in the long term, even though it may be more work for us in the short run. If written with care and with the end in mind, tests serve as living documentation, living because they change as much as the application code changes, and they help us refer back to what some feature does and doesn’t, in as much detail as we want. Tests let us know which areas of the application matters to us, and every time they run they remind us of where our bearings currently are.

Tests may be user journeys in the user interface, a simulation of requests and response through the app’s API, or small tests within the application’s discrete units, most likely a combination of all these types of tests, perhaps more. What matters is that we find some value in whatever test we write, and that value merits its cost of writing and maintenance. What’s important is asking ourselves whether the test is actually significant enough to add to the test suite.

It is valuable to build a good enough suite of tests. It makes sense to add more tests as we find more key scenarios to exercise. It also makes sense to remove tests that were necessary in the past but aren’t anymore. However, I don’t think it is particularly helpful to advocate for 100% test coverage, because that brings the focus on a numbers game, similar to how measuring likes or stars isn’t really the point. I believe it is better when we discuss among ourselves, in the context we’re in, which tests are relevant and which are just diving into minutiae. If our test suite helps us deploy our apps with confidence, if our tests allows us to be effective in the performance of our testing, if we are continuously able to serve our customers as best as we can, then the numbers really doesn’t amount to much.


Doing Things Right, And Doing The Right Things

In testing software, automation is a tool that helps us re-run whatever repeatable checks we have on an application under test. We automate because we never have enough time to re-test everything by hand, and exploring the unknown parts of the apps we test is a far better use of our testing skills than following scripts. To automate is doing one thing right, within context, if it provides us the feedback we need. And that feedback we think we need from automation depends on what suites of tests are best repeated again and again, as well as what sort of tests costs more than the value they give.

There’s also tons of tools that helps us build good quality software. Even though we build more complex applications now than before, we have frameworks, libraries, intelligent IDEs, and other tools to help us spin up apps on a whim now too, ready to be modified as we see fit. Choosing the proper tools for the job is doing another thing right. But before we write any code, we need to be sure about the actual problem we are solving for our customer.

Yes, we need to do things right, from the get go if possible. They help us progress from one point to another faster than otherwise. However, I think it’s more important that we continuously take the necessary time to review whether we are doing the right things too, more important to actually get feedback and solve problems than merely adding tests and features.

We Find Out What Works For Us As We Go, Together

A software development team is composed of people, a combination of of programmers, testers, designers, and product owners. One team is visibly different from another because they have different people working in them, although both teams work in the same organisation and share the same mission. One group may be dealing with passionate but inexperienced new hires, another may be led by an introverted and soft-spoken senior, another team could be a lot more experienced with automating things, one team is probably better with team communication than others. Each team has a life of its own, always growing up, always trying to find out what specific problems itself has and what values it can contribute to the organisation, evolving day after day.

We understand that everyone is similar yet different from each other. Why is it then that we often insist in managing them with a one-size-fits-all process? Or am I missing something? I understand that we want teams to get better at what they do, we want them to release software faster, with better quality, but do we really think that one particular process works for everyone? Instead of that, shouldn’t we just immerse ourselves in a team, one after another, observe, ask where they are and where they want to go, talk to them, share experiences, help them solve problems, care for them, let them grow into a family with us? It seems to me that every time I get to join a team I think about the people in it first, no particular fixed process in mind, and we find out what works for us as we go, together.


The past few years I managed in-office software testing knowledge-sharing and tools-tutoring sessions for junior testers at work. It’s not something fancy, sessions are minimal but somewhat regular, and my goal was only to pass on some of the interesting lessons I’ve come to believe to be true based on studying and experience. I want them to become curious about the industry they’re in and I want them to take ownership of their own growth as testers.

I’ve always done the sessions in a group since there are only a few of them. It’s easier to manage that way. But this year I’ll try working with each tester one on one. And this year I’m having them select a skill they think is something they need to learn more of instead of me just providing some agenda in the group meetings. Each tester will have a weekly one-hour one-on-one schedule with me, we’ll review what the tester understands about the topic of their choosing, we’ll study together, and then I’ll provide challenges for them to go through until the next one-on-one. It’s an engaging setup I haven’t done before, something that’s likely to eat more of my time and attention but something that warrants testing since the organization recently changed work schedules to working remotely about half of the time.

Testing Goals for 2018

I think that I have more or less accomplished my testing goals for this year. As expected there weren’t many changes at work; workload remained pretty much the same and I had enough free time to try the tools that I thought were interesting. I read most of the books on my reading list. I was able to get the testing team curious on API testing. I re-wrote the existing UI test suite to API-based tests because they’re faster, more readable, and easier to extend and maintain that way. I learned new skills – web application pentesting basics and a nifty test-driven software development workflow – that will be put to the test in the coming years.

What about 2018?

Hmm. Besides continued knowledge-sharing sessions and the practice of test-driven software development I don’t really have that many goals next year in terms of testing and development of software. I’m sure I’ll continue to take interesting online courses and be updated on what’s happening in the testing community but other than that I don’t know.

What I do know and what I believe I want to focus on more next year is taking better care of myself, outside of work. I’d like to get better at both preparing good food and exercising. I’d like to put my drawing skills to work once again. I’d like to take better care of friends and family. The skills I want to get better at now are not those that I can practice in front of a computer but on the field, both when alone and with other people.

I’ll have to try changing some of my existing ineffective habits and stories. It’s going to be a difficult 2018.

The Lessons I Learned This Year In Software Testing

It was only several years ago when I started writing Selenium tests, first with Selenium IDE, then in Java with Webdriver, then in Ruby with Watir. Now I don’t write a lot of Selenium tests anymore, ever since I found out that it is often better (faster and more stable) to write automated checks for application features through the API. Or through unit tests. Selenium has its place in checking user flows or automating the UI, but only if I have to, if its value exceeds that of its costs. There lies an important lesson in automation: there is not a single tool that does it all. It is us who decides which tool to use for a particular test, and it helps to understand if a tool fits the specific use case.

And I think I’ve familiarized myself with a number of tools this year: Postman for API testing, Winium for automating Windows applications, BackstopJS for open-source visual testing, Cloud9 for cloud-based software development, Phonegap for HTML-based mobile app development, Docker for building shareable self-contained images of applications for development or testing, PHP testing tools (PHPUnit, Guzzle, Behat), and source code linting tools like Rubocop for Ruby. I’m not a master of these tools, but I know enough to be able to decide whether I need them (or not) for a particular thing I want to achieve. They’re in my tool belt.

Needless to say, I have outgrown the hype of automation. It is programming and tooling at its core. It helps us perform repeatable tasks without breaking a sweat, not limited to testing apps, if done with care. It is not easy. It can be rewarding. It all starts with a deep understanding of what definite task or problem actually needs solving.

And this year’s experiences has lead me to better grasp the nuances of software development, which is actually a problem of people, of teams and their habits and biases. Our team certainly has its defaults, some of which are not helping us get better at what we do. And as such from here on I’d like to contribute in key areas I believe our programmers have not had much time to think about because of project deadlines and resource constraints – dockerized application environments and shift-left testing – solutions for providing testing feedback earlier in the software development cycle, which in turn can help us build better apps and release faster.

Questions I Ask When I Want To Contribute On An Existing Software Development Project

What is this project all about?

Can I set up the project locally? How? What are the dependencies?

How do I know if I installed the project properly on my machine?

How do I contribute? What is the existing team process?

Did I break anything after making changes to the code? Are there any tests I can run to make sure that the project is still stable? What sort of tests do we have?