The Lessons I Learned This Year About Software Testing

Last year I told myself I would take a step back from automation because I’ve already built tests with a working knowledge of Java and TestNG. Apparently, with the way things went this year, I was not satisfied after all with what I had. I went on to study Ruby with Watir and Cucumber for about half of 2016, and I learned how to run web applications without using browsers after that. And for a second year in a row I feel I overdid studying again, and maybe I’m getting too comfortable about writing test code. In any case, programming has become such a useful tool in my workflow – performing checks automatically that otherwise would have eaten up a chunk of my time and enabling me to test other things.

Some other realizations during the year:

  • Re-writing test code using Ruby with Watir and Cucumber and other useful ruby gems pointed out how messy the checks I wrote last year using Java was. It helps that the Ruby language has less boilerplate syntax but my earlier mistakes are more about refactoring and design rather than about the programming language itself. I still need more deliberate practice at that.
  • I’ve found out that I actually do security testing in my normal day-to-day exploratory work, but I’m not especially great at it because nobody has taught me how to perform good web security testing before. It seems that getting comfortable reading and manipulating API calls is a step towards becoming better at it.
  • Teams who develop software with agility does not necessarily mean they implement scrum or kanban. If there’s good communication and teamwork, teams can actually forego many of the rituals and focus on the actual work.
  • Writing automated checks with healthy test description language is an important skill.
  • People only learn to solve problems when they feel that those problems are worth solving. And problems that are worth solving for me does not mean they’re worth solving for other people too, so I should never stop using perspectives.
  • We can leverage REST calls with web automation to make them run a lot faster and more stable.
  • Exploratory testing is a prerequisite to automation.
  • Building a great team is difficult. There are a lot of factors to consider – team composition, empathy, attitude, member skill set, strengths and weaknesses, to name a few. It takes some time too.

Teaching Software Testing

Work, so far in the recent weeks, has been a little different this year than the previous years because I have been setting aside some regular time for teaching software testing to new hires. It feels great, like some of the presentations I’ve made in the past, but more demanding, not that I complain. After all, there are some things one can only learn by indulging in the experience.

Like understanding (and gauging) how much do I really know about software testing, about how software exploration happens, about what automation can and cannot do, about finding risks, and about what matters in testing.

Like how there is still a need for thinking time and note taking and other preparations, in order to deeply grasp what to discuss with the students and what lessons do I want to share, even if I already have the ideas about how to proceed in my head.

Like finding out what teaching techniques work well, and which one’s do not.

Like acknowledging that roles switch some of the time, that teachers are also students, and students are also teachers.

Like how fun it is to challenge colleagues, and see them learning and growing as time passes into remarkable people we expect them to be.

If You Want To Change

Then start small. Really, small.

Sure, write your big goals on (physical or digital) paper, and dream about all the great things you can do when you’re slimmer and healthier, more skillful in your art, more financialy secure, or more social, but please don’t expect those changes to just magically happen because you listed and thought about them. They’re necessary steps for the new you that you’re excited to be, but incomplete. All artists want to draw better, all musicians dream of playing music in the grandest stage, all writers think about being successful authors, all of us want to be fit, want to be less socially awkward, and want to travel more. We all want to make a living where we both have fun and generate enough income for our families. Big goals are fun to think of, but they can be overwhelming. Vague goals are easy to list but they are difficult to implement in our lives.

So start small. Pick the most important goal that you have in your list, and if it is big, split, slice, break it down to incredibly tiny, specific, chewable pieces. Instead of drawing better, you could re-write your goal to draw one portrait (or landscape or idea) for an hour every Saturdays of January in your favorite place. If you want to lose some pounds, you could start by religiously running every morning of every other day of this month just for 30 minutes in a park somewhere near, before everybody else wakes up. If you want to write a novel, you could timebox yourself to writing something about it every day for one month just for an hour right after you wake up, before going to work. If you want to learn a new skill, give yourself 30 minutes to research and practice it everyday at work. Test yourself, experiment for a month without excuses, see if you can do these small things and like your results. If you do, then do them again. If you don’t, ask yourself why. The thing to remember here is that in order to make big stuff come true, you need to make small changes first, no shortcuts. You should make few but bold decisions, you need to re-build your systems of working, you must review and replace habits that don’t work with ones that do, because that’s where it all really starts.

We Don’t Know What We Don’t Know

I came across this free desktop app called f.lux once while on one of my tool-browsing sprees. According to their site, it’s something that automatically changes a display screen’s lighting according to the time of the day to make it sight-friendly. I imagined some similar setting that works well on smartphones, and I thought that it might be cool to try it out. So I installed it on my work desktop to see how it works. I clicked the download button on their site, ran the executable after it finished downloading, then waited for the app to finish installing. After that, I changed some of the app settings as needed and waited for something to happen. One minute went by, two minutes, 15 minutes.. bummer. Okay, nothing happened. Maybe my screen is already at the best lighting condition, adapted to the room where I was working. Happy path, I guess. Hmm, I could turn off some of the lights or turn on some of the others to test if it changes anything on my display, but it would be rude to do so when there are many people in the office. Maybe some other time. I went back to work after thinking that, then pretty much forgot about f.lux when weeks passed by.

Then one day I decided to stay in the office later than the usual. I remember wanting to finish some test code I was writing so that’s what I did, while it got dark outside and people went home one after another. It was a good day, and I was really in the mood to finish what I was doing, typing what’s in my head steadily, like a master conductor in the middle of a beautiful concerto. Then suddenly my Samsung S22B310 desktop monitor started flickering like crazy and the music abruptly stopped. Horizontal lines unexpectedly flew everywhere in my screen, looping back and forth. “Great, things break when you need them,” I muttered to myself.

I started checking the cables first because those are usually the culprit. Remove, re-attach, see if they fit snuggly to both the desktop and the monitor. Looks good, no problem there, but the screen is still acting up. Okay, let’s try restarting the computer.. monitor behaved well at bootup but then several minutes later flickers are all over the place once again. I tried replacing the cables with ones I’m sure was working but the same rapid flutters continued happening. “Okay, what did I do?,” my mind screamed. By this time, I’m thinking that maybe it’s the video card or the screen itself. Hardware failure, darn. I guess I won’t be able to work this evening now that the thing is busted.

Let’s back up data, I thought, trying to calm myself, and let’s just clean the system before I hand the machine to the techs tomorrow. That’s what I often do when similar hardware symptoms start to occur, before they totally break and do their worst, like when hard disk reads begin to fail some of the time. It’s always better to be prepared, I won’t be able to finish what I did today anyway. I synced important files to personal cloud storages, even if the flickers began to hurt my eyes. After that, I decided to clean up installed apps. Okay, there isn’t really much to clean in my system.. but perhaps I should remove this f.lux app because I don’t think it has been of any use since the day I installed it. I remember having high hopes for the app but I guess it wasn’t very helpful. Going to Control Panel, I clicked Programs and Features, then f.lux, clicked Uninstall, and there we go. Done.

As quickly as the flickering started earlier in the evening, they immediately stopped right after the f.lux uninstallation process finished. My monitor went back to normal. Fuck. That was it? After staring blankly at the screen for a few seconds, I suddenly found myself laughing. Haha, I realized I was the culprit after all.

Testing happens that way sometimes. Life too. We sincerely believe that the reason why an event happened is something we know for sure, because we’ve been playing with the system for many years now, because there can’t be any other possible cause after exhausting the known possibilities. But then again, we don’t know what we don’t know, and  when a time comes where unexpected new events happen and we see new information that we’ve never seen before and we realize how things became the way they are, we laugh at ourselves and honestly say that we haven’t known everything yet after all. Lesson learned.

Seeing And Knowing Magic

Working (well-thought) software feels like magic when we use them, even when they do not magically build themselves out of nowhere.

To be a software tester means to see first-hand how software development actually happens, in real-time. They discover how programmers code what they can from concepts and (almost always) incomplete ideas, they catch themselves and their co-testers trying to cover all possible valuable test cases, they find user experience designers hampered down by client whims, and they learn how hard it is for product owners to figure out customer needs. Requests become examples that turn into a collection of links, copies, and buttons, which then grow into prototypes, then features, eventually complex products. The apps that we love in both mobile and desktop or in cars and toys gets built by people who enjoy creating digital stuff that help people do their work or have fun. Here, the software tester finds herself in the middle of an exciting adventure – on journeys to finding solutions to worthwhile problems. There are travels along the way – some smooth, some rough and bordering burnout, some really easy, others enormously difficult and sometimes doesn’t make much sense – and there are lots of stories that can be told. And always, the team tries to go forward on a sustainable pace even when there may be unforgiving deadlines, crappy customers, incorrect examples, heavy rains, failing servers, health problems, even personal irrational worries that’s just around the corner. There are ups and downs, and the software tester sees and knows that everyone in the team that’s actually building the feature from imagined descriptions is doing what’s possible in the moment, everything in their power to solve one problem at a time, day in day out, given the situation. There are successes and failures, and then working software goes live and becomes magic.

All because the people who made it together shared their magic.

Proud Of What We’ve Been Through Together

Two of my software tester teammates (they’re also scrum masters) recently moved on to (hopefully) better offices, in search of new challenges and interesting adventures, and ultimately trying to build better lives for themselves and their families. Nothing more can be said about their decision to leave the group except that they’re heavily thinking about their future, which is great, which has always been a basic and necessary motivation in becoming an employee. In that regard, as a software tester and as their good friend, I am glad that they’ve been running scenarios in their head and practicing their decision-making skills. They’ve chosen a path to follow, they’ve understood the conditions and the consequences, and they’ve started testing that road.

As for me, the most important thing is that I was able to share my stories and my beliefs to them while they were under my wing, to build a thread of connection as software testers working towards common goals, a creative something that I’d like to think they will pass on to all the people they’ll be helping while being under new teams. For that alone, I am ecstatically happy that I have been given the opportunity to work closely with them these past few years, and elatedly proud of everything that we’ve done together.

About Sprint Data Gathering

Looking at a sprint document I designed a few years back (and updated by my scrum master colleagues), what I see are all sorts of data: planned total story points, actual finished story points, planned total task hours, actual finished task hours (total, per developer, team per day), burndown chart, attendance records, retrospective notes, tickets notes, returned tasks, unplanned tasks, underestimated tasks. So many information being gathered and compiled every sprint iteration; it’s not really too difficult to document but it does kind of takes away a good chunk of time to manage. Back then when we were new to scrum I thought that crunching numbers will provide insights as to how we can improve estimates and productivity, I thought that maybe we’d become faster and better at what we do after performing some mathematics.

It turns out that it doesn’t work that way. It turns out that estimates and productive behavior is a lot more complicated than it seems. It turns out that personalities and preferences and work environment and team dynamics and other small stuff we don’t pay attention much to play big roles in being good at software development as a team. Data about these things are not shown in the sprint document, and I don’t have the slightest idea how to measure them. So, looking back at all these stuff we’re trying to analyze, I’m haunted by questions: Which data are important enough to keep taking notes of, sprint after sprint, to keep reviewing? Why are they important? How do these data help me define how my team improves in succeeding iterations? Are they just pointing out problems but no solutions in sight? What if I stop documenting these data, would that help me focus my energy towards other possibly more important to-dos?