An Encounter with the ‘cucumber.yml was found, but could not be parsed’ Error

One day last week I merged a colleague’s new feature tests to the existing test repository. Routinely after that, I logged in to our dedicated test server, updated the existing running test suite by retrieving the latest merged code, and ran a ‘bundle update’ because the updated code required the latest versions of the dependencies. The updates finished without hitches as expected after a minute or so. Task’s done, I just need to run a sample test to be certain everything’s well and good.

An error blows up in my face:

cucumber.yml was found, but cannot be parsed. Please refer to cucumber’s documentation on correct profile usage.

Before all these, I had run a simulation of the gem updates on my local machine including running all tests to make sure there were no surprises when updating the test server. But I was still caught off-guard. I didn’t see that coming.

What? That’s always the first question. I squinted at the screen and looked at the error message again; it says something about the cucumber.yml file unable to be read by the machine, like it was corrupted. I opened the file to check (which contains various cucumber profile configuration set up for running tests on different test environments), was able to view its contents in the text editor, inspected it for unfamiliar code and found nothing. It looks okay, but the machine says otherwise. Why?

I tried an initial Google search for the error message and saw a fix that was written five years ago, saying that the error occurs because of a particular ‘rerun.txt’ file. The post tells me that the problem will go away if I delete that file.

Except that I don’t have that file in my code repository. What now?

Maybe there’s really something going on with the test code. Let’s see what happens if I delete a particular profile in the cucumber.yml file. Done. No changes in behavior, error still exist. What if I delete everything? Done. Error didn’t go away. Hmm.. that’s odd. It seems that the file contents are not the problem, is valid YAML code too according to online checkers, and the file is not damaged in any way I can see.

I went to back to looking at the search results for possible solutions People keep telling me its about the rerun.txt file. Others say I need to edit a cucumber library file in order to see what test code specifically causes the error for cucumber’s runner. No more other clues. Now this is difficult.

I kept researching for plausible fixes online for a few hours, I thought that there may still be something that can help me but I missed. No such luck. Okay, let’s try editing that library file and see what happens. It was a first time viewing library code, because I didn’t have any reason to do it before, and told myself that maybe I should actually check it out more often.

I found the command-line interface profile_loader file and the particular code which loads the cucumber.yml  file in question:

Found you, cucumber's profile loader!

Found you, cucumber’s profile loader!

Commented out some few lines of code as suggested:

Now let's see what your problem really is

Now let’s see what your problem really is

Then ran the sample cucumber test again:

A problem with the Psych module! What's that? :O

A problem with the Psych module! What’s that? :O

Okay. It says that a missing private method named ‘load’ is being called for a particular Psych module. No wonder cucumber is failing. Bummer, I don’t have a single idea about a Psych module that cucumber runs. All I can do is another Google search for the new error message and maybe find a workaround.

I am reminded that problems in building systems for automated test suites are not limited to writing test code and the application under test. Just like any other software, they can sometimes break in areas we do not know anything.

Eventually I found this enlightening post on a Github repository:

Interestingly, a very short solution. It was a rubygems bug after all, and what I needed to do was to run a ‘gem update –system’ command to get our cucumber tests back up and running.


The Lessons I Learned This Year About Software Testing

Last year I told myself I would take a step back from automation because I’ve already built tests with a working knowledge of Java and TestNG. Apparently, with the way things went this year, I was not satisfied after all with what I had. I went on to study Ruby with Watir and Cucumber for about half of 2016, and I learned how to run web applications without using browsers after that. And for a second year in a row I feel I overdid studying again, and maybe I’m getting too comfortable about writing test code. In any case, programming has become such a useful tool in my workflow – performing checks automatically that otherwise would have eaten up a chunk of my time and enabling me to test other things.

Some other realizations during the year:

  • Re-writing test code using Ruby with Watir and Cucumber and other useful ruby gems pointed out how messy the checks I wrote last year using Java was. It helps that the Ruby language has less boilerplate syntax but my earlier mistakes are more about refactoring and design rather than about the programming language itself. I still need more deliberate practice at that.
  • I’ve found out that I actually do security testing in my normal day-to-day exploratory work, but I’m not especially great at it because nobody has taught me how to perform good web security testing before. It seems that getting comfortable reading and manipulating API calls is a step towards becoming better at it.
  • Teams who develop software with agility does not necessarily mean they implement scrum or kanban. If there’s good communication and teamwork, teams can actually forego many of the rituals and focus on the actual work.
  • Writing automated checks with healthy test description language is an important skill.
  • People only learn to solve problems when they feel that those problems are worth solving. And problems that are worth solving for me does not mean they’re worth solving for other people too, so I should never stop using perspectives.
  • We can leverage REST calls with web automation to make them run a lot faster and more stable.
  • Exploratory testing is a prerequisite to automation.
  • Building a great team is difficult. There are a lot of factors to consider – team composition, empathy, attitude, member skill set, strengths and weaknesses, to name a few. It takes some time too.

It’s About Communication

One big motivation for learning programming and testing applications technically is understanding what feelings and experiences do programmers face in their day-to-day work. Why do many of  them have a strong dislike towards bug fixing and other distractions? What sort of things go through their heads when they talk about databases and algorithms? Why do they say that a task is difficult when the changes needed from a user’s point of view are only minimal? Is working with legacy code really that terrible? How do programmers test their code? These are some of the questions that’s tough to answer, so I sought to learn how to write code myself. I wanted to put myself in their shoes and discover how I would feel in situations they often go through in building and maintaining applications, sort of. I thought that if I knew how they generally feel during such circumstances and knew how to code, it would greatly help improve my communication with them in the long run.

Based on my experience so far, it does.

Teaching Software Testing

Work, so far in the recent weeks, has been a little different this year than the previous years because I have been setting aside some regular time for teaching software testing to new hires. It feels great, like some of the presentations I’ve made in the past, but more demanding, not that I complain. After all, there are some things one can only learn by indulging in the experience.

Like understanding (and gauging) how much do I really know about software testing, about how software exploration happens, about what automation can and cannot do, about finding risks, and about what matters in testing.

Like how there is still a need for thinking time and note taking and other preparations, in order to deeply grasp what to discuss with the students and what lessons do I want to share, even if I already have the ideas about how to proceed in my head.

Like finding out what teaching techniques work well, and which one’s do not.

Like acknowledging that roles switch some of the time, that teachers are also students, and students are also teachers.

Like how fun it is to challenge colleagues, and see them learning and growing as time passes into remarkable people we expect them to be.

Underrated Software Testing Tools

  • Pen and paper
  • Whiteboard and markers
  • Notepads
  • Flow charts
  • Mind maps

There’s a lot of hype now about test management and automated checking tools, and I understand why. They’re powerful, they make testing tasks management, documentation, and output validation faster and easier for software testers if used properly, if integrated to the testing process by testers who fully comprehend the complexities of what testing is as well as the tools themselves. But the thing is, that big if matters. Such tools often don’t help beginners much, and are usually counterproductive. When trainees focus on automation and test scripting, they can forget what testing is really all about, and they can actually hinder the practice of testing skills that are more important when they concentrate on syntax and assertions instead of test ideas. Problem solving, collaboration, empathy, storytelling, and thinking skills matter more, and the listed tools above help us master those.

About Passion

It certainly was not passion that got me started walking on a road to a software testing career. When I got my first job in 2010, I did not know anything about the industry, not one bit of it. At the time I was just like most college graduates, I didn’t know what kind of work I should pursue, only that I am comfortable working with computers all day, I like how people can create interesting games and programs using such machines, I enjoyed the internet. The only things I knew then were things a lot of people can do: going to forums, playing LAN and online games, chatting with people who are miles away, googling for answers. I got a job in software testing because I was desperate to find a way to earn my own keep, seriously trying to prove to myself that I can do things that’s worth something (monetarily) to somebody. There wasn’t any passion in play, only a little bit of curiosity about the job position and a willingness to try what’s on offer. I told myself that I could always give my resignation if I did not enjoy the work.

Days passed. There were training on computer hardware and software, targeted to quickly familiarize ourselves about what the company does. I saw how programs were being built, I learned how software requirements were written and documented, I understood how the gears of software development process turned for the first time, and, eventually, I was taught how to validate products according to what they were supposed to do. Things were fairly straightforward, I immediately knew that there were still a lot of things I needed to learn in order to do the gig well. I had to pair with programmers and business analysts to be able to send out the best custom software versions on time. I had to talk to clients because ultimately it is their opinions that matter, and because I had to relay their wishes properly to the developer team. I had to learn about databases and bug reporting and programming and listening intently to people’s problems. Essentially, I saw how a group of people, individually having their own agenda, together, is able to build something of worth to our customers out of incomplete written notes, decisions experiments, and research. Sometimes we’ve done it with flying colors, sometimes we’ve released it behind schedule, a lot of the times we’ve finished the work just barely on time.

Weeks and months and years passed, and the more I learned about the job my confidence in what I can do grew. Small successes on all those days when lined up together collectively made memories of triumphs: documents that helps a client troubleshoot or manage software by themselves, test results that help business people decide what to do next, investigation data that helps a programmer debug an ongoing major issue, systems that provide accurate testing feedback quickly, questions that help bridge clients, product managers, and software engineers together in building a worthwhile product. Now I realize that what I’ve been doing has turned up to be a fun gig, because of all the inexhaustible challenges and things to learn, from testing heuristics, to programming concepts, to automation, to software development and testing infrastructure and systems, to finding out explicit and implicit requirements, to psychology and task management. Now it seems to me that this is what passion feels like, how it is only formed after failing so much as an amateur and after learning from mistakes and tiny successes, after trying something we don’t know if it will work and we enjoy it one time so we decide to try it again and again and again even if we fail countless times because we desire to be the best at it. Passion sounds to me a consequence of those actions and decisions, not a blueprint that is revealed to us at the very beginning of a journey to success.

Fiction and Non-Fiction

Mid-year recommended agile and testing books for software testers (or product owners):

I have always enjoyed reading, but back in college it was mostly Stephen King and Bob Ong novels, and the occasional Ayn Rand. They’re entertaining, I imagine characters in all sorts of dire circumstances and challenging situations, they’re educational and inspirational. They talk about the valuable things in life through fiction – adventure, friendship, philosophy, humanity, love  – in all sorts of ways, on all kinds of levels. Years passed since then and my favorite authors list have grown to include Richard Bach, Thomas Harris, Mitch Albom, William Golding, John Steinbeck, C.S. Lewis, John Grisham, among others, people who write fiction too.

But this year I have found myself surprised that I’ve been reading more testing-related, non-fiction books. What’s more surprising is that I have found them to be as refreshing, as educational, and as inspirational as fiction. I still imagine characters in both difficult and fun conditions, the books still talk about important things in life, although the lessons are now more specific, practical, especially to the kind of work that I do.