Cypress: Printing Information to the CLI

Most people will run tests written in Cypress via the built-in test runner. It provides much of the information people need when looking at tests and checking whether it is doing what it is supposed to do – test status, command logs, app preview for each test run, plus snapshots of test steps. Very useful when looking at test failure points.

Some people, however, like to run Cypress tests using the terminal or the command-line interface. Sometimes, we don’t necessarily have to see the test running over the app UI. For one, I can start writing a draft of a new set of tests while the existing test suite runs in the background; I will want to know if there are failing tests, but not get distracted by them running on the test runner. That’s how tests run on a CI server too; we just want the test results.

Cypress works well for both scenarios. There’s just less information displayed on the CLI from running tests by default. To help us debug failing tests without opening the test runner, we could print out information we need from the test as it runs. If we want, we can show network request and response pairs, or simple variable values. Often, I find that printing out the URL at certain point for some tests helps me understand test results better. I can also click that URL to redirect me to the page afterwards if there are details I want to check out later for myself.

In order to print that URL to the terminal, we’ll need to add a task to Cypress as a plugin. To do that, we open the plugins/index.js file and add a log task there, like so:

After that, we can now use the log task inside our tests. Here’s an example:

Help Yourself Perform Tasks With Personal Shortcut Commands

In a recent automation work I’ve been asked to take part in, running a single test safely, according to peers, generally follow a three-step process:

1. Delete existing feature files:  rm -rf *.feature
2. Create updated feature files:  PROJECT_DIRECTORY=../<dir> APP=<app> TEST=automated ruby create_tests.rb, where app is the name of the app you want to test
3. Run the desired specific test:  cucumber --tags @<tag>

This means I have to re-run these three steps over and over for every single test I want to review. That can get tiring easily, especially if I have to re-type them repeatedly, or even if I copy-paste. Using the arrow keys in the terminal helps, but sometimes commands can get lost in the history and then it becomes a bit of a hassle to find them.

There should be a way for me to run an automated check I want to review, given just the app name and the test tag. It would be nice if I can run a specific test using a shorter command but still following the same process as provided.

I decided to set aside a little bit of time to write a personal script, and used Makefile because I’ve had experience with that before.

My personal short command for running a test:  make test app=<app> tag=<tag>

And here’s what the script looks like, which runs the known commands step-by-step plus displays some information on what’s currently happening:

Now I won’t have to worry about remembering what commands to type in as well as the sequence of the commands. I can give more focus on actually reviewing the test itself, which is what’s more important.

Is There Really Nothing We Can Do?

“I’ll wait for the code to get pushed to the Staging server before I continue testing because the data there is a lot more stable.”

“It’d be a waste of time trying to teach people how to program when it feels like they don’t have the motivation for it.”

“There’s a high chance things will break after a code merge with the production branch. It’s been that way ever since. We’ll just have to fix those that we can until the release.”

“I wish there was an easy way to spin up a version of our apps in a local environment. That should help us test things faster.”

…

Some problems become the status quo. It’s worth revisiting them, asking ourselves whether there’s really nothing we can do about it or we’ve just become comfortable with complaining.

Lessons from Richard Bach’s “Jonathan Livingston Seagull”

I’ve always kept a copy of Richard Bach’s “Jonathan Livingston Seagull” within reach, for those days when it seems necessary to remind myself of Jonathan’s story – of struggle, of questions and learning, of constantly challenging himself, and of keeping true to his curiosities. I’ve re-read it a number of times in the past several years (it’s a short book, digestible in a few hours), and it has never failed to uplift my spirits. It always feels like talking to a great friend that I’ve never seen for an extended period of time.

Here are some favorite lines from the book:

• Do you have any idea how many lives we must have gone through before we even got the first idea that there is more to life than eating, or fighting, or power in the Flock? A thousand lives, Jon, ten thousand! And then another hundred lives until we began to learn that there is such a thing as perfection, and another hundred again to get the idea that our purpose for living is to find that perfection and show it forth. The same rule holds for us now, of course: we choose our next world though what we learn in this one. Learn nothing, and the next world is the same as this one, all the same limitations and lead weights to overcome.
• Jonathan Seagull discovered that boredom and fear and anger are the reasons that a gull’s life is so short, and with these gone from his thought, he lived a long fine life indeed.
• “You can go to any place and to any time that you wish to go,” the Elder said. “I’ve gone everywhere and everywhere I can think of.” He looked across the sea. “It’s strange. The gulls who scorn perfection for the sake of travel go nowhere, slowly. Those who put aside travel for the sake of perfection go anywhere, instantly. Remember, Jonathan, heaven isn’t a place or a time, because place and time are so very meaningless.”
• “To fly as fast as he thought, to anywhere that is,” he said, “you must begin by knowing that you have already arrived.”
• “Don’t believe what your eyes are telling you. All they show is limitation. Look with your understanding, find out what you already know, and you’ll see the way to fly.”
• “Look at Fletcher! Lowell! Charles-Roland! Judy Lee! Are they also special and gifted and divine? No more than you are, no more than I am. The only difference, the very only one, is that they have begun to understand what they really are and have begun to practice it.”
• “Why is it,” Jonathan puzzled, “that the hardest thing in the world is to convince a bird that he is free, and that he can prove it for himself if he’d just spend a little time practicing? Why should that be hard?”
• “Oh, Fletch, you don’t love that! You don’t love hatred and evil, of course. You have to practice and see the real gull, the good in every one of them, and to help them see it in themselves. That’s what I mean by love. It’s fun, when you get the knack of it.”
• We’re free to go where we wish and to be what we are.

Trying Out Cypress with Circle CI

Circle CI has been around a while but I’ve never tried their service before. There hasn’t been much of a reason to do so; at work I mostly stuck with a local Jenkins instance because it got the job done. But I recently had the urge to try it out for a private project that I have. I wanted to see if it plays well with Cypress, a test tool I’ve been using for some time.

A search tells me that there’s already a Cypress-Docker-CircleCI example on Github, from Cypress themselves. Cool, I just need to copy their settings to my own project and make changes accordingly.

Here’s how the package.json file looks like:

You’ll see a cy:circle-junit command in there, which does the same thing as cy:run (which runs tests on a terminal) but also builds a JUnit report.

To run our tests on Circle CI, we’re going to need a circle.yml file inside our project, which contains the following:

From this file Circle CI is supposed to checkout the test code, run the cy:circle-junit command, which installs Cypress and other dependencies inside a Docker image, saves that image in a cache (and restores that cache for every succeeding test runs as long as the dependencies remain the same), runs our tests inside a container, creates a simple report, and stores test artifacts afterwards. Looks straightforward. 🙂

Now let’s see if things work!

Connecting my Cypress project on Circle CI and pushing code changes on the remote repository automatically creates test runs that shows up on a neat dashboard like so:

And we can zoom in on the details of every test run, which looks like this:

Awesome! Looks like I’ll continue using their service for this project, and maybe on other projects as well. I’m off to adding proper tests to the suite!

 

Remember to Focus on Helping People

It’s easy to get caught up on writing a lot of automation, adding more process steps, and implementing stricter policies, believing we need them, because we think we’ll be safe from risks that way. We might have had a history of being carelessly bitten in the past.

So it is good that we’ve chosen to do something. But we have to be aware of the potential to get lost in the details.

Why did we have to do this work in the first place?

We don’t just want to add one more test to the suite. We don’t just want another meeting. We don’t just want customers to pay a fee when they request something for the nth time. What we want is to remember the big picture, what we want is to focus on helping people.

Rookie Blog Post Mistake

I’ve made a rookie blog post mistake early in the new year:

Why am I posting a 2017 blog post when it’s already 2019? :O

That is, I forgot to remove a scheduled blog post I’ve made in the past and it got published recently. Some of you might have gotten the post through email and wondered why I’m talking about 2017 and maps. It’s a little embarrassing; that post should have been in the trash a long time ago but for some odd reason I didn’t delete it until now.

Maybe I thought there was an idea worth saving in there which I forgot to review. Or perhaps I just got lazy.

I’ll just think of this as a reminder that mistakes are always bound to happen. And what’s important is how we behave and what we do when they happen.

The post is gone now. And I am sorry about the weird blog post. I’ll make adjustments on my end so that it won’t happen again.

Lessons from Richard Bach’s “Life with my Guardian Angel”

The new year is always a great time to remember to talk to our guardian angel, to look at what beliefs do we have come to accept, and to remind ourselves of which sort of heaven do we really want to go to. 🙂 Happy New Year!

Some favorite lines from the book:

• “You’ll never learn it all,” he said. “Nobody has learned everything about flying and they never will. But if you practice, you can learn enough to be a good pilot, even if you don’t know everything.”
• Every context, every subtext means the same: heaven will be joy when we reach that place of mind, whether or not we reach its borders alive, or not-alive.
• All the books I’ve read about death and dying, filled with stories and accounts and proofs that we go on after this lifetime, how can my heaven be exactly the same place that it is for everyone who’s died in the last thousand years or so? Is there one place in paradise where some folks enjoy cigars, a different heaven where others travel instantly to other places, others have houses, others don’t need shelter from winds or rains, which don’t exist in their heaven? Does heaven require space and time? Can heaven be kind of a dream, where we imagine space and time, but know that space-time doesn’t exist? How many heavens can there be? Two? Twenty? A million heavens, an indefinite number of heavens? Not just for mortals, but for every expression of life and love who recovers from the belief of dying? Do mortals share the same heaven with animals, with bees? There must be a heaven for dogs and cats who love people, and different heavens for dogs, cats, and other animals who prefer to continue without any humans at all. There must be a heaven for chipmunks and a separate heaven for crocodiles, and different ones for recently arrived spirits who’ve spent a lifetime thinking that no such place exists. There must be separate heavens to take care of the people who believe the various religions. Then different heavens for the different ways of thinking. Separate heavens.
• Don’t you know? What you call ‘death’ is your chance to make different choices, make different stories about life and love.
• What will your mortal’s life be like without tests, without adventure, without risk? Does the word ‘graveyard’ come to mind?
• Your mission in every one of your lifetimes is to create love, to express love, to treasure love, to paint love in the biggest letters in every color of every language you will ever learn!
• The world of space, time, and appearances can be wondrous beautiful. Have care, though, and don’t let your mortal mistake them for real.
• You can work right now, today, as a mortal, to refine your beliefs, to make your heaven happier than the squirrelly heaven you’ve already built with your not-much-thought-about belief system. You can do this now.
• We’re connected by the bonds of love that we’ve built. With anyone. With our parents, with our friends, with our pets, with fictional characters from books or motion pictures, with characters in our own imagination!

Flutter: Assorted Notes

Just a few useful notes about Flutter I want to remember:

Update NPM on Windows:

1. Open PowerShell as Administrator
2. Run: Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force
3. Run: npm install -g npm-windows-upgrade
4. Run: npm-windows-upgrade

Update Android app APK version:

1. Go to android/app/build.gradle
2. Increment the versionCode found inside the defaultConfig settings
3. Update the versionName accordingly

Add Flutter to Mac Path:

1. Find where the Flutter SDK is located
2. Open your bash profile, i.e. sudo vi ~/.bash_profile
3. Add the following: export PATH=[PATH_TO_FLUTTER_GIT_DIRECTORY]/flutter/bin:$PATH
4. Save changes

Change Package Name for an Existing app:

1. Go to your AndroidManifest.xml file
2. Change the label name:
   <application
android:name="io.flutter.app.FlutterApplication"
android:label="[APP_NAME]"
3. Change the package name:
   xmlns:android="http://schemas.android.com/apk/res/android"
package="your.package.name">
4. Then go to your android/app/build.gradle file
5. Update the applicationId found inside the defaultConfig settings
6. with the package name:
   defaultConfig { applicationId "your.package.name" }
7. And finally, go to the MainActivity.java file:
   package your.package.name;

Change Package Name for a New app:

1. Run: flutter create --org com.yourdomain appname

Notes from Jonathan Bach’s “Session-Based Test Management”

Tracking exploratory testing work is difficult for test managers. We don’t want to micro-manage testers, we want them to explore to their hearts content when they test, but we won’t know how much progress there is if we don’t track the work. We also won’t know what sort of problems testers encounter during testing, unless they have the nerve to tell us immediately. Jonathan Bach’s “Session-Based Test Management” article has one suggestion: use sessions, uninterrupted blocks of reviewable and chartered test effort.

Here are a few favorite notes from the article:

• Unlike traditional scripted testing, exploratory testing is an ad hoc process. Everything we do is optimized to find bugs fast, so we continually adjust our plans to re-focus on the most promising risk areas; we follow hunches; we minimize the time spent on documentation. That leaves us with some problems. For one thing, keeping track of each tester’s progress can be like herding snakes into a burlap bag.
• The first thing we realized in our effort to reinvent exploratory test management was that testers do a lot of things during the day that aren’t testing. If we wanted to track testing, we needed a way to distinguish testing from everything else. Thus, “sessions” were born. In our practice of exploratory testing, a session, not a test case or bug report, is the basic testing work unit . What we call a session is an uninterrupted block of reviewable, chartered test effort. By “chartered,” we mean that each session is associated with a mission—what we are testing or what problems we are looking for. By “uninterrupted,” we mean no significant interruptions, no email, meetings, chatting or telephone calls. By “reviewable,” we mean a report, called a session sheet, is produced that can be examined by a third-party, such as the test manager, that provides information about what happened.
• From a distance, exploratory testing can look like one big amorphous task. But it’s actually an aggregate of sub-tasks that appear and disappear like bubbles in a Jacuzzi. We’d like to know what tasks happen during a test session, but we don’t want the reporting to be too much of a burden. Collecting data about testing takes energy away from doing testing.
• We separate test sessions into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We call these the “TBS” metrics. We then ask the testers to estimate the relative proportion of time they spent on each kind of task. Test design and execution means scanning the product and looking for problems. Bug investigation and reporting is what happens once the tester stumbles into behavior that looks like it might be a problem. Session setup is anything else testers do that makes the first two tasks possible, including tasks such as configuring equipment, locating materials, reading manuals, or writing a session report
• We also ask testers to report the portion of their time they spend “on charter” versus “on opportunity”. Opportunity testing is any testing that doesn’t fit the charter of the session. Since we’re in doing exploratory testing, we remind and encourage testers that it’s okay to divert from their charter if they stumble into an off-charter problem that looks important.
• Although these metrics can provide better visibility and insight about what we’re doing in our test process, it’s important to realize that the session-based testing process and associated metrics could easily be distorted by a confused or biased test manager. A silver-tongued tester could bias the sheets and manipulate the debriefing in such a way as to fool the test manager about the work being done. Even if everyone is completely sober and honest, the numbers may be distorted by confusion over the reporting protocol, or the fact that some testers may be far more productive than other testers. Effective use of the session sheets and metrics requires continual awareness about the potential for these problems.
• One colleague of mine, upon hearing me talk about this approach, expressed the concern that senior testers would balk at all the paperwork associated with the session sheets. All that structure, she felt, would just get in the way of what senior testers already know how to do. Although my first instinct was to argue with her, on second thought, she was giving me an important reality check. This approach does impose a structure that is not strictly necessary in order to achieve the mission of good testing. Segmenting complex and interwoven test tasks into distinct little sessions is not always easy or natural. Session-based test management is simply one way to bring more accountability to exploratory testing, for those situations where accountability is especially important.