Cypress: Printing Information to the CLI

Most people will run tests written in Cypress via the built-in test runner. It provides much of the information people need when looking at tests and checking whether it is doing what it is supposed to do – test status, command logs, app preview for each test run, plus snapshots of test steps. Very useful when looking at test failure points.

Some people, however, like to run Cypress tests using the terminal or the command-line interface. Sometimes, we don’t necessarily have to see the test running over the app UI. For one, I can start writing a draft of a new set of tests while the existing test suite runs in the background; I will want to know if there are failing tests, but not get distracted by them running on the test runner. That’s how tests run on a CI server too; we just want the test results.

Cypress works well for both scenarios. There’s just less information displayed on the CLI from running tests by default. To help us debug failing tests without opening the test runner, we could print out information we need from the test as it runs. If we want, we can show network request and response pairs, or simple variable values. Often, I find that printing out the URL at certain point for some tests helps me understand test results better. I can also click that URL to redirect me to the page afterwards if there are details I want to check out later for myself.

In order to print that URL to the terminal, we’ll need to add a task to Cypress as a plugin. To do that, we open the plugins/index.js file and add a log task there, like so:

After that, we can now use the log task inside our tests. Here’s an example:

Help Yourself Perform Tasks With Personal Shortcut Commands

In a recent automation work I’ve been asked to take part in, running a single test safely, according to peers, generally follow a three-step process:

1. Delete existing feature files:  rm -rf *.feature
2. Create updated feature files:  PROJECT_DIRECTORY=../<dir> APP=<app> TEST=automated ruby create_tests.rb, where app is the name of the app you want to test
3. Run the desired specific test:  cucumber --tags @<tag>

This means I have to re-run these three steps over and over for every single test I want to review. That can get tiring easily, especially if I have to re-type them repeatedly, or even if I copy-paste. Using the arrow keys in the terminal helps, but sometimes commands can get lost in the history and then it becomes a bit of a hassle to find them.

There should be a way for me to run an automated check I want to review, given just the app name and the test tag. It would be nice if I can run a specific test using a shorter command but still following the same process as provided.

I decided to set aside a little bit of time to write a personal script, and used Makefile because I’ve had experience with that before.

My personal short command for running a test:  make test app=<app> tag=<tag>

And here’s what the script looks like, which runs the known commands step-by-step plus displays some information on what’s currently happening:

Now I won’t have to worry about remembering what commands to type in as well as the sequence of the commands. I can give more focus on actually reviewing the test itself, which is what’s more important.

Is There Really Nothing We Can Do?

“I’ll wait for the code to get pushed to the Staging server before I continue testing because the data there is a lot more stable.”

“It’d be a waste of time trying to teach people how to program when it feels like they don’t have the motivation for it.”

“There’s a high chance things will break after a code merge with the production branch. It’s been that way ever since. We’ll just have to fix those that we can until the release.”

“I wish there was an easy way to spin up a version of our apps in a local environment. That should help us test things faster.”

…

Some problems become the status quo. It’s worth revisiting them, asking ourselves whether there’s really nothing we can do about it or we’ve just become comfortable with complaining.

Lessons from Richard Bach’s “Jonathan Livingston Seagull”

I’ve always kept a copy of Richard Bach’s “Jonathan Livingston Seagull” within reach, for those days when it seems necessary to remind myself of Jonathan’s story – of struggle, of questions and learning, of constantly challenging himself, and of keeping true to his curiosities. I’ve re-read it a number of times in the past several years (it’s a short book, digestible in a few hours), and it has never failed to uplift my spirits. It always feels like talking to a great friend that I’ve never seen for an extended period of time.

Here are some favorite lines from the book:

• Do you have any idea how many lives we must have gone through before we even got the first idea that there is more to life than eating, or fighting, or power in the Flock? A thousand lives, Jon, ten thousand! And then another hundred lives until we began to learn that there is such a thing as perfection, and another hundred again to get the idea that our purpose for living is to find that perfection and show it forth. The same rule holds for us now, of course: we choose our next world though what we learn in this one. Learn nothing, and the next world is the same as this one, all the same limitations and lead weights to overcome.
• Jonathan Seagull discovered that boredom and fear and anger are the reasons that a gull’s life is so short, and with these gone from his thought, he lived a long fine life indeed.
• “You can go to any place and to any time that you wish to go,” the Elder said. “I’ve gone everywhere and everywhere I can think of.” He looked across the sea. “It’s strange. The gulls who scorn perfection for the sake of travel go nowhere, slowly. Those who put aside travel for the sake of perfection go anywhere, instantly. Remember, Jonathan, heaven isn’t a place or a time, because place and time are so very meaningless.”
• “To fly as fast as he thought, to anywhere that is,” he said, “you must begin by knowing that you have already arrived.”
• “Don’t believe what your eyes are telling you. All they show is limitation. Look with your understanding, find out what you already know, and you’ll see the way to fly.”
• “Look at Fletcher! Lowell! Charles-Roland! Judy Lee! Are they also special and gifted and divine? No more than you are, no more than I am. The only difference, the very only one, is that they have begun to understand what they really are and have begun to practice it.”
• “Why is it,” Jonathan puzzled, “that the hardest thing in the world is to convince a bird that he is free, and that he can prove it for himself if he’d just spend a little time practicing? Why should that be hard?”
• “Oh, Fletch, you don’t love that! You don’t love hatred and evil, of course. You have to practice and see the real gull, the good in every one of them, and to help them see it in themselves. That’s what I mean by love. It’s fun, when you get the knack of it.”
• We’re free to go where we wish and to be what we are.

Trying Out Cypress with Circle CI

Circle CI has been around a while but I’ve never tried their service before. There hasn’t been much of a reason to do so; at work I mostly stuck with a local Jenkins instance because it got the job done. But I recently had the urge to try it out for a private project that I have. I wanted to see if it plays well with Cypress, a test tool I’ve been using for some time.

A search tells me that there’s already a Cypress-Docker-CircleCI example on Github, from Cypress themselves. Cool, I just need to copy their settings to my own project and make changes accordingly.

Here’s how the package.json file looks like:

You’ll see a cy:circle-junit command in there, which does the same thing as cy:run (which runs tests on a terminal) but also builds a JUnit report.

To run our tests on Circle CI, we’re going to need a circle.yml file inside our project, which contains the following:

From this file Circle CI is supposed to checkout the test code, run the cy:circle-junit command, which installs Cypress and other dependencies inside a Docker image, saves that image in a cache (and restores that cache for every succeeding test runs as long as the dependencies remain the same), runs our tests inside a container, creates a simple report, and stores test artifacts afterwards. Looks straightforward. 🙂

Now let’s see if things work!

Connecting my Cypress project on Circle CI and pushing code changes on the remote repository automatically creates test runs that shows up on a neat dashboard like so:

And we can zoom in on the details of every test run, which looks like this:

Awesome! Looks like I’ll continue using their service for this project, and maybe on other projects as well. I’m off to adding proper tests to the suite!

 

Lessons from Richard Bach’s “Life with my Guardian Angel”

The new year is always a great time to remember to talk to our guardian angel, to look at what beliefs do we have come to accept, and to remind ourselves of which sort of heaven do we really want to go to. 🙂 Happy New Year!

Some favorite lines from the book:

• “You’ll never learn it all,” he said. “Nobody has learned everything about flying and they never will. But if you practice, you can learn enough to be a good pilot, even if you don’t know everything.”
• Every context, every subtext means the same: heaven will be joy when we reach that place of mind, whether or not we reach its borders alive, or not-alive.
• All the books I’ve read about death and dying, filled with stories and accounts and proofs that we go on after this lifetime, how can my heaven be exactly the same place that it is for everyone who’s died in the last thousand years or so? Is there one place in paradise where some folks enjoy cigars, a different heaven where others travel instantly to other places, others have houses, others don’t need shelter from winds or rains, which don’t exist in their heaven? Does heaven require space and time? Can heaven be kind of a dream, where we imagine space and time, but know that space-time doesn’t exist? How many heavens can there be? Two? Twenty? A million heavens, an indefinite number of heavens? Not just for mortals, but for every expression of life and love who recovers from the belief of dying? Do mortals share the same heaven with animals, with bees? There must be a heaven for dogs and cats who love people, and different heavens for dogs, cats, and other animals who prefer to continue without any humans at all. There must be a heaven for chipmunks and a separate heaven for crocodiles, and different ones for recently arrived spirits who’ve spent a lifetime thinking that no such place exists. There must be separate heavens to take care of the people who believe the various religions. Then different heavens for the different ways of thinking. Separate heavens.
• Don’t you know? What you call ‘death’ is your chance to make different choices, make different stories about life and love.
• What will your mortal’s life be like without tests, without adventure, without risk? Does the word ‘graveyard’ come to mind?
• Your mission in every one of your lifetimes is to create love, to express love, to treasure love, to paint love in the biggest letters in every color of every language you will ever learn!
• The world of space, time, and appearances can be wondrous beautiful. Have care, though, and don’t let your mortal mistake them for real.
• You can work right now, today, as a mortal, to refine your beliefs, to make your heaven happier than the squirrelly heaven you’ve already built with your not-much-thought-about belief system. You can do this now.
• We’re connected by the bonds of love that we’ve built. With anyone. With our parents, with our friends, with our pets, with fictional characters from books or motion pictures, with characters in our own imagination!

Notes from Jonathan Bach’s “Session-Based Test Management”

Tracking exploratory testing work is difficult for test managers. We don’t want to micro-manage testers, we want them to explore to their hearts content when they test, but we won’t know how much progress there is if we don’t track the work. We also won’t know what sort of problems testers encounter during testing, unless they have the nerve to tell us immediately. Jonathan Bach’s “Session-Based Test Management” article has one suggestion: use sessions, uninterrupted blocks of reviewable and chartered test effort.

Here are a few favorite notes from the article:

• Unlike traditional scripted testing, exploratory testing is an ad hoc process. Everything we do is optimized to find bugs fast, so we continually adjust our plans to re-focus on the most promising risk areas; we follow hunches; we minimize the time spent on documentation. That leaves us with some problems. For one thing, keeping track of each tester’s progress can be like herding snakes into a burlap bag.
• The first thing we realized in our effort to reinvent exploratory test management was that testers do a lot of things during the day that aren’t testing. If we wanted to track testing, we needed a way to distinguish testing from everything else. Thus, “sessions” were born. In our practice of exploratory testing, a session, not a test case or bug report, is the basic testing work unit . What we call a session is an uninterrupted block of reviewable, chartered test effort. By “chartered,” we mean that each session is associated with a mission—what we are testing or what problems we are looking for. By “uninterrupted,” we mean no significant interruptions, no email, meetings, chatting or telephone calls. By “reviewable,” we mean a report, called a session sheet, is produced that can be examined by a third-party, such as the test manager, that provides information about what happened.
• From a distance, exploratory testing can look like one big amorphous task. But it’s actually an aggregate of sub-tasks that appear and disappear like bubbles in a Jacuzzi. We’d like to know what tasks happen during a test session, but we don’t want the reporting to be too much of a burden. Collecting data about testing takes energy away from doing testing.
• We separate test sessions into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We call these the “TBS” metrics. We then ask the testers to estimate the relative proportion of time they spent on each kind of task. Test design and execution means scanning the product and looking for problems. Bug investigation and reporting is what happens once the tester stumbles into behavior that looks like it might be a problem. Session setup is anything else testers do that makes the first two tasks possible, including tasks such as configuring equipment, locating materials, reading manuals, or writing a session report
• We also ask testers to report the portion of their time they spend “on charter” versus “on opportunity”. Opportunity testing is any testing that doesn’t fit the charter of the session. Since we’re in doing exploratory testing, we remind and encourage testers that it’s okay to divert from their charter if they stumble into an off-charter problem that looks important.
• Although these metrics can provide better visibility and insight about what we’re doing in our test process, it’s important to realize that the session-based testing process and associated metrics could easily be distorted by a confused or biased test manager. A silver-tongued tester could bias the sheets and manipulate the debriefing in such a way as to fool the test manager about the work being done. Even if everyone is completely sober and honest, the numbers may be distorted by confusion over the reporting protocol, or the fact that some testers may be far more productive than other testers. Effective use of the session sheets and metrics requires continual awareness about the potential for these problems.
• One colleague of mine, upon hearing me talk about this approach, expressed the concern that senior testers would balk at all the paperwork associated with the session sheets. All that structure, she felt, would just get in the way of what senior testers already know how to do. Although my first instinct was to argue with her, on second thought, she was giving me an important reality check. This approach does impose a structure that is not strictly necessary in order to achieve the mission of good testing. Segmenting complex and interwoven test tasks into distinct little sessions is not always easy or natural. Session-based test management is simply one way to bring more accountability to exploratory testing, for those situations where accountability is especially important.

Flutter Basics: HTTP Post Requests

Myself and the rest of Team Tiger (from the Flutter dev internship program we’ve recently joined) have just been handed a challenge to work with a simple API, to send HTTP requests and receive responses which a mobile app should understand. I’ve always been fond of reading and understanding web network requests, and knew the challenge would be fun.

If we want to work with HTTP requests, we need to import several dependencies:

The dart packages are built into Flutter, but we also need an external http package, which we have to include in our pubspec.yaml file, like so:

And now we can try sending post requests through our mobile app! Here’s an example code of asynchronously deleting something through an API:

The code is straightforward, sending a .post() method containing the URL of where we want to perform an activity and a  body field which tells us what information we need to send to said URL, all inside an asynchronous Future. We specify that we want to close the HTTP client after the POST request gets completed by sending a client.close command inside the .whenComplete() method.

We can change the app state by running desired commands inside the setState() method. In the example code above, we’re just printing the response received from the POST request. What you want to do about that response is up to you. You can also return the response to someplace in the app if you want.

The if (!mounted) clause is there just to guard us from situations where the user suddenly moves to another part of the application while the POST request is still running. In that case, the request immediately stops and sends a desired return.

Takeaways from Timothy Ferriss’ “The 4-Hour Body”

This year, I decided I was going to get better at exercising. To do that, I thought about reading a few books to give myself an idea on how to go about it. One such book was Timothy Ferriss’ “The 4-Hour Body“, which is a goldmine of content. In it are suggested exercises that gets the job done, walkthroughs, and some science of how things work. But the book is so much more than just a guide on physical exercises. There’s ideas on self-experimentation, adherence, being proactively skeptical, harajuku moments, a slow-carb diet, and more.

Here are some favorite takeaways:

• Science starts with educated (read: wild-ass) guesses. Then it’s all trial and error. Sometimes you predict correctly from the outset. More often, you make mistakes and stumble across unexpected findings, which lead to new questions. If you want to sit on the sidelines and play full-time skeptic, suspending actions until a scientific consensus is reached, that’s your choice. But don’t use skepticism as a thinly veiled excuse for inaction or remaining in your comfort zone. Be skeptical, but for the right reason: because you’re looking for the most promising option to test in real life. Be proactively skeptical, not defensively skeptical.
• We break commitments to ourselves with embarrassing regularity. How can someone trying to lose weight binge on an entire pint of ice cream before bed? How can even the most disciplined of executives fail to make 30 minutes of time per week for exercise? How can someone whose marriage depends on quitting smoking pick up a cigarette? Simple: logic fails.
• Take adherence seriously: will you actually stick with this change until you hit your goal? If not, find another method, even if it’s less effective and less efficient. The decent method that you follow is better than the perfect method you quit.
• Self-experimentation can be used by non-experts to (a) see if the experts are right and (b) learn something they don’t know. When you study your own problem (e.g. acne), you care more about finding a solution than others are likely to care.
• If results are fast and measurable, self-discipline isn’t needed.
• If you want to be more confident or effective, rather than relying on easily-defeated positive thinking and mental gymnastics, learn to run faster, lift more than your peers, or lose those last ten pounds. It’s measurable, it’s clear, you can’t lie to yourself. It therefore works. The Cartesian separation of mind and body is false. They’re reciprocal. Start with the precision of changing physical reality and a domino effect will often take care of the internal.
• Job not going well? Company having issues? Some idiot making life difficult? If you add ten laps to your swimming, or if you cut five seconds off your best mile time, it can still be a great week. Controlling your body puts you in life’s driver’s seat.
• Recreation is for fun. Exercise is for producing changes. Don’t confuse the two.
• People suck at following advice. Even the most effective people in the world are terrible at it. There are two reasons:
  • Most people have an insufficient reason for action. The pain isn’t painful enough. It’s a nice-to-have, not a must-have.
  • There are no reminders. No consistent tracking = no awareness = no behavioral change. Consistent tracking, even if you have no knowledge of fat-loss or exercise, will often beat advice from world-class trainers.
• For a long time, I’ve known that the key to getting started down the path of being remarkable in anything is to simply act with the intention of being remarkable. If I want a better-than-average career, I can’t simply go with the flow and get it. Most people do just that: they wish for an outcome but make no intention-driven actions toward that outcome. If they would just do something most people would find that they get some version of the outcome they’re looking for. That’s been my secret. Stop wishing and start doing.

Web Accessibility Visualization Tool: tota11y

Testing web sites and apps come in many forms. Testers try their best to test everything, but obviously there’s only so much they can do within a schedule. Some forms of testing are more prioritized than others, and that’s not inherently bad; for a solo tester on a team, one usually tests in a way that covers more bases at the beginning.

Web accessibility testing is one of those forms of testing that often takes a backseat, sometimes even forgotten. Web accessibility helps people with disabilities get better surfing experience. Although websites are typically not built with such functionality in mind, it matters.

And tota11y is a tool from Khan Academy we can leverage for testing accessibility. It is available as an easy-to-use bookmarklet. For whatever page we want to test, we just need to go there and click the bookmarklet, after which the tool will appear on the bottom left corner of the page. Clicking the tool reveals options and using each helps us spot common accessibility violations.

Here are some screenshots of using it on a page I test at work, checking headings, contrast, and link text:

Spotted: Nonconsecutive heading level use

Multiple insufficient contrast ratio violations

And unclear link texts

Looks like there’s room for improvement, although these violations are not necessarily errors.