Cypress: Printing Information to the CLI

Most people will run tests written in Cypress via the built-in test runner. It provides much of the information people need when looking at tests and checking whether it is doing what it is supposed to do – test status, command logs, app preview for each test run, plus snapshots of test steps. Very useful when looking at test failure points.

Some people, however, like to run Cypress tests using the terminal or the command-line interface. Sometimes, we don’t necessarily have to see the test running over the app UI. For one, I can start writing a draft of a new set of tests while the existing test suite runs in the background; I will want to know if there are failing tests, but not get distracted by them running on the test runner. That’s how tests run on a CI server too; we just want the test results.

Cypress works well for both scenarios. There’s just less information displayed on the CLI from running tests by default. To help us debug failing tests without opening the test runner, we could print out information we need from the test as it runs. If we want, we can show network request and response pairs, or simple variable values. Often, I find that printing out the URL at certain point for some tests helps me understand test results better. I can also click that URL to redirect me to the page afterwards if there are details I want to check out later for myself.

In order to print that URL to the terminal, we’ll need to add a task to Cypress as a plugin. To do that, we open the plugins/index.js file and add a log task there, like so:

After that, we can now use the log task inside our tests. Here’s an example:

Advertisements

Trying Out Cypress with Circle CI

Circle CI has been around a while but I’ve never tried their service before. There hasn’t been much of a reason to do so; at work I mostly stuck with a local Jenkins instance because it got the job done. But I recently had the urge to try it out for a private project that I have. I wanted to see if it plays well with Cypress, a test tool I’ve been using for some time.

A search tells me that there’s already a Cypress-Docker-CircleCI example on Github, from Cypress themselves. Cool, I just need to copy their settings to my own project and make changes accordingly.

Here’s how the package.json file looks like:

You’ll see a cy:circle-junit command in there, which does the same thing as cy:run (which runs tests on a terminal) but also builds a JUnit report.

To run our tests on Circle CI, we’re going to need a circle.yml file inside our project, which contains the following:

From this file Circle CI is supposed to checkout the test code, run the cy:circle-junit command, which installs Cypress and other dependencies inside a Docker image, saves that image in a cache (and restores that cache for every succeeding test runs as long as the dependencies remain the same), runs our tests inside a container, creates a simple report, and stores test artifacts afterwards. Looks straightforward. 🙂

Now let’s see if things work!

Connecting my Cypress project on Circle CI and pushing code changes on the remote repository automatically creates test runs that shows up on a neat dashboard like so:

And we can zoom in on the details of every test run, which looks like this:

Awesome! Looks like I’ll continue using their service for this project, and maybe on other projects as well. I’m off to adding proper tests to the suite!

 

Cypress and Mochawesome

A week ago I was working on a quick automation project which asks for an HTML report of the test results to go along with the scripts. It was a straightforward request, but it was something that I don’t usually generate. Personally I find test results being displayed in a terminal to be enough, but for the said task I needed a report generator. I had already decided to use Cypress for the job so I needed something that plays well with it. In their docs I found a custom reporter called Mochawesome.

To install it, I updated the package.json file inside the project directory to include the reporter:

The Cypress documentation on Reporters also said that for Mochawesome to properly work I should also install mocha as a dev dependency, so that’s what I did.

And then run npm install on the terminal to actually install the reporter.

Before running tests, there’s a tiny change we need to write on the cypress.json file inside the project directory, which tells Cypress which reporter do we want to use for generating test reports.

And we’re all set after all that. 🙂

Run Cypress tests by running cypress run --reporter mochawesomeOr if you specified a script in the package.json file the same way I did in the first photo above, just run npm test.

After running tests, we’re going to find out that a mochawesome-report directory has been added to our project directory which houses both HTML and JSON reports of the tests.

A sample HTML test report looks something like this:

Looks nice and simple and ready for archiving.

Cypress: Stubbing Network Requests with Cy.Route()

I very much enjoy testing web apps by simulating their network requests via code. This allows me to visit websites, login, and replicate functionality, all without a browser, a slightly different sort of testing than many testers are accustomed to. I love to explore what’s happening under the hood when we click elements and submit forms, I like to play with cookies and payloads, I try to find out what bare minimum of data do I need to pass through HTTP requests to recreate a particular user behavior. People often do this kind of testing with Postman but I’ve been accustomed to implementing tests with Ruby and the rest-client gem. Recently though I looked at how Cypress plays with network requests, especially curious about how they take it further with their built-in request stubbing feature using cy.route() because I have never tried stubbing before.

First, some context on HTTP requests:

  • GET requests often simulate visiting (or redirecting to) a web page or retrieving a resource (like an image or another file)
  • POST requests often simulate a form submission, like logins or payments, and as such deals with passing inputted data in order to proceed to the next application state
  • There are other types of HTTP requests but mastering how these two work is enough at the start

And here is an example of how Cypress helps you perform a said GET request:

and for a POST request:

Pretty straightforward and easy to follow. Notice that POST requests have more information in them than GET requests, since we’re passing data – the body field is concerned with user inputs while the headers field is concerned with the user session, among other things. Of course, both requests need a url field, some place to send the request to.

And when we send a request, we receive a response. That response tells us about how a web application behaved after the request – was the user redirected to another page? was the user able to log in? did an expected web element got displayed or hidden? were we sent to an error page, perhaps?

Cypress takes network requests further by introducing routing to testing. Here’s an example:

What the above code says is that we want to use Cypress as a server and we want to wait and listen for a POST request that’s going to the /login URL after a submit button (with an id of #submitButton) is clicked, after which we want to respond with a { success: false } result. This means that the actual response from our application from that url is going to be taken over by a fake response that we designed ourselves. This is what stubbing a network request looks like.

Now why would we want to do this? Some reasons:

  • We want to check how an application behaves for scenarios where a request fails to reach the application server. Do we redirect the user? Or do we show an error popup? Or does the application also work offline? To do this without stubbing, we would need some help from a programmer to shut down the app server at the right time after we perform the scenario.
  • We want to speed up tests by stubbing the response of some requests with less data than the actual responses deliver. We can even have the response data to be empty, if we don’t necessarily need that specific data for a test.
  • We want to see what happens to the application when it receives an incorrect response value from a request.

This is one thing I am loving about Cypress. Out of the box, they allow me to play with network requests alongside testing the user interface, and lets me tinker with it some more.

6 Curious New Tools to Try for Writing Automated Checks for Browser Apps

While I don’t find myself writing a lot of browser-based automated checks these days, I still am on the look out for interesting new tools in that space. The reason: the new tool solves an existing problem I have with setting up such a testing suite from scratch or provides a solution for certain curious use cases I’ve never experienced before. While using Ruby and Watir together in writing tests running through the browser for me is sufficient for common tasks, such a new tool could be a better fit for another project.

Here’s a list of such tools that popped up in my feed in recent months:

  • Cypress. What I like about Cypress, aside from the standalone package installation option and the built-in pretty test report page, is that the pre-defined browser tests that the actual team runs on its own site is included out-of-the-box. This way they made it easy for me to write custom tests; I just had to search for an example of what I wanted to do, copy-pasted it to my own test, and updated the parts that needed changing. Tests are written in Javascript. I have yet to try running the tests via the terminal though, which is important when running tests on a CI server. Using their test runner is free for all projects, however there is a pricing plan for using their dashboard service which helps keep test recordings private.
  • Katalon Studio. This is a full-blown automation solution that is completely free. There’s a pricing plan for business support services. The record-and-playback feature built-in to the tool failed to impress me when I ran it through our legacy apps, but perhaps writing the actual test code through their GUI fares better (using which there will be a high learning curve for people like me who like to use the CLI and personally-configured IDEs).
  • PuppeteerBuild to control Google’s headless Chrome or Chromium browser, running over the DevTools protocol. Tests are written in Javascript. Easy to try and get into using their web playground. Alister Scott has tried it running with Mocha and Circle CI on a demo project.
  • Chromeless. Similar to Puppeteer, but built to automate an army of Chrome browsers running in parallel. It gives us the option to run tests on AWS Lamba too. Again, tests are written in Javascript, which we can try on their demo playground.
  • Laravel Dusk. This gives PHP developers familiar with Laravel the ability to write and run their own browser app tests, using a programming language they’re much accustomed to.
  • Appraise. Similar to BackstopJS, a tool for visually validating browsers apps. Tests are written in Markdown.