Notes from David Bryant Copeland’s “Build Awesome Command-Line Applications in Ruby 2”

The experience of writing and running automated checks, as well as building some personal apps that run on the terminal, in recent years, has given me a keen sense of appreciation on how effective command-line applications can be as tools. I’ve grown fond of quick programming experiments (scraping data, playing with Excel files, among others), which are relatively easy to write, powerful, dependable, and maintainable. Tons of libraries online help interface well with the myriad of programs in our desktop or out in the web.

Choosing to read “Build Awesome Command-Line Applications in Ruby 2” is choosing to go on an adventure about writing better CLI apps, finding out how options are designed and built, understanding how they are configured and distributed, and learning how to actually test them.

Some notes from the book:

  • Graphical user interfaces (GUIs) are great for a lot of things; they are typically much kinder to newcomers than the stark glow of a cold, blinking cursor. This comes at a price: you can get only so proficient at a GUI before you have to learn its esoteric keyboard shortcuts. Even then, you will hit the limits of productivity and efficiency. GUIs are notoriously hard to script and automate, and when you can, your script tends not to be very portable.
  • An awesome command-line app has the following characteristics:
    • Easy to use. The command-line can be an unforgiving place to be, so the easier an app is to use, the better.
    • Helpful. Being easy to use isn’t enough; the user will need clear direction on how to use an app and how to fix things they might’ve done wrong.
    • Plays well with others. The more an app can interoperate with other apps and systems, the more useful it will be, and the fewer special customizations that will be needed.
    • Has sensible defaults but is configurable. Users appreciate apps that have a clear goal and opinion on how to do something. Apps that try to be all things to all people are confusing and difficult to master. Awesome apps, however, allow advanced users to tinker under the hood and use the app in ways not imagined by the author. Striking this balance is important.
    • Installs painlessly. Apps that can be installed with one command, on any environment, are more likely to be used.
    • Fails gracefully. Users will misuse apps, trying to make them do things they weren’t designed to do, in environments where they were never designed to run. Awesome apps take this in stride and give useful error messages without being destructive. This is because they’re developed with a comprehensive test suite.
    • Gets new features and bug fixes easily. Awesome command-line apps aren’t awesome just to use; they are awesome to hack on. An awesome app’s internal structure is geared around quickly fixing bugs and easily adding new features.
    • Delights users. Not all command-line apps have to output monochrome text. Color, formatting, and interactive input all have their place and can greatly contribute to the user experience of an awesome command-line app.
  • Three guiding principles for designing command-line applications:
    • Make common tasks easy to accomplish
    • Make uncommon tasks possible (but not easy)
    • Make default behavior nondestructive
  • Options come in two-forms: short and long. Short-form options allow frequent users who use the app on the command line to quickly specify things without a lot of typing. Long-form options allow maintainers of systems that use our app to easily understand what the options do without having to go to the documentation. The existence of a short-form option signals to the user that that option is common and encouraged. The absence of a short-form option signals the opposite— that using it is unusual and possibly dangerous. You might think that unusual or dangerous options should simply be omitted, but we want our application to be as flexible as is reasonable. We want to guide our users to do things safely and correctly, but we also want to respect that they know what they’re doing if they want to do something unusual or dangerous.
  • Thinking about which behavior of an app is destructive is a great way to differentiate the common things from the uncommon things and thus drive some of your design decisions. Any feature that does something destructive shouldn’t be a feature we make easy to use, but we should make it possible.
  • The future of development won’t just be manipulating buttons and toolbars and dragging and dropping icons to create code; the efficiency and productivity inherent to a command-line interface will always have a place in a good developer’s tool chest.
Advertisements

Favorite Talks from Agile Testing Days 2017

There are two things that’s wonderful from last year’s Agile Testing Days conference talks: content focusing on other valuable stuff for testers and teams (not automation), and having as many women speakers as there are men. I hope they continue on with that trend.

Here’s a list of my favourite talks from said conference (enjoy!):

  • How To Tell People They Failed and Make Them Feel Great (by Liz Keogh, about Cynefin, our innate dislike of uncertainty and love of making things predictable, putting safety nets and allowing for failure, learning reviews, letting people change themselves, building robust probes, and making it a habit to come from a place of care)
  • Pivotal Moments (by Janet Gregory, on living in a dairy farm, volunteering, traveling, toastmasters, Lisa Crispin, Mary Poppindieck and going on adventures, sharing failures and taking help, and reflecting on pivotal moments)
  • Owning Our Narrative (by Angie Jones, on the history of the music industry so far, the changes in environment, tools, and business models musicians have had to go through so survive, and embracing changes and finding ways to fulfil our roles as software testers)
  • Learning Through Osmosis (by Maaret Pyhäjärvi, on mob programming and osmosis,  creating safe spaces to facilitate learning, and the power of changing some of our beliefs and behaviour)
  • There and Back Again: A Hobbit’s/Developer’s/Tester’s Journey (by Pete Walen, on how software was built in the old days, how testing and programming broke up into silos, and a challenge for both parties to go back at excelling at each other’s skills and teaming up
  • 10 Behaviours of Effective Agile Teams (by Rob Lambert, about shipping software and customer service, becoming a more effective employee, behaviours, and communicating well)

Cypress: Stubbing Network Requests with Cy.Route()

I very much enjoy testing web apps by simulating their network requests via code. This allows me to visit websites, login, and replicate functionality, all without a browser, a slightly different sort of testing than many testers are accustomed to. I love to explore what’s happening under the hood when we click elements and submit forms, I like to play with cookies and payloads, I try to find out what bare minimum of data do I need to pass through HTTP requests to recreate a particular user behavior. People often do this kind of testing with Postman but I’ve been accustomed to implementing tests with Ruby and the rest-client gem. Recently though I looked at how Cypress plays with network requests, especially curious about how they take it further with their built-in request stubbing feature using cy.route() because I have never tried stubbing before.

First, some context on HTTP requests:

  • GET requests often simulate visiting (or redirecting to) a web page or retrieving a resource (like an image or another file)
  • POST requests often simulate a form submission, like logins or payments, and as such deals with passing inputted data in order to proceed to the next application state
  • There are other types of HTTP requests but mastering how these two work is enough at the start

And here is an example of how Cypress helps you perform a said GET request:

and for a POST request:

Pretty straightforward and easy to follow. Notice that POST requests have more information in them than GET requests, since we’re passing data – the body field is concerned with user inputs while the headers field is concerned with the user session, among other things. Of course, both requests need a url field, some place to send the request to.

And when we send a request, we receive a response. That response tells us about how a web application behaved after the request – was the user redirected to another page? was the user able to log in? did an expected web element got displayed or hidden? were we sent to an error page, perhaps?

Cypress takes network requests further by introducing routing to testing. Here’s an example:

What the above code says is that we want to use Cypress as a server and we want to wait and listen for a POST request that’s going to the /login URL after a submit button (with an id of #submitButton) is clicked, after which we want to respond with a { success: false } result. This means that the actual response from our application from that url is going to be taken over by a fake response that we designed ourselves. This is what stubbing a network request looks like.

Now why would we want to do this? Some reasons:

  • We want to check how an application behaves for scenarios where a request fails to reach the application server. Do we redirect the user? Or do we show an error popup? Or does the application also work offline? To do this without stubbing, we would need some help from a programmer to shut down the app server at the right time after we perform the scenario.
  • We want to speed up tests by stubbing the response of some requests with less data than the actual responses deliver. We can even have the response data to be empty, if we don’t necessarily need that specific data for a test.
  • We want to see what happens to the application when it receives an incorrect response value from a request.

This is one thing I am loving about Cypress. Out of the box, they allow me to play with network requests alongside testing the user interface, and lets me tinker with it some more.

On 100% Coverage

Yes, we need to write tests because it is something that we think will help us in the long term, even though it may be more work for us in the short run. If written with care and with the end in mind, tests serve as living documentation, living because they change as much as the application code changes, and they help us refer back to what some feature does and doesn’t, in as much detail as we want. Tests let us know which areas of the application matters to us, and every time they run they remind us of where our bearings currently are.

Tests may be user journeys in the user interface, a simulation of requests and response through the app’s API, or small tests within the application’s discrete units, most likely a combination of all these types of tests, perhaps more. What matters is that we find some value in whatever test we write, and that value merits its cost of writing and maintenance. What’s important is asking ourselves whether the test is actually significant enough to add to the test suite.

It is valuable to build a good enough suite of tests. It makes sense to add more tests as we find more key scenarios to exercise. It also makes sense to remove tests that were necessary in the past but aren’t anymore. However, I don’t think it is particularly helpful to advocate for 100% test coverage, because that brings the focus on a numbers game, similar to how measuring likes or stars isn’t really the point. I believe it is better when we discuss among ourselves, in the context we’re in, which tests are relevant and which are just diving into minutiae. If our test suite helps us deploy our apps with confidence, if our tests allows us to be effective in the performance of our testing, if we are continuously able to serve our customers as best as we can, then the numbers really doesn’t amount to much.

Doing Things Right, And Doing The Right Things

In testing software, automation is a tool that helps us re-run whatever repeatable checks we have on an application under test. We automate because we never have enough time to re-test everything by hand, and exploring the unknown parts of the apps we test is a far better use of our testing skills than following scripts. To automate is doing one thing right, within context, if it provides us the feedback we need. And that feedback we think we need from automation depends on what suites of tests are best repeated again and again, as well as what sort of tests costs more than the value they give.

There’s also tons of tools that helps us build good quality software. Even though we build more complex applications now than before, we have frameworks, libraries, intelligent IDEs, and other tools to help us spin up apps on a whim now too, ready to be modified as we see fit. Choosing the proper tools for the job is doing another thing right. But before we write any code, we need to be sure about the actual problem we are solving for our customer.

Yes, we need to do things right, from the get go if possible. They help us progress from one point to another faster than otherwise. However, I think it’s more important that we continuously take the necessary time to review whether we are doing the right things too, more important to actually get feedback and solve problems than merely adding tests and features.

Dockerizing Our Legacy Apps: Some Notes

I spent the recent weeks of January tinkering with Docker in both Windows 7 and Mac OS. I played with it a lot because I thought it’s something that’s useful for a grand new project we have at work, and I thought that integrating our legacy application code to it would help me learn about it more. And the exercise did help me understand the tool better, including some nuances with application performance and database connections. I was able to dockerize our legacy apps too! 🙂

Some notes to remember related to the exercise:

  • Windows 7 Docker Toolbox and Docker for Mac has a performance issue with volume mounts through docker-compose. Legacy apps composed of a large number of files (especially with dependency directories) will run, but they will be painfully slow out-of-the-box. Fortunately for Docker for Mac users, docker-sync has an effective workaround for this problem. It involves running an in-sync container for the application code, separate from the docker-compose file. Unfortunately, I have not found any workarounds for said performance issue for Windows 7 (and perhaps Windows 8) Docker Toolbox users.
  • Often we have to update the host file of our server machine so that we can run applications locally using a distinct, easy-to-remember URL through a browser. This means we need to add extra hosts to necessary docker containers too if we dockerize our apps. We can do this by using the extra_hosts command in docker-compose.
  • The official Postgresql docker container does not include the pdo, pdo_pgsql, and pgsql drivers, which handles the connection between the application and the database. To install those drivers inside the official container, we’ll need to use a Dockerfile and build it from the docker-compose file with the build and context commands.
  • Sometimes we have a need to copy the Postgresql DB files from a running container to set up a proper volume mount of database data from host to container. We can copy that data by using the convenient docker cp <source> <destination> command. I had this work in Docker for Mac. However, for Windows 7 Docker Toolbox users, a docker container is unable to use such copied data, perhaps because of the difference in OS between host and container, so I had to resort to restoring and backing up data every time I start and stop my application containers.
  • As a tester, what Docker provides me is a convenient tool to test all sorts of interesting application configurations as much as I want to on a single machine, see if the apps break if I changed some service config, and find out which configurations work or not. I can add or remove a new service, or even update an existing service to a new version, like updating PHP from 5.6 to 7.1, and immediately see what impact it has on the apps themselves. These kinds of tests are often left to operations engineers, but I’m glad there is a now a way to do such tests on my own machine, before application changes even reach a dedicated testing server.
  • Even if Docker makes it easy to setup an application development environment from scratch with docker-compose and Dockerfiles, it is still important to maintain a wiki of the necessary machine configurations a programmer needs to perform in order to reset or build the apps with only a single command, or two. Subtle things like custom docker-compose files, .env and php.ini files, host files, Nginx configs, or turning long docker commands into shortcuts with shell scripts or make commands.
  • Makefile tasks can help put specific scripts into an easy-to-remember command with context. I assume Rakefile does the same thing in Ruby, or Jakefile in JavaScript.
  • Dockerizing our legacy apps pushed me on a discussion with programmers about the ways they run said applications on their machines. Most of them actually just test code changes directly on Staging or another available development server. That speaks about one habit we have as a development team, and likely the reason why our apps are a pain to setup locally.

6 Curious New Tools to Try for Writing Automated Checks for Browser Apps

While I don’t find myself writing a lot of browser-based automated checks these days, I still am on the look out for interesting new tools in that space. The reason: the new tool solves an existing problem I have with setting up such a testing suite from scratch or provides a solution for certain curious use cases I’ve never experienced before. While using Ruby and Watir together in writing tests running through the browser for me is sufficient for common tasks, such a new tool could be a better fit for another project.

Here’s a list of such tools that popped up in my feed in recent months:

  • Cypress. What I like about Cypress, aside from the standalone package installation option and the built-in pretty test report page, is that the pre-defined browser tests that the actual team runs on its own site is included out-of-the-box. This way they made it easy for me to write custom tests; I just had to search for an example of what I wanted to do, copy-pasted it to my own test, and updated the parts that needed changing. Tests are written in Javascript. I have yet to try running the tests via the terminal though, which is important when running tests on a CI server. Using their test runner is free for all projects, however there is a pricing plan for using their dashboard service which helps keep test recordings private.
  • Katalon Studio. This is a full-blown automation solution that is completely free. There’s a pricing plan for business support services. The record-and-playback feature built-in to the tool failed to impress me when I ran it through our legacy apps, but perhaps writing the actual test code through their GUI fares better (using which there will be a high learning curve for people like me who like to use the CLI and personally-configured IDEs).
  • PuppeteerBuild to control Google’s headless Chrome or Chromium browser, running over the DevTools protocol. Tests are written in Javascript. Easy to try and get into using their web playground. Alister Scott has tried it running with Mocha and Circle CI on a demo project.
  • Chromeless. Similar to Puppeteer, but built to automate an army of Chrome browsers running in parallel. It gives us the option to run tests on AWS Lamba too. Again, tests are written in Javascript, which we can try on their demo playground.
  • Laravel Dusk. This gives PHP developers familiar with Laravel the ability to write and run their own browser app tests, using a programming language they’re much accustomed to.
  • Appraise. Similar to BackstopJS, a tool for visually validating browsers apps. Tests are written in Markdown.