Automating the Windows Desktop Calculator’s UI using Winium

Ever since I learned how to use Selenium to automate browsers and web applications a few years back, I’ve wondered from time to time whether I can use the same process or technology to automate Windows desktop applications via its user interface. There’s no particular use case at work considering our apps are all in the web, but I challenged myself to find an answer only for curiosity’s sake. It didn’t turn out to be a thought-provoking experiment but at least it was somehow amusing.

Here’s what I got from the quick study:

Click the image to view the GIF full-size on another browser tab 🙂

And a few notes:

  • We can’t use Selenium to automate Windows desktop applications.
  • A quick search about automating Windows apps tells us that there’s actually a number of tools we can use for such purpose, some paid, others free. Other than Winium, I found out that we can also use Sikuli (or its successor SikuliX), WinTask, autoIT, TestStack.White, Automa, MacroSchedulerCobra, or Pywinauto. There’s probably more tools out there. I just chose to use Winium because writing tests and automating the app with it is similar to how Selenium works, which means very little learning curve for me.
  • Source code for this short experiment can be found here: Win-Calculator. It uses Maven for handling installation of project dependencies while the test code is written in Java. The tests run locally, I’m not sure whether there’s a way to manipulate a Windows app on another machine using Winium. I have not researched that far.
  • Winium will actually take control of the machine’s mouse to move to the element locations set in the tests. This means that interrupting the position of the mouse while the tests are running would likely fail the test. This is annoying, and I’m not sure whether the other Windows app automation tools behave otherwise or the same.

Takeaways from Timothy Ferriss’ “The 4-Hour Workweek”

Timothy Ferriss‘ “The 4-Hour Workweek” might be my favorite book this year. Tons of lessons about how to be productive, as well as how to enjoy life to the fullest, and reading the book has left me excited about the future while reconsidering many of my other choices.

Some favorite takeaways:

  • The perfect job is the one that takes the least time. The goal is to free time and automate income.
  • ‘If only I had more money’ is the easiest way to postpone the intense self-examination and decision-making necessary to create a life of enjoyment – now and not later. Busy yourself with the routine of the money wheel, pretend it’s the fix-all, and you artfully create a constant distraction that prevents you from seeing just how pointless it is. Deep down, you know it’s all an illusion, but with everyone participating in the same game of make-believe, it’s easy to forget.
  • What are you waiting for? If you cannot answer this without resorting to the previously rejected concept of good timing, the answer is simple: You’re afraid, just like the rest of the world. Measure the cost of inaction, realize the unlikelihood and repairability of most missteps, and develop the most important habit of those who excel, and enjoy doing so: action.
  • Retirement as a goal or final redemption is flawed for at least three solid reasons:
    1. It is predicated on the assumption that you dislike what you are doing during the most physically capable years of your life.
    2. Most people will never be able to retire and maintain even a hotdogs-for-dinner standard of living. Even one million is chump change in a world where traditional retirement could span 30 years and inflation lowers your purchasing power 2-4% per year. The math doesn’t work.
    3. If the math doesn’t work, it means that you are one ambitious hardworking machine. If that’s the case, guess what? One week into retirement, you’ll be so damn bored that you’ll want to stick bicycle spokes in your eyes.
  • If it isn’t going to devastate those around you, try it and then justify it. Get good at being a troublemaker and saying sorry when you really screw up. Ask for forgiveness, not permission.
  • Ninety-nine percent of people in the world are convinced they are incapable of achieving great things, so they aim for the mediocre. The level of competition is thus fiercest for ‘realistic’ goals, paradoxically making them the most time- and energy-consuming. So do not overestimate the competition and underestimate yourself. You are better than you think.
  • Doing something unimportant well does not make it important. Requiring a lot of time does not make a task important. What you do is infinitely more important than how you do it. Efficiency is still important, but is useless unless applied to the right things.
  • Remember that most things make no difference. Being busy is a form of laziness – lazy thinking and indiscriminate action. Being overwhelmed is often as unproductive as doing nothing and is far more unpleasant. Being selective – doing less – is the path of the productive. Focus on the important few and ignore the rest.
  • If you haven’t identified the mission-critical tasks and set aggressive start and end times for their completion, the unimportant becomes the important. Even if you know what’s critical, without deadlines that create focus, the minor tasks forced upon you (or invented) will swell to consume time until another bit of minutiae jumps in to replace it, leaving you at the end of the day with nothing accomplished.
  • The key to having more time is doing less, and there are two paths to getting there, both of which should be used together:
    1. Define a short to-do list
    2. Define a not-to-do list
  • Don’t ever arrive at the office or in front of your computer without a clear list of priorities. There should be no more than 2 mission-critical items to complete each day. Never. It just isn’t necessary if they’re actually high-impact.

How To See The Thing You Test

Several days ago I came across a YouTube video by Alphonso Dunn giving tips about the different ways of seeing the thing you’re drawing. As I was watching, I couldn’t help relate what he was talking about to testing software. I’ll try to properly explain what I mean.

His first tip was that I should learn to see through the object I’m drawing (or testing) as if it was transparent. In drawing a box, some its flaps may be hidden from view and the artist remembers to take that into account since it may affect drawing the whole form of the box, including light and shadow. In testing, I felt that this might mean seeing through the UI of the thing under test, remembering that applications often have hidden functionalities. I may just be testing what the user interface can do, but understanding how the application runs under the HTTP layer or how Javascript and CSS affects its behavior helps me diagnose what risks the surprising things I find could pose including knowing where the surprise really occurs.

Then, he tells me that I should practice seeing the thing I’m drawing (or testing) in terms of its whole shape. In the box example, that could indicate only outlining the overall shape the box in order to practice thinking about proportions or completeness first instead of going deeply into the details too early. In testing, it is like asking about what problem a certain app feature is actually solving for a certain persona or asking whether the thing being tested is more or less complete with regards to what it is supposed to do. Is the functionality small or huge compared to the entire application, and what does it mean to the testing that needs to be done if it’s tiny or exceedingly big? Can we test enough within the deadline, and if not what sort of testing should be prioritized first or last?

Next, he explains that I should see the object I’m drawing (or testing) as an explosion of flat broken pieces too. This intends practicing to see the form I’m trying to draw as a collection of simple two-dimensional shapes, which means teaching myself not to get overwhelmed by the complexity of the form. In software, I thought of this as simply giving focus on testing one small functionality at a time to learn the details of how it works without thinking so much about how it helps the application as a whole. If I pass in a malicious parameter to a function they maybe I can hack the app to perform things that it isn’t supposed to do. This might also mean considering about what kind of testing is most appropriate for a particular piece. Should I care for some unit testing here? Perhaps security testing? Or possibly performance tests?

Lastly, he tells me that I should also train to see the thing I’m drawing (or testing) as a simple 3D volume. Is the object a cube, a sphere, a cone, or a cylinder? In drawing, this is supposed to help us look where the planes are, which in turn helps us in placing shadows and capturing the object in paper close to the actual thing. Now I don’t know if there are 3D volumes in software applications but I feel that this is like thinking about the app as a flowchart of processes and features, looking at the places where various integration happens and why they matter. If I can’t make out the flow, if I can’t model it properly, then is that a good thing or not? What happens to the testing I have to do if the thing is complex?

The reason why drawing is sometimes difficult, according to Alphonso as he concludes in the video, is because the artist switches to see the object from one mode to another quickly as needed. That’s the same reason why it’s engaging too. I’d like to think that testing is similar – demanding and interesting the same way because it forces the tester to think about the app under test in many perspectives.

An Experiment on Executable Specifications

What: Create executable specifications for an ongoing project sprint. Executable specifications are written examples of a business need that can be run anytime and acts as a source of truth for how applications behave. It is a living documentation of what our software does, and it helps us focus more on solving the unusual and noteworthy problems instead of wasting time with the ones that we know we shouldn’t have been worrying about.

How:

  1. Join a software development team for one sprint duration.
  2. Discuss project tasks for that sprint with everyone.
  3. Ask about requirements examples and create executable specifications for those tasks as the application is being written.
  4. Refine the specifications when necessary.
  5. Continuously update the specifications until they pass.
  6. Keep the specifications running on a preferred schedule.

Why: To see if writing executable specifications alongside development is feasible during the duration of a sprint.

Limitations: Tester writes the executable specifications, programmers work as is.

Experiment Realizations:

  • Writing executable specifications can be done during actual app development, provided that tester is experienced with the tools for implementing it and understands well why the need for such specifications.
  • It is of course more beneficial if everyone in the team learn how executable specifications work and write/run the specifications, and why the need to implement it.
  • It will take quite a long while before executable specifications becomes a norm in the existing software development process, if ever. This is a function of whether everyone believes that such specifications are actually useful in practice, and then building the skills and habit of including these specifications in the team’s definition of done.

A Mismatch Between Expectations and Practices

We want performant, scalable, and quality software. We wish to build and test applications that our customers profess their love to and share to their friends.

And yet:

  • We have nonexistent to little unit, performance, API, and integration tests
  • The organization do not closely monitor feature usage statistics
  • Some of us do not exactly feel the pains our customers face
  • We don’t have notifications for outdated dependencies, messy migration scripts, among other failures
  • Some are not curious about understanding how the apps they test and own actually work
  • We have not implemented continuous build tools
  • It is a pain to setup local versions of our applications, even to our own programmers
  • We do not write checks alongside development, we lack executable specifications
  • Some still think that testing and development happen in silos
  • It is difficult to get support for useful infrastructure, as well as recognition for good work
  • Many are comfortable with the status quo

It seems that we usually have our expectations mismatched with our practices. We’re frequently eager to show off our projects but are in many instances less diligent in taking measures about baking quality in, and therefore we fail more often than not. What we need are short feedback loops, continuous monitoring, and improved developer productivity, ownership, and happiness. The difficult thing is, it all starts with better communication and culture.

An Encounter with the ‘cucumber.yml was found, but could not be parsed’ Error

One day last week I merged a colleague’s new feature tests to the existing test repository. Routinely after that, I logged in to our dedicated test server, updated the existing running test suite by retrieving the latest merged code, and ran a ‘bundle update’ because the updated code required the latest versions of the dependencies. The updates finished without hitches as expected after a minute or so. Task’s done, I just need to run a sample test to be certain everything’s well and good.

An error blows up in my face:

cucumber.yml was found, but cannot be parsed. Please refer to cucumber’s documentation on correct profile usage.

Before all these, I had run a simulation of the gem updates on my local machine including running all tests to make sure there were no surprises when updating the test server. But I was still caught off-guard. I didn’t see that coming.

What? That’s always the first question. I squinted at the screen and looked at the error message again; it says something about the cucumber.yml file unable to be read by the machine, like it was corrupted. I opened the file to check (which contains various cucumber profile configuration set up for running tests on different test environments), was able to view its contents in the text editor, inspected it for unfamiliar code and found nothing. It looks okay, but the machine says otherwise. Why?

I tried an initial Google search for the error message and saw a fix that was written five years ago, saying that the error occurs because of a particular ‘rerun.txt’ file. The post tells me that the problem will go away if I delete that file.

Except that I don’t have that file in my code repository. What now?

Maybe there’s really something going on with the test code. Let’s see what happens if I delete a particular profile in the cucumber.yml file. Done. No changes in behavior, error still exist. What if I delete everything? Done. Error didn’t go away. Hmm.. that’s odd. It seems that the file contents are not the problem, is valid YAML code too according to online checkers, and the file is not damaged in any way I can see.

I went to back to looking at the search results for possible solutions People keep telling me its about the rerun.txt file. Others say I need to edit a cucumber library file in order to see what test code specifically causes the error for cucumber’s runner. No more other clues. Now this is difficult.

I kept researching for plausible fixes online for a few hours, I thought that there may still be something that can help me but I missed. No such luck. Okay, let’s try editing that library file and see what happens. It was a first time viewing library code, because I didn’t have any reason to do it before, and told myself that maybe I should actually check it out more often.

I found the command-line interface profile_loader file and the particular code which loads the cucumber.yml  file in question:

Found you, cucumber's profile loader!

Found you, cucumber’s profile loader!

Commented out some few lines of code as suggested:

Now let's see what your problem really is

Now let’s see what your problem really is

Then ran the sample cucumber test again:

A problem with the Psych module! What's that? :O

A problem with the Psych module! What’s that? :O

Okay. It says that a missing private method named ‘load’ is being called for a particular Psych module. No wonder cucumber is failing. Bummer, I don’t have a single idea about a Psych module that cucumber runs. All I can do is another Google search for the new error message and maybe find a workaround.

I am reminded that problems in building systems for automated test suites are not limited to writing test code and the application under test. Just like any other software, they can sometimes break in areas we do not know anything.

Eventually I found this enlightening post on a Github repository:

Interestingly, a very short solution. It was a rubygems bug after all, and what I needed to do was to run a ‘gem update –system’ command to get our cucumber tests back up and running.

 

Contemplating In-Office Knowledge-Sharing Sessions

In the past I tend to prepare presentation slides if I want to share something to tester colleagues at work, often clippings of interesting articles which I felt could be useful for our knowledge-sharing sessions. It worked, but after some time the sessions felt monotonous and tedious. Probably because I always have to explain in detail the ideas and the lessons behind those clippings. I try to make my presentations interesting, but I think that after a while hearing the same voice over and over can get old.

These days I’m sharing videos instead. The videos are usually recorded conference talks or tutorials I have watched and learned from in recent years, and I have taken care in listing the the ones that are insightful, fun, and relatively short. It’s like I’m inviting officemates to watch a short movie for free. The big change: I don’t take a lot of time talking during the knowledge-sharing anymore. There are of course still bits of discussions before, after, or during the showing of a video, whenever necessary, for explaining why I have taken a liking to the talk or to ask them about what they understood. We take turns telling stories about our experiences related to the ideas shared by the speaker, which is nice. And compared to the powerpoint presentations I did before, I felt that because the speaker is someone from outside it makes the ideas shared in the talks and tutorials feel more fresh and real than when I’m merely showing them quoted paragraphs from blogs. That makes it easier for my colleagues to get curious and actually learn something, which is exactly the point of the activity.