Notes from Jonathan Bach’s “Session-Based Test Management”

Tracking exploratory testing work is difficult for test managers. We don’t want to micro-manage testers, we want them to explore to their hearts content when they test, but we won’t know how much progress there is if we don’t track the work. We also won’t know what sort of problems testers encounter during testing, unless they have the nerve to tell us immediately. Jonathan Bach’s “Session-Based Test Management” article has one suggestion: use sessions, uninterrupted blocks of reviewable and chartered test effort.

Here are a few favorite notes from the article:

  • Unlike traditional scripted testing, exploratory testing is an ad hoc process. Everything we do is optimized to find bugs fast, so we continually adjust our plans to re-focus on the most promising risk areas; we follow hunches; we minimize the time spent on documentation. That leaves us with some problems. For one thing, keeping track of each tester’s progress can be like herding snakes into a burlap bag.
  • The first thing we realized in our effort to reinvent exploratory test management was that testers do a lot of things during the day that aren’t testing. If we wanted to track testing, we needed a way to distinguish testing from everything else. Thus, “sessions” were born. In our practice of exploratory testing, a session, not a test case or bug report, is the basic testing work unit . What we call a session is an uninterrupted block of reviewable, chartered test effort. By “chartered,” we mean that each session is associated with a mission—what we are testing or what problems we are looking for. By “uninterrupted,” we mean no significant interruptions, no email, meetings, chatting or telephone calls. By “reviewable,” we mean a report, called a session sheet, is produced that can be examined by a third-party, such as the test manager, that provides information about what happened.
  • From a distance, exploratory testing can look like one big amorphous task. But it’s actually an aggregate of sub-tasks that appear and disappear like bubbles in a Jacuzzi. We’d like to know what tasks happen during a test session, but we don’t want the reporting to be too much of a burden. Collecting data about testing takes energy away from doing testing.
  • We separate test sessions into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We call these the “TBS” metrics. We then ask the testers to estimate the relative proportion of time they spent on each kind of task. Test design and execution means scanning the product and looking for problems. Bug investigation and reporting is what happens once the tester stumbles into behavior that looks like it might be a problem. Session setup is anything else testers do that makes the first two tasks possible, including tasks such as configuring equipment, locating materials, reading manuals, or writing a session report
  • We also ask testers to report the portion of their time they spend “on charter” versus “on opportunity”. Opportunity testing is any testing that doesn’t fit the charter of the session. Since we’re in doing exploratory testing, we remind and encourage testers that it’s okay to divert from their charter if they stumble into an off-charter problem that looks important.
  • Although these metrics can provide better visibility and insight about what we’re doing in our test process, it’s important to realize that the session-based testing process and associated metrics could easily be distorted by a confused or biased test manager. A silver-tongued tester could bias the sheets and manipulate the debriefing in such a way as to fool the test manager about the work being done. Even if everyone is completely sober and honest, the numbers may be distorted by confusion over the reporting protocol, or the fact that some testers may be far more productive than other testers. Effective use of the session sheets and metrics requires continual awareness about the potential for these problems.
  • One colleague of mine, upon hearing me talk about this approach, expressed the concern that senior testers would balk at all the paperwork associated with the session sheets. All that structure, she felt, would just get in the way of what senior testers already know how to do. Although my first instinct was to argue with her, on second thought, she was giving me an important reality check. This approach does impose a structure that is not strictly necessary in order to achieve the mission of good testing. Segmenting complex and interwoven test tasks into distinct little sessions is not always easy or natural. Session-based test management is simply one way to bring more accountability to exploratory testing, for those situations where accountability is especially important.
Advertisements

Flutter Basics: HTTP Post Requests

Myself and the rest of Team Tiger (from the Flutter dev internship program we’ve recently joined) have just been handed a challenge to work with a simple API, to send HTTP requests and receive responses which a mobile app should understand. I’ve always been fond of reading and understanding web network requests, and knew the challenge would be fun.

If we want to work with HTTP requests, we need to import several dependencies:

The dart packages are built into Flutter, but we also need an external http package, which we have to include in our pubspec.yaml file, like so:

And now we can try sending post requests through our mobile app! Here’s an example code of asynchronously deleting something through an API:

The code is straightforward, sending a .post() method containing the URL of where we want to perform an activity and a  body field which tells us what information we need to send to said URL, all inside an asynchronous Future. We specify that we want to close the HTTP client after the POST request gets completed by sending a client.close command inside the .whenComplete() method.

We can change the app state by running desired commands inside the setState() method. In the example code above, we’re just printing the response received from the POST request. What you want to do about that response is up to you. You can also return the response to someplace in the app if you want.

The if (!mounted) clause is there just to guard us from situations where the user suddenly moves to another part of the application while the POST request is still running. In that case, the request immediately stops and sends a desired return.

Takeaways from Timothy Ferriss’ “The 4-Hour Body”

This year, I decided I was going to get better at exercising. To do that, I thought about reading a few books to give myself an idea on how to go about it. One such book was Timothy Ferriss’ “The 4-Hour Body“, which is a goldmine of content. In it are suggested exercises that gets the job done, walkthroughs, and some science of how things work. But the book is so much more than just a guide on physical exercises. There’s ideas on self-experimentation, adherence, being proactively skeptical, harajuku moments, a slow-carb diet, and more.

Here are some favorite takeaways:

  • Science starts with educated (read: wild-ass) guesses. Then it’s all trial and error. Sometimes you predict correctly from the outset. More often, you make mistakes and stumble across unexpected findings, which lead to new questions. If you want to sit on the sidelines and play full-time skeptic, suspending actions until a scientific consensus is reached, that’s your choice. But don’t use skepticism as a thinly veiled excuse for inaction or remaining in your comfort zone. Be skeptical, but for the right reason: because you’re looking for the most promising option to test in real life. Be proactively skeptical, not defensively skeptical.
  • We break commitments to ourselves with embarrassing regularity. How can someone trying to lose weight binge on an entire pint of ice cream before bed? How can even the most disciplined of executives fail to make 30 minutes of time per week for exercise? How can someone whose marriage depends on quitting smoking pick up a cigarette? Simple: logic fails.
  • Take adherence seriously: will you actually stick with this change until you hit your goal? If not, find another method, even if it’s less effective and less efficient. The decent method that you follow is better than the perfect method you quit.
  • Self-experimentation can be used by non-experts to (a) see if the experts are right and (b) learn something they don’t know. When you study your own problem (e.g. acne), you care more about finding a solution than others are likely to care.
  • If results are fast and measurable, self-discipline isn’t needed.
  • If you want to be more confident or effective, rather than relying on easily-defeated positive thinking and mental gymnastics, learn to run faster, lift more than your peers, or lose those last ten pounds. It’s measurable, it’s clear, you can’t lie to yourself. It therefore works. The Cartesian separation of mind and body is false. They’re reciprocal. Start with the precision of changing physical reality and a domino effect will often take care of the internal.
  • Job not going well? Company having issues? Some idiot making life difficult? If you add ten laps to your swimming, or if you cut five seconds off your best mile time, it can still be a great week. Controlling your body puts you in life’s driver’s seat.
  • Recreation is for fun. Exercise is for producing changes. Don’t confuse the two.
  • People suck at following advice. Even the most effective people in the world are terrible at it. There are two reasons:
    • Most people have an insufficient reason for action. The pain isn’t painful enough. It’s a nice-to-have, not a must-have.
    • There are no reminders. No consistent tracking = no awareness = no behavioral change. Consistent tracking, even if you have no knowledge of fat-loss or exercise, will often beat advice from world-class trainers.
  • For a long time, I’ve known that the key to getting started down the path of being remarkable in anything is to simply act with the intention of being remarkable. If I want a better-than-average career, I can’t simply go with the flow and get it. Most people do just that: they wish for an outcome but make no intention-driven actions toward that outcome. If they would just do something most people would find that they get some version of the outcome they’re looking for. That’s been my secret. Stop wishing and start doing.

Web Accessibility Visualization Tool: tota11y

Testing web sites and apps come in many forms. Testers try their best to test everything, but obviously there’s only so much they can do within a schedule. Some forms of testing are more prioritized than others, and that’s not inherently bad; for a solo tester on a team, one usually tests in a way that covers more bases at the beginning.

Web accessibility testing is one of those forms of testing that often takes a backseat, sometimes even forgotten. Web accessibility helps people with disabilities get better surfing experience. Although websites are typically not built with such functionality in mind, it matters.

And tota11y is a tool from Khan Academy we can leverage for testing accessibility. It is available as an easy-to-use bookmarklet. For whatever page we want to test, we just need to go there and click the bookmarklet, after which the tool will appear on the bottom left corner of the page. Clicking the tool reveals options and using each helps us spot common accessibility violations.

Here are some screenshots of using it on a page I test at work, checking headings, contrast, and link text:

Spotted: Nonconsecutive heading level use

Multiple insufficient contrast ratio violations

And unclear link texts

Looks like there’s room for improvement, although these violations are not necessarily errors.

Flutter Dev Intern: One Month In

A month ago, I got accepted into a no-pay internship program volunteering as a Flutter programmer. It’s an interesting learning opportunity; I’m working with four other programmers from around the globe who personally don’t know each other, building native mobile apps on our free time. The program runs for six months, so I suppose I’ll be writing code with these guys until around April next year.

Team chat over at Discord 🙂

Even though there’s no pay and I spend time to help the team finish given app challenges, what I get in return is an insider experience working with a remote team as a programmer myself, instead of being a tester. That means I need to pitch in on the actual application code, and pitch in with a respectable level of quality. Although I already work with programmers on my day job, the communication dynamics is a bit different from what I encounter on a daily basis. It’s a good change of pace, and I’m somehow broadening my horizons a little.

Most of this past month’s challenge erred on the side of communication: talking to each other over at Discord, familiarizing ourselves with other people’s style of writing code, and gauging our roles within the team. It’s been fun so far. 🙂

Lessons from Mark Manson’s “Models: Attract Women Through Honesty”

I picked up Mark Manson’s Models: Attract Women Through Honesty” because I was intrigued by the title and because I am at a time in my life where I’d like to meet more interesting women. It did not disappoint. The book was insightful, and, like other compelling reads, it pushes me to look hard at myself and how I’ve been living my life, this time particularly on the subject of women. The concepts of neediness and vulnerability are, for me, the main takeaways.

Here are just a few of the noteworthy lines from the book:

  • In our post-industrial, post-feminist world, we lack a clear model of what an attractive man is. Centuries ago, a man’s role and duty was power and protection. Decades ago, it was to provide. But now? We’re not quite sure. We are either the first or second generation of men to grow up without a clear definition of our social roles, and without a model of what it is to be strong and attractive men.
  • Seduction is an interplay of emotions. Your movement or lack of movement reflects and alters emotions, not the words. Words are the side-effect. Sex is the side-effect. The game is emotions, emotions through movement.
  • In surveys among literally tens of thousands of women, across all cultures, ethnicities, age groups, and socio-economic standing, and even time periods, there’s one universal quality in men that they all find desirable: social status and access to resources. The amount in which they desire it varies from culture to culture and from age group to age group, but the desire for it is universal.
  • Social status is determined by how you behave around other people, how other people behave around you, and how you treat yourself.
  • Female arousal is somewhat narcissistic in nature. Women are turned on by being wanted, by being desired. The more physical assertiveness you pursue a woman with, the more aroused she becomes — sometimes even if she wasn’t interested in you to begin with.
  • How attractive a man is is inversely proportional to how emotionally needy he is. The more emotionally needy he is in his life, the less attractive he is and vice-versa. Neediness is defined by being more highly invested in other people’s perceptions of you than your perception of yourself.
  • All people eventually return to their baseline levels of investment. And until one is able to permanently alter his baseline level of identity investment in themselves, they will continue to attract the same types of women, and end up in the same failed relationships. Permanent change to one’s investment and neediness in their relationships with women is hard and a process that encompasses all facets of one’s life. But it’s a worthwhile journey. As a man, it may be the most worthwhile journey. And the key to it is probably something you wouldn’t expect. In fact, it’s something that most men turn their nose up to when they hear it. It’s vulnerability.
  • Making yourself vulnerable doesn’t just mean being willing to share your fears or insecurities. It can mean putting yourself in a position where you can be rejected, saying a joke that may not be funny, asserting an opinion that may offend others, joining a table of people you don’t know, telling a woman that you like her and want to date her. All of these things require you to stick your neck out on the line emotionally in some way. You’re making yourself vulnerable when you do them. In this way, vulnerability represents a form of power, a deep and subtle form of power. A man who’s able to make himself vulnerable is saying to the world, “I don’t care what you think of me; this is who I am, and I refuse to be anyone else.” He’s saying he’s not needy and that he’s high status.
  • Show your rough edges. Stop trying to be perfect. Expose yourself and share yourself without inhibition. Take the rejections and lumps and move on because you’re a bigger and stronger man. And when you find a woman who loves who you are (and you will), revel in her affection.
  • The biggest criticism of showing interest to a woman that you want to be with is that it immediately shows you as highly invested in her responses. When you say, “You’re cute and I wanted to meet you,” that translates roughly to, “Hi, I want to be with you and am officially invested in the prospect of it happening.” What they miss though is the sub-communication going on underneath what’s actually being said. The sub-communication is, “I’m totally OK with the idea of you rejecting me, otherwise I would not be approaching you in this manner.”
  • True honesty is only possible when it is unconditional. The truth is only the truth when it is given as a gift — when nothing is expected in return. When I tell a girl that she is beautiful, I say it not expecting anything in return. Whether she rejects me or falls in love with me isn’t important in that moment. What’s important is that I’m expressing my feelings to her. I will give compliments only when I am honestly inspired to give them, and usually after already meeting a woman and displaying to her that I’m willing to disagree with her, willing to be rejected by her and willing to walk away from her if it ever comes to that.

Automating the Windows 10 Desktop Calculator using PyAutoGui

Last year, I’ve tried automating a Windows 7 desktop calculator using a Java library called Winium. A year later, someone from the comments told me that the example didn’t work on Windows 10. I ignored the comment for a while because I didn’t have a Windows 10 machine to perform a test back then, but now that I’ve recently upgraded my home PC I decided to try it out.

What I found:

Running a Windows 7 desktop calculator automation example (using Winium) on a Windows 10 machine

The Windows 10 calculator opened up but the tests didn’t run properly. The error log told me that the program was unable to find the calculator elements it was supposed to click. Bummer. Maybe the names of the calculator elements were different, Windows 10 versus Windows 7? It wasn’t; the element names were still the same according to UI Spy. Perhaps there’s something from Winium that can point me to a clue? Oh, it seems that the library has not been updated in recent years.

If I can’t use Winium, how then can I automate the Windows 10 calculator? A Google search pointed me to PyAutoGui. Instead of Java, it says that I’ll need Python (and Pip) for this tool to work. And yes, being Windows, I also need to properly set the environment variables so I can use the Python and Pip commands on a terminal.

Let’s install PyAutoGui:

Installing PyAutoGui

And after writing some code, let’s see if we can automate the Windows 10 calculator with it:

Automating the Windows 10 desktop calculator with PyAutoGui. Click the image to view the GIF full-size on another browser tab 🙂

It works!

But here are some catches:

  • I had to rely on PyAutoGui’s keyboard control functions for performing the calculator actions, instead of finding elements via the user interface. Well, from what I’ve seen so far from the docs is that the only way to locate a UI element is by locating elements using screenshots. I tried that the first time and it was very flaky, so I opted for using the keyboard control functions instead.
  • The code introduces a time variable to wait for the calculator to appear on screen. The code also introduces a time variable to pause a portion of a second in-between each keyboard action so the actions don’t happen too fast for a person’s eye.
  • There are no assertions in the example code, because I couldn’t find any assertion functions I could use from the PyAutoGui docs. It is not a tool built for testing, only for automating desktop apps.

Source code for this experiment can be found on: Win-Calculator-PyAutoGui.