Tracking exploratory testing work is difficult for test managers. We don’t want to micro-manage testers, we want them to explore to their hearts content when they test, but we won’t know how much progress there is if we don’t track the work. We also won’t know what sort of problems testers encounter during testing, unless they have the nerve to tell us immediately. Jonathan Bach’s “Session-Based Test Management” article has one suggestion: use sessions, uninterrupted blocks of reviewable and chartered test effort.
Here are a few favorite notes from the article:
Unlike traditional scripted testing, exploratory testing is an ad hoc process. Everything we do is optimized to find bugs fast, so we continually adjust our plans to re-focus on the most promising risk areas; we follow hunches; we minimize the time spent on documentation. That leaves us with some problems. For one thing, keeping track of each tester’s progress can be like herding snakes into a burlap bag.
The first thing we realized in our effort to reinvent exploratory test management was that testers do a lot of things during the day that aren’t testing. If we wanted to track testing, we needed a way to distinguish testing from everything else. Thus, “sessions” were born. In our practice of exploratory testing, a session, not a test case or bug report, is the basic testing work unit . What we call a session is an uninterrupted block of reviewable, chartered test effort. By “chartered,” we mean that each session is associated with a mission—what we are testing or what problems we are looking for. By “uninterrupted,” we mean no significant interruptions, no email, meetings, chatting or telephone calls. By “reviewable,” we mean a report, called a session sheet, is produced that can be examined by a third-party, such as the test manager, that provides information about what happened.
From a distance, exploratory testing can look like one big amorphous task. But it’s actually an aggregate of sub-tasks that appear and disappear like bubbles in a Jacuzzi. We’d like to know what tasks happen during a test session, but we don’t want the reporting to be too much of a burden. Collecting data about testing takes energy away from doing testing.
We separate test sessions into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We call these the “TBS” metrics. We then ask the testers to estimate the relative proportion of time they spent on each kind of task. Test design and execution means scanning the product and looking for problems. Bug investigation and reporting is what happens once the tester stumbles into behavior that looks like it might be a problem. Session setup is anything else testers do that makes the first two tasks possible, including tasks such as configuring equipment, locating materials, reading manuals, or writing a session report
We also ask testers to report the portion of their time they spend “on charter” versus “on opportunity”. Opportunity testing is any testing that doesn’t fit the charter of the session. Since we’re in doing exploratory testing, we remind and encourage testers that it’s okay to divert from their charter if they stumble into an off-charter problem that looks important.
Although these metrics can provide better visibility and insight about what we’re doing in our test process, it’s important to realize that the session-based testing process and associated metrics could easily be distorted by a confused or biased test manager. A silver-tongued tester could bias the sheets and manipulate the debriefing in such a way as to fool the test manager about the work being done. Even if everyone is completely sober and honest, the numbers may be distorted by confusion over the reporting protocol, or the fact that some testers may be far more productive than other testers. Effective use of the session sheets and metrics requires continual awareness about the potential for these problems.
- One colleague of mine, upon hearing me talk about this approach, expressed the concern that senior testers would balk at all the paperwork associated with the session sheets. All that structure, she felt, would just get in the way of what senior testers already know how to do. Although my first instinct was to argue with her, on second thought, she was giving me an important reality check. This approach does impose a structure that is not strictly necessary in order to achieve the mission of good testing. Segmenting complex and interwoven test tasks into distinct little sessions is not always easy or natural. Session-based test management is simply one way to bring more accountability to exploratory testing, for those situations where accountability is especially important.
I picked up Mark Manson’s “Models: Attract Women Through Honesty” because I was intrigued by the title and because I am at a time in my life where I’d like to meet more interesting women. It did not disappoint. The book was insightful, and, like other compelling reads, it pushes me to look hard at myself and how I’ve been living my life, this time particularly on the subject of women. The concepts of neediness and vulnerability are, for me, the main takeaways.
Here are just a few of the noteworthy lines from the book:
In our post-industrial, post-feminist world, we lack a clear model of what an attractive man is. Centuries ago, a man’s role and duty was power and protection. Decades ago, it was to provide. But now? We’re not quite sure. We are either the first or second generation of men to grow up without a clear definition of our social roles, and without a model of what it is to be strong and attractive men.
Seduction is an interplay of emotions. Your movement or lack of movement reflects and alters emotions, not the words. Words are the side-effect. Sex is the side-effect. The game is emotions, emotions through movement.
- In surveys among literally tens of thousands of women, across all cultures, ethnicities, age groups, and socio-economic standing, and even time periods, there’s one universal quality in men that they all find desirable: social status and access to resources. The amount in which they desire it varies from culture to culture and from age group to age group, but the desire for it is universal.
- Social status is determined by how you behave around other people, how other people behave around you, and how you treat yourself.
Female arousal is somewhat narcissistic in nature. Women are turned on by being wanted, by being desired. The more physical assertiveness you pursue a woman with, the more aroused she becomes — sometimes even if she wasn’t interested in you to begin with.
How attractive a man is is inversely proportional to how emotionally needy he is. The more emotionally needy he is in his life, the less attractive he is and vice-versa. Neediness is defined by being more highly invested in other people’s perceptions of you than your perception of yourself.
All people eventually return to their baseline levels of investment. And until one is able to permanently alter his baseline level of identity investment in themselves, they will continue to attract the same types of women, and end up in the same failed relationships. Permanent change to one’s investment and neediness in their relationships with women is hard and a process that encompasses all facets of one’s life. But it’s a worthwhile journey. As a man, it may be the most worthwhile journey. And the key to it is probably something you wouldn’t expect. In fact, it’s something that most men turn their nose up to when they hear it. It’s vulnerability.
Making yourself vulnerable doesn’t just mean being willing to share your fears or insecurities. It can mean putting yourself in a position where you can be rejected, saying a joke that may not be funny, asserting an opinion that may offend others, joining a table of people you don’t know, telling a woman that you like her and want to date her. All of these things require you to stick your neck out on the line emotionally in some way. You’re making yourself vulnerable when you do them. In this way, vulnerability represents a form of power, a deep and subtle form of power. A man who’s able to make himself vulnerable is saying to the world, “I don’t care what you think of me; this is who I am, and I refuse to be anyone else.” He’s saying he’s not needy and that he’s high status.
Show your rough edges. Stop trying to be perfect. Expose yourself and share yourself without inhibition. Take the rejections and lumps and move on because you’re a bigger and stronger man. And when you find a woman who loves who you are (and you will), revel in her affection.
- The biggest criticism of showing interest to a woman that you want to be with is that it immediately shows you as highly invested in her responses. When you say, “You’re cute and I wanted to meet you,” that translates roughly to, “Hi, I want to be with you and am officially invested in the prospect of it happening.” What they miss though is the sub-communication going on underneath what’s actually being said. The sub-communication is, “I’m totally OK with the idea of you rejecting me, otherwise I would not be approaching you in this manner.”
- True honesty is only possible when it is unconditional. The truth is only the truth when it is given as a gift — when nothing is expected in return. When I tell a girl that she is beautiful, I say it not expecting anything in return. Whether she rejects me or falls in love with me isn’t important in that moment. What’s important is that I’m expressing my feelings to her. I will give compliments only when I am honestly inspired to give them, and usually after already meeting a woman and displaying to her that I’m willing to disagree with her, willing to be rejected by her and willing to walk away from her if it ever comes to that.
How does your day-to-day work look like?
What do you know about software testing? How about being a scrum master?
What is your most favorite part of the work? Your least favorite?
Have you taken part in a project that you’re really proud of?
What made you decide to apply for this position?
Why not apply for a programmer job?
What three skills do you think matters most to a tester? To a scrum master?
How do you rate yourself in those skills?
What skill would you say you’ve actively pursued recently?
Are you familiar with the following?
- Exploratory testing
- Test-driven development
- Behavior-driven development
Do you have hobbies that are important enough to waste time on?
What is one of your favorite movies? A favorite book? A favorite TV series?
Do you have specific goals in mind that you want to achieve when you get this role?
In what ways do you think we can help you achieve those goals?
Do you have questions for me or for your employer?
(Because I’ve been scheduled to interview a number of applicants in the coming week and I often forget to ask some of the important questions when I go at it head-on, I thought I needed some sort of template I can always grab and bring)
There’s a lot I don’t know about being a kick-ass boss, or being a great manager. I’m pretty much still a work-in-progress in that area. Thankfully, Kim Scott’s “Radical Candor” points me to the questions I need to ask myself about it, and suggests steps I should consider taking in order to do better.
Some memorable lines from the book:
- Relationships don’t scale. But the relationships you have with the handful of people who report directly to you will have an enormous impact on the results your team achieves.
- You strengthen your relationships by learning the best ways to get, give, and encourage guidance; by putting the right people in the right roles on your team; and by achieving results collectively that you couldn’t dream of individually. When you fail to give people the guidance they need to succeed in their work, or put people into roles they don’t want or aren’t well-suited for, or push people to achieve results that are unrealistic, you erode trust.
- It turns out that when people trust you and believe you care about them, they are much more likely to 1) accept and act on your praise and criticism; 2) tell you what they really think about what you are doing well and, more importantly, not doing so well; 3) engage in this same behavior with one another, meaning less pushing the rock up the hill again and again; 4) embrace their role on the team; and 5) focus on getting results.
- We are all human beings, with human feelings, and, even at work, we need to be seen as such. When that doesn’t happen, when we feel we must repress who we really are to earn a living, we become alienated. That makes us hate going to work. To most bosses, being professional means: show up at work on time, do your job, don’t show feelings unless engaged in motivation or some such end-driven effort. The results is that nobody feels comfortable being who they really are at work.
- It’s about finding time for real conversations; about getting to know each other at a human level; about learning what’s important to people, about sharing with one another what makes us want to get out of bed in the morning and go to work – and what has the opposite effect.
- Challenging others and encouraging them to challenge you helps build trusting relationships because it shows 1) you care enough to point out both the things that aren’t going well and those that are and that 2) you are willing to admit when you’re wrong and that you are committed to fixing mistakes that you or otherwise have made. But challenging people directly takes real energy – not only from the people you’re challenging but from you as well – so do it only for things that really matter.
- If nobody is ever mad at you, you probably aren’t challenging your team enough. The key, as in any relationship, is how you handle the anger. When what you say hurts, acknowledge the other person’s pain. Don’t pretend it doesn’t hurt or say it shouldn’t hurt – just show that you care. Eliminate the phrase “don’t take it personally” from your vocabulary – it’s insulting. Instead, offer to help fix the problem. But don’t pretend it isn’t a problem just to try to make somebody feel better.
- If we have the data about what works, let’s look at the data, but if all we have are opinions, let’s use yours.
- What growth trajectory does each person on my team want to be on right now? or Have I given everybody opportunities that re in line with what they really want? or What growth trajectory do my direct reports believe they are on? Do I agree? And if I don’t, why don’t I? Sometimes people really want to grow and are capable of contributing more than they have allowed to; at other times, they simply want more money or recognition but don’t really want to change the way they work or contribute any more than they do already. As the boss, you’re the one who’s going to have to know your direct reports well enough to make these distinctions and then have some radically candid conversations when you see things differently.
- There’s nothing wrong with working hard to earn a pay check that supports the life you want to lead. That has plenty of meaning. Only about 5% of people have a real vocation in life, and they confuse the hell out of the rest of us. Your job is not to provide purpose but instead to get to know each of your direct reports well enough to understand how each one derives meaning from their work.
- You don’t want to be the absentee manager any more than you want to be a micro-manager. Instead, you want to be a partner – that is, you must take the time to help the people doing the best work overcome obstacles and make their good work even better. This is time-consuming because it requires that you know enough about the details of the person’s work to understand the nuances. It often requires you to help do the work, rather than just advising. It requires that you ask a lot of questions and challenge people – that you roll up your own sleeves.
- It won’t get better all by itself. So stop and ask yourself, how exactly, will it get better? What are you going to do differently? What will the person in question do differently? How might circumstances change? Even if things have gotten a little better, have they improved enough? If you don’t have a pretty precise answer to those questions, it probably won’t get better.
- If you never ask a single question about a person’s life, it’s hard to move up on the “care personally” axis. probably the most important thing you can do to build trust is to spend a little time alone with each of your direct reports on a regular basis. Holding regular 1:1s in which your direct report sets the agenda and you ask questions is a good way to begin building trust.
- The platinum rule says, figure out what makes the other person comfortable, and do that.
Today, here’s a new batch of compelling videos I’d like to share, for the curious:
- What is Shift Left Testing? (by Alan Richardson, about shifting left and its meaning, consultancy speak, testing at all points of the software development process, and finding ways to improve our testing without resorting to consultancy speak)
- How We Work #4: Known and Unknowns (by Basecamp, on creative work, known and unknown tasks and why it’s more important to track them rather than estimates, and using the hill chart)
- A Better Technical Interview: 2018 (by Josh Greenwood, representing Test Double, about the binary hiring process existing in most industries, setting job and interview expectations, thinking about making the life of candidates better including the ones we don’t end up hiring, saying ‘Not Yet’ and providing constructive feedback, and the Bridge Agent)
- How Open Source has Made Me and the Stuff I Make Better (by Kent Dodds, on open source software, remote work, improving technical and interpersonal skills, documentation, and asking better questions and learning from other people)
- Office Politics for the Thin-Skinned Developer (by Justin Searls, about politics in the workplace, crazy organizational growth, building awareness, targeted empathy, being in the same page, how to make a difference, and allowing others to fail)
Recently, I was asked by the Human Capital Management team at work for a list of specific requirements for our scrum master role. I obliged, and at the top of my head wrote the following:
A scrum master is someone who –
- has great written and verbal communication skills
- understands the software development process, has experience working with product managers and programmers; preferably with customers too
- well-versed in the practice of software testing, enjoys exploring systems, thinking in various perspectives, and putting on different sorts of hats
- delights in shouldering a support role to the software development team
- is a self-starter, regularly updates himself/herself on what’s happening in the software development and testing industry
- someone who takes pleasure in a bit of scripting / programming is a plus (Webdriver, Watir, Cypress)
It’s not an extensive list, and I may have gotten some of the details wrong about what skills scrum masters are supposed to have based on the ideal definitions that’s out there in the web, but it’s alright. These are just the things I initially thought would suffice, in the context of what I and my team does and experience most days. Our testers are scrum masters too, and I’m proud that so far we’ve been able to make stuff work on our end.
Scrum masters in other places probably need a dissimilar set of requirements, because those are what allows their systems and processes to be effective, and that’s just fine.
There are two things that’s wonderful from last year’s Agile Testing Days conference talks: content focusing on other valuable stuff for testers and teams (not automation), and having as many women speakers as there are men. I hope they continue on with that trend.
Here’s a list of my favourite talks from said conference (enjoy!):
- How To Tell People They Failed and Make Them Feel Great (by Liz Keogh, about Cynefin, our innate dislike of uncertainty and love of making things predictable, putting safety nets and allowing for failure, learning reviews, letting people change themselves, building robust probes, and making it a habit to come from a place of care)
- Pivotal Moments (by Janet Gregory, on living in a dairy farm, volunteering, traveling, toastmasters, Lisa Crispin, Mary Poppindieck and going on adventures, sharing failures and taking help, and reflecting on pivotal moments)
- Owning Our Narrative (by Angie Jones, on the history of the music industry so far, the changes in environment, tools, and business models musicians have had to go through so survive, and embracing changes and finding ways to fulfil our roles as software testers)
- Learning Through Osmosis (by Maaret Pyhäjärvi, on mob programming and osmosis, creating safe spaces to facilitate learning, and the power of changing some of our beliefs and behaviour)
- There and Back Again: A Hobbit’s/Developer’s/Tester’s Journey (by Pete Walen, on how software was built in the old days, how testing and programming broke up into silos, and a challenge for both parties to go back at excelling at each other’s skills and teaming up
- 10 Behaviours of Effective Agile Teams (by Rob Lambert, about shipping software and customer service, becoming a more effective employee, behaviours, and communicating well)