Elisabeth Hendrickson’s book “There’s Always A Duck” has been around for a number of years but I have only been able to read it recently. Now I know what she meant about ducks. They’re literally about ducks, but also about people too. People are different, and yet we share similarities. We experience things, we communicate with each other, and we learn and get better because of those experiences. Her book tells us stories of her adventures and the lessons she’s discovered along the way, and it was nice to have had a glimpse of what she saw and felt with her encounters with software project teams and everyone involved.
- The vast majority of programmers I have met are diligent, capable folk. They truly care about the quality of their work and want the software they produce to be useful. They work hard to make sure they are implementing the right features and writing solid code.
- The next time you’re tempted to think of your programmers as idiots, incompetents, or quality hostile, remember that no matter what else they may be, they’re people first. Even if it seems like they’re hostile or incapable, it is far more likely that they are having a very human reaction to a particularly bad situation.
- And before you blame someone else for a mistake, remember the last time you made one. I’ve made some real whopper mistakes in my time. We all have, whether or not we choose to admit them or even remember them. It may be that some programmers don’t care about users, but it’s more likely that bugs are honest mistakes made under difficult circumstances.
- Even when we are speaking the same language and about the same thing, it’s hard enough to communicate.
- The point wasn’t to catch every possible error. What seems to go wrong most often? What errors are difficult to see at first glance, and thus require concentration to prevent? What causes the most damage when it happens?
- Janet doesn’t know anything about the ins and outs of creating software. She probably doesn’t want to know. She just wants to serve her customers well. And this software is not helping. Back at corporate, the Steering Committee, Requirements Analysts, Designers, Programmers and Testers are congratulating themselves on a solid release. What they don’t see is Janet’s pain. The feedback loop is broken. The team back at corporate has no mechanism to find out whether the software is any good. Oh, sure, they’ll detect catastrophic problems that cause servers to go down. But they won’t see the little things that cause long queues at the front desk of the hotel.
- Testers naturally notice details. Not only do we notice, but we think about what we noticed, we attempt to interpret our observations to explain why things might be that way, we ask others if they noticed, we question our assumptions, and we poke and prod at things to see if our understanding is correct. We use our observations to inform us, and in doing so discover underlying causes and effects we otherwise might miss.
- I sometimes fall into the trap of thinking that the first problem I see must be THE problem that needs to be solved. Perhaps the problem I spotted is indeed worth correcting, but I almost never manage to spot the true critical issue at first glance.
- Both fear and excitement stem not from observable reality but rather from speculation. We are speculating that the bugs that we know about and have chosen not to fix are actually as unimportant to our users as they are to us. We are speculating that the fact we have not found any serious defects is because they don’t exist and not because we simply stopped looking. We are speculating that we knew what the users actually wanted in the first place. We are speculating that the tests we decided not to run wouldn’t have found anything interesting. We are speculating that the tests we did run told us something useful. None of it is real until it is in the hands of actual users. The experience those users report is reality. Everything else is speculation.
- It’s not because Agile is about going faster. It’s because structuring our work so that we can ship a smaller set of capabilities sooner means that we can collapse that probability wave more often. We can avoid living in the land of speculation, fooling ourselves into thinking that the release is alive (or dead) based on belief rather than fact. In short, frequent delivery means we live in reality, not probability.
- Hire the right people. If that means keeping a critical position on the team open longer than anticipated, so be it. It’s better to have an under- staffed team of highly motivated, talented, skilled people than a fully staffed but ineffective team. Remember that hiring mistakes often take only a few minutes to make, and months of wasted time to undo.
- Listen. There are always signs when a project is in trouble: missed milestones, recurrent attitude problems, general confusion about the project. Sometimes these signs indicate a dysfunctional team, sometimes they’re just normal bumps along the road, and sometimes they are early warning signs of major problems. The only way to tell the difference is to listen carefully to what the team members have to say.
- The best way to get people to accept change is to make it more fun, and more rewarding, to do things the new way.
- Choose a path that takes you in the direction you want to go. Don’t choose a path simply because it takes you away from the swamp you want to avoid.
Ultimately what makes software testing fun and challenging is because it needs you to interact with people. You have to talk to both the software development team and the application users. You have to understand what they care for and why. You have to ask questions and the answers to those questions guide what you do when you test. You can test in a silo but that doesn’t help you learn anything deeper, you can’t thrive if you stick to the things you already know. You grow because you take in all these perspectives and beliefs that other people have and test them against your own, for the goal of making better judgments about how to test the thing your testing and why you’re testing them in the first place. It’s not easy, because you have to care. The challenge is that you have to learn a great deal of many things, you have to regularly check the landscape of where you are, and you often need to reflect about what you’ve done and what you want to do next. And on the journey of overcoming the challenges you set for yourself is where the fun is.
I’m currently in the midst of a test code overhaul, a re-writing project of sorts. It started about a week ago and so far I’ve made considerable progress on what I’ve wanted to achieve with the rewrite, which is basically cleaner and more maintainable code, mostly in the sense of test data management and test description language. The number of tests running everyday in our Jenkins system has grown noticeably and I’ve felt that it’s been difficult to add certain tests because of how I structured the test data in the past, which I have not upgraded since then. The two possible avenues for running tests – on the UI and HTTP layers – also adds a bit of complexity and it’d be nice if I can integrate the two smoothly. It’s an interesting development because I did not plan on any re-writing to be done anytime soon but I guess at the back of my mind I knew it’ll happen eventually. And so I decided to take a step back from writing more tests and do some cleanup before it gets tougher to change things. I plan to finish everything in about a month or so.
At the moment, I’m reminded of the phases I’ve gone through in learning to code and writing automated checks in the past few years:
- Early 2014. It all begins with Selenium IDE, with giving the self some time to study the basic Selenese commands for writing automated checks and (more importantly) understand how to properly retrieve the page elements you want to manipulate.
- Mid 2014. Test management in Selenium IDE becomes difficult as the number of tests grow, hence the decision to switch to Selenium WebDriver. The only programming language background I had back then was C++, which was limited to only functions and logical/conditional operators, so I chose Java to work with to lessen the learning curve.
- Late 2014. Familiarized myself with Git, which hooked me on making daily commits and appreciating version control. Along the way I learned the concepts of classes and objects.
- All of 2015 up to Early 2016. I was in a trance, writing code daily and pushing myself to create all the automated checks that I wanted to run for our apps before every release. Tests run on the Eclipse IDE using TestNG and I was happy with what I had, except that those end-to-end tests are really slow. Running everything took overnight to finish, which was okay for my employer but annoying for me personally.
- Mid 2016. Re-writing existing tests in Ruby with Cucumber integration started off (when I found Jeff Morgan’s “Cucumber & Cheese” book online) as a side project for fun and testing my skill level in programming. And I did have buckets of fun! The experiment told me that there’s still a lot I need to practice on if I want to write better code, and it also told me that I can be more productive if I switch programming languages. There’s a bit less code to type in when writing code in Ruby than Java and I liked that, plus all the interesting libraries I can use. I switched to Sublime Text and used both Jenkins and the command-line interface more extensively too.
- Late 2016. As I was looking for ways to speed up end-to-end tests total execution, which by then takes about 4 hours to complete, I ended up exploring testing apps in the HTTP layer instead of in the UI. That took a lot of studying of how our apps actually behave under the hood, what data are being passed around, how images are actually sent, how to view pages without a browser, how redirections work, among other things. After years of testing apps via the user interface, this was such a refreshing and valuable period, and I completely wondered why I never knew such a thing existed until then. It wasn’t being taught extensively to testers, perhaps because it all depends on how the app was structured to run through an API.
And these phases brings me to now, where there’s a healthy dose of API and UI layer tests all checking major app features. It’s all good, just several pieces needing a cleanup, a little parallelization, better test description language, and great documentation. It’s all good, because the lessons in both programming and testing keep piling. The two practices differ in mindset but I think they complement each other, and I think that there’s no reason anyone can’t do both.
Timothy Ferriss‘ “The 4-Hour Workweek” might be my favorite book this year. Tons of lessons about how to be productive, as well as how to enjoy life to the fullest, and reading the book has left me excited about the future while reconsidering many of my other choices.
Some favorite takeaways:
- The perfect job is the one that takes the least time. The goal is to free time and automate income.
- ‘If only I had more money’ is the easiest way to postpone the intense self-examination and decision-making necessary to create a life of enjoyment – now and not later. Busy yourself with the routine of the money wheel, pretend it’s the fix-all, and you artfully create a constant distraction that prevents you from seeing just how pointless it is. Deep down, you know it’s all an illusion, but with everyone participating in the same game of make-believe, it’s easy to forget.
- What are you waiting for? If you cannot answer this without resorting to the previously rejected concept of good timing, the answer is simple: You’re afraid, just like the rest of the world. Measure the cost of inaction, realize the unlikelihood and repairability of most missteps, and develop the most important habit of those who excel, and enjoy doing so: action.
- Retirement as a goal or final redemption is flawed for at least three solid reasons:
- It is predicated on the assumption that you dislike what you are doing during the most physically capable years of your life.
- Most people will never be able to retire and maintain even a hotdogs-for-dinner standard of living. Even one million is chump change in a world where traditional retirement could span 30 years and inflation lowers your purchasing power 2-4% per year. The math doesn’t work.
- If the math doesn’t work, it means that you are one ambitious hardworking machine. If that’s the case, guess what? One week into retirement, you’ll be so damn bored that you’ll want to stick bicycle spokes in your eyes.
- If it isn’t going to devastate those around you, try it and then justify it. Get good at being a troublemaker and saying sorry when you really screw up. Ask for forgiveness, not permission.
- Ninety-nine percent of people in the world are convinced they are incapable of achieving great things, so they aim for the mediocre. The level of competition is thus fiercest for ‘realistic’ goals, paradoxically making them the most time- and energy-consuming. So do not overestimate the competition and underestimate yourself. You are better than you think.
- Doing something unimportant well does not make it important. Requiring a lot of time does not make a task important. What you do is infinitely more important than how you do it. Efficiency is still important, but is useless unless applied to the right things.
- Remember that most things make no difference. Being busy is a form of laziness – lazy thinking and indiscriminate action. Being overwhelmed is often as unproductive as doing nothing and is far more unpleasant. Being selective – doing less – is the path of the productive. Focus on the important few and ignore the rest.
- If you haven’t identified the mission-critical tasks and set aggressive start and end times for their completion, the unimportant becomes the important. Even if you know what’s critical, without deadlines that create focus, the minor tasks forced upon you (or invented) will swell to consume time until another bit of minutiae jumps in to replace it, leaving you at the end of the day with nothing accomplished.
- The key to having more time is doing less, and there are two paths to getting there, both of which should be used together:
- Define a short to-do list
- Define a not-to-do list
- Don’t ever arrive at the office or in front of your computer without a clear list of priorities. There should be no more than 2 mission-critical items to complete each day. Never. It just isn’t necessary if they’re actually high-impact.
We want performant, scalable, and quality software. We wish to build and test applications that our customers profess their love to and share to their friends.
- We have nonexistent to little unit, performance, API, and integration tests
- The organization do not closely monitor feature usage statistics
- Some of us do not exactly feel the pains our customers face
- We don’t have notifications for outdated dependencies, messy migration scripts, among other failures
- Some are not curious about understanding how the apps they test and own actually work
- We have not implemented continuous build tools
- It is a pain to setup local versions of our applications, even to our own programmers
- We do not write checks alongside development, we lack executable specifications
- Some still think that testing and development happen in silos
- It is difficult to get support for useful infrastructure, as well as recognition for good work
- Many are comfortable with the status quo
It seems that we usually have our expectations mismatched with our practices. We’re frequently eager to show off our projects but are in many instances less diligent in taking measures about baking quality in, and therefore we fail more often than not. What we need are short feedback loops, continuous monitoring, and improved developer productivity, ownership, and happiness. The difficult thing is, it all starts with better communication and culture.
Then start small. Really, small.
Sure, write your big goals on (physical or digital) paper, and dream about all the great things you can do when you’re slimmer and healthier, more skillful in your art, more financialy secure, or more social, but please don’t expect those changes to just magically happen because you listed and thought about them. They’re necessary steps for the new you that you’re excited to be, but incomplete. All artists want to draw better, all musicians dream of playing music in the grandest stage, all writers think about being successful authors, all of us want to be fit, want to be less socially awkward, and want to travel more. We all want to make a living where we both have fun and generate enough income for our families. Big goals are fun to think of, but they can be overwhelming. Vague goals are easy to list but they are difficult to implement in our lives.
So start small. Pick the most important goal that you have in your list, and if it is big, split, slice, break it down to incredibly tiny, specific, chewable pieces. Instead of drawing better, you could re-write your goal to draw one portrait (or landscape or idea) for an hour every Saturdays of January in your favorite place. If you want to lose some pounds, you could start by religiously running every morning of every other day of this month just for 30 minutes in a park somewhere near, before everybody else wakes up. If you want to write a novel, you could timebox yourself to writing something about it every day for one month just for an hour right after you wake up, before going to work. If you want to learn a new skill, give yourself 30 minutes to research and practice it everyday at work. Test yourself, experiment for a month without excuses, see if you can do these small things and like your results. If you do, then do them again. If you don’t, ask yourself why. The thing to remember here is that in order to make big stuff come true, you need to make small changes first, no shortcuts. You should make few but bold decisions, you need to re-build your systems of working, you must review and replace habits that don’t work with ones that do, because that’s where it all really starts.
For new apps and software features, it’s perfectly normal to exhaustively test the matrix of all possible scenarios with equal importance before release – the happy path, the confused path, the forgetful or greedy paths, etcetera – whenever the tester can. After all, no one knows yet how exactly the new application will behave in Production, and it is always better (and cheaper) to find problems and fix them early instead of reacting as they occur.
For small software that are already in the wild and which do not change often, it is still a good rule of thumb to test everything with equal priority, if the budget allows for it. It’s still nice to catch bugs before the next version release rather than have a client surprise us at odd times with a system failure.
For complex legacy systems which frequently change and usually do not automatically test themselves, it is still a great idea to test features in full-scale, if we can, if it’s possible to perform all those complicated use cases within hours before the deadline. That’s a noble but a hugely difficult task for testers, if not impossible to accomplish, and something that might be a complete waste of time and creativity, or a foolish thing to do, even if it is very important for teams to have confidence in the stability of their applications. For such systems it might be better to re-evaluate what goals does our testing need to reach, re-think what sort of stuff do we want to find, re-learn what services do we want to keep providing to our customers as well as how effectively do we want to provide those services, and go from there. It might be better to spend some of our time on data monitoring and analysis, on understanding how our clients actually use our products and focus our testing those usage scenarios. It might be better for us to amplify testing and developing the features that our clients love. And it might be more worth it to regularly identify the riskiest parts of the system and improve their design and testability, rather than trying to keep up with the matrix we know we couldn’t possibly deal with everytime.
Apps evolve, and I think that the testing that we perform needs to change too according to our current contexts.