Ultimately what makes software testing fun and challenging is because it needs you to interact with people. You have to talk to both the software development team and the application users. You have to understand what they care for and why. You have to ask questions and the answers to those questions guide what you do when you test. You can test in a silo but that doesn’t help you learn anything deeper, you can’t thrive if you stick to the things you already know. You grow because you take in all these perspectives and beliefs that other people have and test them against your own, for the goal of making better judgments about how to test the thing your testing and why you’re testing them in the first place. It’s not easy, because you have to care. The challenge is that you have to learn a great deal of many things, you have to regularly check the landscape of where you are, and you often need to reflect about what you’ve done and what you want to do next. And on the journey of overcoming the challenges you set for yourself is where the fun is.
I’m currently in the midst of a test code overhaul, a re-writing project of sorts. It started about a week ago and so far I’ve made considerable progress on what I’ve wanted to achieve with the rewrite, which is basically cleaner and more maintainable code, mostly in the sense of test data management and test description language. The number of tests running everyday in our Jenkins system has grown noticeably and I’ve felt that it’s been difficult to add certain tests because of how I structured the test data in the past, which I have not upgraded since then. The two possible avenues for running tests – on the UI and HTTP layers – also adds a bit of complexity and it’d be nice if I can integrate the two smoothly. It’s an interesting development because I did not plan on any re-writing to be done anytime soon but I guess at the back of my mind I knew it’ll happen eventually. And so I decided to take a step back from writing more tests and do some cleanup before it gets tougher to change things. I plan to finish everything in about a month or so.
At the moment, I’m reminded of the phases I’ve gone through in learning to code and writing automated checks in the past few years:
- Early 2014. It all begins with Selenium IDE, with giving the self some time to study the basic Selenese commands for writing automated checks and (more importantly) understand how to properly retrieve the page elements you want to manipulate.
- Mid 2014. Test management in Selenium IDE becomes difficult as the number of tests grow, hence the decision to switch to Selenium WebDriver. The only programming language background I had back then was C++, which was limited to only functions and logical/conditional operators, so I chose Java to work with to lessen the learning curve.
- Late 2014. Familiarized myself with Git, which hooked me on making daily commits and appreciating version control. Along the way I learned the concepts of classes and objects.
- All of 2015 up to Early 2016. I was in a trance, writing code daily and pushing myself to create all the automated checks that I wanted to run for our apps before every release. Tests run on the Eclipse IDE using TestNG and I was happy with what I had, except that those end-to-end tests are really slow. Running everything took overnight to finish, which was okay for my employer but annoying for me personally.
- Mid 2016. Re-writing existing tests in Ruby with Cucumber integration started off (when I found Jeff Morgan’s “Cucumber & Cheese” book online) as a side project for fun and testing my skill level in programming. And I did have buckets of fun! The experiment told me that there’s still a lot I need to practice on if I want to write better code, and it also told me that I can be more productive if I switch programming languages. There’s a bit less code to type in when writing code in Ruby than Java and I liked that, plus all the interesting libraries I can use. I switched to Sublime Text and used both Jenkins and the command-line interface more extensively too.
- Late 2016. As I was looking for ways to speed up end-to-end tests total execution, which by then takes about 4 hours to complete, I ended up exploring testing apps in the HTTP layer instead of in the UI. That took a lot of studying of how our apps actually behave under the hood, what data are being passed around, how images are actually sent, how to view pages without a browser, how redirections work, among other things. After years of testing apps via the user interface, this was such a refreshing and valuable period, and I completely wondered why I never knew such a thing existed until then. It wasn’t being taught extensively to testers, perhaps because it all depends on how the app was structured to run through an API.
And these phases brings me to now, where there’s a healthy dose of API and UI layer tests all checking major app features. It’s all good, just several pieces needing a cleanup, a little parallelization, better test description language, and great documentation. It’s all good, because the lessons in both programming and testing keep piling. The two practices differ in mindset but I think they complement each other, and I think that there’s no reason anyone can’t do both.
Timothy Ferriss‘ “The 4-Hour Workweek” might be my favorite book this year. Tons of lessons about how to be productive, as well as how to enjoy life to the fullest, and reading the book has left me excited about the future while reconsidering many of my other choices.
Some favorite takeaways:
- The perfect job is the one that takes the least time. The goal is to free time and automate income.
- ‘If only I had more money’ is the easiest way to postpone the intense self-examination and decision-making necessary to create a life of enjoyment – now and not later. Busy yourself with the routine of the money wheel, pretend it’s the fix-all, and you artfully create a constant distraction that prevents you from seeing just how pointless it is. Deep down, you know it’s all an illusion, but with everyone participating in the same game of make-believe, it’s easy to forget.
- What are you waiting for? If you cannot answer this without resorting to the previously rejected concept of good timing, the answer is simple: You’re afraid, just like the rest of the world. Measure the cost of inaction, realize the unlikelihood and repairability of most missteps, and develop the most important habit of those who excel, and enjoy doing so: action.
- Retirement as a goal or final redemption is flawed for at least three solid reasons:
- It is predicated on the assumption that you dislike what you are doing during the most physically capable years of your life.
- Most people will never be able to retire and maintain even a hotdogs-for-dinner standard of living. Even one million is chump change in a world where traditional retirement could span 30 years and inflation lowers your purchasing power 2-4% per year. The math doesn’t work.
- If the math doesn’t work, it means that you are one ambitious hardworking machine. If that’s the case, guess what? One week into retirement, you’ll be so damn bored that you’ll want to stick bicycle spokes in your eyes.
- If it isn’t going to devastate those around you, try it and then justify it. Get good at being a troublemaker and saying sorry when you really screw up. Ask for forgiveness, not permission.
- Ninety-nine percent of people in the world are convinced they are incapable of achieving great things, so they aim for the mediocre. The level of competition is thus fiercest for ‘realistic’ goals, paradoxically making them the most time- and energy-consuming. So do not overestimate the competition and underestimate yourself. You are better than you think.
- Doing something unimportant well does not make it important. Requiring a lot of time does not make a task important. What you do is infinitely more important than how you do it. Efficiency is still important, but is useless unless applied to the right things.
- Remember that most things make no difference. Being busy is a form of laziness – lazy thinking and indiscriminate action. Being overwhelmed is often as unproductive as doing nothing and is far more unpleasant. Being selective – doing less – is the path of the productive. Focus on the important few and ignore the rest.
- If you haven’t identified the mission-critical tasks and set aggressive start and end times for their completion, the unimportant becomes the important. Even if you know what’s critical, without deadlines that create focus, the minor tasks forced upon you (or invented) will swell to consume time until another bit of minutiae jumps in to replace it, leaving you at the end of the day with nothing accomplished.
- The key to having more time is doing less, and there are two paths to getting there, both of which should be used together:
- Define a short to-do list
- Define a not-to-do list
- Don’t ever arrive at the office or in front of your computer without a clear list of priorities. There should be no more than 2 mission-critical items to complete each day. Never. It just isn’t necessary if they’re actually high-impact.
We want performant, scalable, and quality software. We wish to build and test applications that our customers profess their love to and share to their friends.
- We have nonexistent to little unit, performance, API, and integration tests
- The organization do not closely monitor feature usage statistics
- Some of us do not exactly feel the pains our customers face
- We don’t have notifications for outdated dependencies, messy migration scripts, among other failures
- Some are not curious about understanding how the apps they test and own actually work
- We have not implemented continuous build tools
- It is a pain to setup local versions of our applications, even to our own programmers
- We do not write checks alongside development, we lack executable specifications
- Some still think that testing and development happen in silos
- It is difficult to get support for useful infrastructure, as well as recognition for good work
- Many are comfortable with the status quo
It seems that we usually have our expectations mismatched with our practices. We’re frequently eager to show off our projects but are in many instances less diligent in taking measures about baking quality in, and therefore we fail more often than not. What we need are short feedback loops, continuous monitoring, and improved developer productivity, ownership, and happiness. The difficult thing is, it all starts with better communication and culture.
Then start small. Really, small.
Sure, write your big goals on (physical or digital) paper, and dream about all the great things you can do when you’re slimmer and healthier, more skillful in your art, more financialy secure, or more social, but please don’t expect those changes to just magically happen because you listed and thought about them. They’re necessary steps for the new you that you’re excited to be, but incomplete. All artists want to draw better, all musicians dream of playing music in the grandest stage, all writers think about being successful authors, all of us want to be fit, want to be less socially awkward, and want to travel more. We all want to make a living where we both have fun and generate enough income for our families. Big goals are fun to think of, but they can be overwhelming. Vague goals are easy to list but they are difficult to implement in our lives.
So start small. Pick the most important goal that you have in your list, and if it is big, split, slice, break it down to incredibly tiny, specific, chewable pieces. Instead of drawing better, you could re-write your goal to draw one portrait (or landscape or idea) for an hour every Saturdays of January in your favorite place. If you want to lose some pounds, you could start by religiously running every morning of every other day of this month just for 30 minutes in a park somewhere near, before everybody else wakes up. If you want to write a novel, you could timebox yourself to writing something about it every day for one month just for an hour right after you wake up, before going to work. If you want to learn a new skill, give yourself 30 minutes to research and practice it everyday at work. Test yourself, experiment for a month without excuses, see if you can do these small things and like your results. If you do, then do them again. If you don’t, ask yourself why. The thing to remember here is that in order to make big stuff come true, you need to make small changes first, no shortcuts. You should make few but bold decisions, you need to re-build your systems of working, you must review and replace habits that don’t work with ones that do, because that’s where it all really starts.
For new apps and software features, it’s perfectly normal to exhaustively test the matrix of all possible scenarios with equal importance before release – the happy path, the confused path, the forgetful or greedy paths, etcetera – whenever the tester can. After all, no one knows yet how exactly the new application will behave in Production, and it is always better (and cheaper) to find problems and fix them early instead of reacting as they occur.
For small software that are already in the wild and which do not change often, it is still a good rule of thumb to test everything with equal priority, if the budget allows for it. It’s still nice to catch bugs before the next version release rather than have a client surprise us at odd times with a system failure.
For complex legacy systems which frequently change and usually do not automatically test themselves, it is still a great idea to test features in full-scale, if we can, if it’s possible to perform all those complicated use cases within hours before the deadline. That’s a noble but a hugely difficult task for testers, if not impossible to accomplish, and something that might be a complete waste of time and creativity, or a foolish thing to do, even if it is very important for teams to have confidence in the stability of their applications. For such systems it might be better to re-evaluate what goals does our testing need to reach, re-think what sort of stuff do we want to find, re-learn what services do we want to keep providing to our customers as well as how effectively do we want to provide those services, and go from there. It might be better to spend some of our time on data monitoring and analysis, on understanding how our clients actually use our products and focus our testing those usage scenarios. It might be better for us to amplify testing and developing the features that our clients love. And it might be more worth it to regularly identify the riskiest parts of the system and improve their design and testability, rather than trying to keep up with the matrix we know we couldn’t possibly deal with everytime.
Apps evolve, and I think that the testing that we perform needs to change too according to our current contexts.
Lately I’ve been thinking about the software development process we have at work, the stuff that has been keeping us in shape for the past few years. It’s a system of smaller systems, which in turn makes up a flow – requirements gathering, sprint planning, sprint iteration and monitoring, demo and retrospectives, testing, project release. Good flow helps deliver quality service, bad flow makes it difficult to produce value.
I actually think we’re actually doing good so far on the flow. 🙂
Still, there are some areas in our current process where I believe we could be many times better in the near future, such as tasks estimation, business rules presentation and documentation, sprint data analysis, testing (especially unit testing), and continuous integration. It’ll take a while, of course. The challenge is for everyone in the team to keep seeking the possible small improvements we can easily implement, baby steps that makes our good flow better than ever. Delivering quality has always been a team effort, not a task reserved to testers (nor to any one person) alone.