Ultimately what makes software testing fun and challenging is because it needs you to interact with people. You have to talk to both the software development team and the application users. You have to understand what they care for and why. You have to ask questions and the answers to those questions guide what you do when you test. You can test in a silo but that doesn’t help you learn anything deeper, you can’t thrive if you stick to the things you already know. You grow because you take in all these perspectives and beliefs that other people have and test them against your own, for the goal of making better judgments about how to test the thing your testing and why you’re testing them in the first place. It’s not easy, because you have to care. The challenge is that you have to learn a great deal of many things, you have to regularly check the landscape of where you are, and you often need to reflect about what you’ve done and what you want to do next. And on the journey of overcoming the challenges you set for yourself is where the fun is.
I’m currently in the midst of a test code overhaul, a re-writing project of sorts. It started about a week ago and so far I’ve made considerable progress on what I’ve wanted to achieve with the rewrite, which is basically cleaner and more maintainable code, mostly in the sense of test data management and test description language. The number of tests running everyday in our Jenkins system has grown noticeably and I’ve felt that it’s been difficult to add certain tests because of how I structured the test data in the past, which I have not upgraded since then. The two possible avenues for running tests – on the UI and HTTP layers – also adds a bit of complexity and it’d be nice if I can integrate the two smoothly. It’s an interesting development because I did not plan on any re-writing to be done anytime soon but I guess at the back of my mind I knew it’ll happen eventually. And so I decided to take a step back from writing more tests and do some cleanup before it gets tougher to change things. I plan to finish everything in about a month or so.
At the moment, I’m reminded of the phases I’ve gone through in learning to code and writing automated checks in the past few years:
- Early 2014. It all begins with Selenium IDE, with giving the self some time to study the basic Selenese commands for writing automated checks and (more importantly) understand how to properly retrieve the page elements you want to manipulate.
- Mid 2014. Test management in Selenium IDE becomes difficult as the number of tests grow, hence the decision to switch to Selenium WebDriver. The only programming language background I had back then was C++, which was limited to only functions and logical/conditional operators, so I chose Java to work with to lessen the learning curve.
- Late 2014. Familiarized myself with Git, which hooked me on making daily commits and appreciating version control. Along the way I learned the concepts of classes and objects.
- All of 2015 up to Early 2016. I was in a trance, writing code daily and pushing myself to create all the automated checks that I wanted to run for our apps before every release. Tests run on the Eclipse IDE using TestNG and I was happy with what I had, except that those end-to-end tests are really slow. Running everything took overnight to finish, which was okay for my employer but annoying for me personally.
- Mid 2016. Re-writing existing tests in Ruby with Cucumber integration started off (when I found Jeff Morgan’s “Cucumber & Cheese” book online) as a side project for fun and testing my skill level in programming. And I did have buckets of fun! The experiment told me that there’s still a lot I need to practice on if I want to write better code, and it also told me that I can be more productive if I switch programming languages. There’s a bit less code to type in when writing code in Ruby than Java and I liked that, plus all the interesting libraries I can use. I switched to Sublime Text and used both Jenkins and the command-line interface more extensively too.
- Late 2016. As I was looking for ways to speed up end-to-end tests total execution, which by then takes about 4 hours to complete, I ended up exploring testing apps in the HTTP layer instead of in the UI. That took a lot of studying of how our apps actually behave under the hood, what data are being passed around, how images are actually sent, how to view pages without a browser, how redirections work, among other things. After years of testing apps via the user interface, this was such a refreshing and valuable period, and I completely wondered why I never knew such a thing existed until then. It wasn’t being taught extensively to testers, perhaps because it all depends on how the app was structured to run through an API.
And these phases brings me to now, where there’s a healthy dose of API and UI layer tests all checking major app features. It’s all good, just several pieces needing a cleanup, a little parallelization, better test description language, and great documentation. It’s all good, because the lessons in both programming and testing keep piling. The two practices differ in mindset but I think they complement each other, and I think that there’s no reason anyone can’t do both.
How does one become a creative person? Are we only being creative when we’re delving into the realm of the arts? Why do we say that some people are more creative than others? Does creativity contribute to success? We admire accomplished illustrators, actors, programmers, singers, writers, painters, dancers, poets, and designers of all kinds because of their creativity in their chosen fields of endeavor, because we feel they have good taste in whatever it is that they do, and often we desire to be like them, wishing to churn out great pieces of work too in our lifetimes. But what does it really take to be creative? Scott Berkun gives his take on his latest book, “The Dance of the Possible: The Mostly Honest Completely Irreverent Guide to Creativity“.
Personally, I don’t think much about what it means to be creative. I feel that thinking about its meaning doesn’t practically help me, because I’ve learned through experience that that only way to create good work is to do it, failing multiple times, learning along the way, and then succeeding. That’s what creativity has always meant for me – building things or solving problems that matter (to me or to other people I provide service to), whenever the itch is there or if the problems bug me enough. And reading Scott’s book felt like a confirmation of some sort of the beliefs I’ve had about creativity, plus a reminder to handle some of my habits better.
Here are some favorite lines from the book:
- It’s far wiser to think about the effect you want an idea to have. If the goal is to make someone laugh, fix their car or increase the revenue of the widgets their company makes, that matters far more than how “creative” an idea is or is not.
- We always have more freedom than we think, we just forget. We spend so much time trying to be efficient that doing anything interesting feels like a waste of time. And in this tendency is another misconception: creativity is rarely efficient. It always involves taking chances and trying things that might work but might not. The question then is: are we willing to spend time to be interesting, to think interesting thoughts and make interesting things? We all have the power, but perhaps not the willingness.
- To create means to make something new, at least for you, and to do something new is like going off of the map, or more precisely, deliberately choosing to go to a part of the map that is unknown. In this case it rarely matters where or how you start.
- You will, no matter how talented you are, have your finished works challenged and called not-so-good names. Good is surprisingly subjective, and the more creative the domain you work in, the more subjective it is.
- The ability to see an idea, or a thing, from many different perspectives is among the greatest assets a thinking person can have. Of course you don’t have to agree with someone else’s perspective, but you owe it to yourself to try to see what they do. They may see something important you’ve never noticed before, however small, that can improve what you’re making or what you make in the future.
- I don’t know a single productive creative person who doesn’t have a scrap pile of unfinished projects at different states of completion. This is not waste, but a precious archive of projects that might need to breathe, or of spare parts that may be perfect for other projects.
- Some movies and books are poorly received when they’re released but become popular decades later. Others are big hits at first but fade over time. What is good? The answer depends on what your goals are and what problems you choose to solve.
- No matter how great your idea is, there will be energy you have to spend, often on relatively ordinary work, to deliver it to the world.
- Take pleasure in making things for the sake of making them: what a gift to have the time to make at all! If you were born 200 years ago, or to different parents in a different country, you wouldn’t have the time to feel bad about your work, because you wouldn’t have the wealth and time required to try. If you feel love for your craft, honor it by showing up, even when it’s hard. Especially when it’s hard. Working when it’s hardest often teaches rare lessons that will earn you easy rides now and then. Take pleasure in small progressions when you see them, and know those hard-won gains are the only way anyone in history has ever achieved anything noteworthy—for themselves or for the world.
- If you truly love your craft, there is an infinite number of projects in your future. There will be other chapters. There will be other canvases and other songs. Perfection is a prison and a self- made one. Whatever you’re making, it doesn’t have to be perfect. Perfection is an illusion.
- Expect to be rejected. You will be. It will happen no matter how successful you are.
- Do one project for commerce and one for art. It’s an interesting approach: maybe the best work can only be made if it serves only one master at a time. It’s a healthy exercise to both make something entirely for yourself and entirely for other people. In each case you will stretch your boundaries for what you are capable of, as so often those conflicting desires of satisfying ourselves and satisfying other people bind us to conservative choices.
- Too often we are distracted away from what we say is important by things that are more pleasurable or convenient. This means a central skill any creative person needs is a mastery of time, which means a mastery of habits. There will always be easier things in our lives than creative work. There will always be demands on our time that are more logical and lucrative than chasing an idea. If you are truly passionate about something you must be willing to make sacrifices to make it possible. What good is that passion if you can’t use it to help you do the work? Merely saying you are passionate, or feeling passionate, is not enough.
- The simplest habit is to work on your project every day. If you don’t have a project, go to your private journal or drawing notebook daily until you do. It can be for ten minutes or an hour, but you must touch the work at least once a day. It can be in the morning, or late at night, or during your lunch break at work. At first when and where won’t matter. All that counts is that you commit to the discipline of honoring your ideas.
- We all have the same time limit of 24 hours every day, which means the difference between a productive creative person and an unsatisfied dreamer is in how they choose to use it. Most of us forget how much of our time goes to entertainment, things we do purely for pleasure. We have plenty of time—it’s just we have to protect it for the things we claim are most important.
- “Will anyone care about my work?” people often ask. Yes—you. It starts with you.
If it feels difficult, exhausting, and inconvenient to update an existing test (or system) code, it may be that you don’t completely understand what the code does or it may be that the code really is written badly. Either way both situations present a learning opportunity: the former to fully grasp the business rules for the particular code you want to change, the latter to review the purpose of the code and to rewrite it in a way that’s helpful to the team.
Complaining doesn’t help; it only prolongs the agony. Care instead, discuss with the team to check if the change is actually necessary, and, if you think that it is, leave the code better than when you found it.
Timothy Ferriss‘ “The 4-Hour Workweek” might be my favorite book this year. Tons of lessons about how to be productive, as well as how to enjoy life to the fullest, and reading the book has left me excited about the future while reconsidering many of my other choices.
Some favorite takeaways:
- The perfect job is the one that takes the least time. The goal is to free time and automate income.
- ‘If only I had more money’ is the easiest way to postpone the intense self-examination and decision-making necessary to create a life of enjoyment – now and not later. Busy yourself with the routine of the money wheel, pretend it’s the fix-all, and you artfully create a constant distraction that prevents you from seeing just how pointless it is. Deep down, you know it’s all an illusion, but with everyone participating in the same game of make-believe, it’s easy to forget.
- What are you waiting for? If you cannot answer this without resorting to the previously rejected concept of good timing, the answer is simple: You’re afraid, just like the rest of the world. Measure the cost of inaction, realize the unlikelihood and repairability of most missteps, and develop the most important habit of those who excel, and enjoy doing so: action.
- Retirement as a goal or final redemption is flawed for at least three solid reasons:
- It is predicated on the assumption that you dislike what you are doing during the most physically capable years of your life.
- Most people will never be able to retire and maintain even a hotdogs-for-dinner standard of living. Even one million is chump change in a world where traditional retirement could span 30 years and inflation lowers your purchasing power 2-4% per year. The math doesn’t work.
- If the math doesn’t work, it means that you are one ambitious hardworking machine. If that’s the case, guess what? One week into retirement, you’ll be so damn bored that you’ll want to stick bicycle spokes in your eyes.
- If it isn’t going to devastate those around you, try it and then justify it. Get good at being a troublemaker and saying sorry when you really screw up. Ask for forgiveness, not permission.
- Ninety-nine percent of people in the world are convinced they are incapable of achieving great things, so they aim for the mediocre. The level of competition is thus fiercest for ‘realistic’ goals, paradoxically making them the most time- and energy-consuming. So do not overestimate the competition and underestimate yourself. You are better than you think.
- Doing something unimportant well does not make it important. Requiring a lot of time does not make a task important. What you do is infinitely more important than how you do it. Efficiency is still important, but is useless unless applied to the right things.
- Remember that most things make no difference. Being busy is a form of laziness – lazy thinking and indiscriminate action. Being overwhelmed is often as unproductive as doing nothing and is far more unpleasant. Being selective – doing less – is the path of the productive. Focus on the important few and ignore the rest.
- If you haven’t identified the mission-critical tasks and set aggressive start and end times for their completion, the unimportant becomes the important. Even if you know what’s critical, without deadlines that create focus, the minor tasks forced upon you (or invented) will swell to consume time until another bit of minutiae jumps in to replace it, leaving you at the end of the day with nothing accomplished.
- The key to having more time is doing less, and there are two paths to getting there, both of which should be used together:
- Define a short to-do list
- Define a not-to-do list
- Don’t ever arrive at the office or in front of your computer without a clear list of priorities. There should be no more than 2 mission-critical items to complete each day. Never. It just isn’t necessary if they’re actually high-impact.
What: Create executable specifications for an ongoing project sprint. Executable specifications are written examples of a business need that can be run anytime and acts as a source of truth for how applications behave. It is a living documentation of what our software does, and it helps us focus more on solving the unusual and noteworthy problems instead of wasting time with the ones that we know we shouldn’t have been worrying about.
- Join a software development team for one sprint duration.
- Discuss project tasks for that sprint with everyone.
- Ask about requirements examples and create executable specifications for those tasks as the application is being written.
- Refine the specifications when necessary.
- Continuously update the specifications until they pass.
- Keep the specifications running on a preferred schedule.
Why: To see if writing executable specifications alongside development is feasible during the duration of a sprint.
Limitations: Tester writes the executable specifications, programmers work as is.
- Writing executable specifications can be done during actual app development, provided that tester is experienced with the tools for implementing it and understands well why the need for such specifications.
- It is of course more beneficial if everyone in the team learn how executable specifications work and write/run the specifications, and why the need to implement it.
- It will take quite a long while before executable specifications becomes a norm in the existing software development process, if ever. This is a function of whether everyone believes that such specifications are actually useful in practice, and then building the skills and habit of including these specifications in the team’s definition of done.
Automated checking is not a new concept. Gojko Adzic, however, provides us a way to make better integration of it in our software development processes. In his book titled “Specification by Example”, he talks about executable specifications that double as a living documentation. These are examples which continuously exercise business rules, they help teams collaborate, and, along with software code, they’re supposed to be the source of truth for understanding how our applications work. He builds a strong case about the benefits of writing specifications by example by presenting case studies and testimonials of teams who have actually used it in their projects, and I think that it is a great way of moving forward, of baking quality in.
Some favorite takeaways from the book:
- Tests are specifications; specifications are tests.
- “If I cannot have the documentation in an automated fashion, I don’t trust it. It’s not exercised.” -Tim Andersen
- Beginners think that there is no documentation in agile, which is not true. It’s about choosing the types of documentation that are useful. There is still documentation in an agile process, and that’s not a two-feet-high pile of paper, but something lighter, bound to the real code. When you ask, “does your system have this feature?” you don’t have a Word document that claims that something is done; you have something executable that proves that the system really does what you want. That’s real documentation.
- Fred Brooks quote: In The Mythical Man-Month 4 he wrote, “The hardest single part of building a software system is deciding precisely what to build.” Albert Einstein himself said that “the formulation of a problem is often more essential than its solution.”
- We don’t really want to bother with estimating stories. If you start estimating stories, with Fibonacci numbers for example, you soon realize that anything eight or higher is too big to deliver in an iteration, so we’ll make it one, two, three, and five. Then you go to the next level and say five is really big. Now that everything is one, two, and three, they’re now really the same thing. We can just break that down into stories of that size and forget about that part of estimating, and then just measure the cycle time to when it is actually delivered.
- Sometimes people still struggle with explaining what the value of a given feature would be (even when asking them for an example). As a further step, I ask them to give an example and say what they would need to do differently (work around) if the system would not provide this feature. Usually this helps them then to express the value of a given feature.
- QA doesn’t write [acceptance] tests for developers; they work together. The QA person owns the specification, which is expressed through the test plan, and continues to own that until we ship the feature. Developers write the feature files [specifications] with the QA involved to advise what should be covered. QA finds the holes in the feature files, points out things that are not covered, and also produces test scripts for manual testing.
- If we don’t have enough information to design good test cases, we definitely don’t have enough information to build the system.
- Postponing automation is just a local optimization. You might get through the stories quicker from the initial development perspective, but they’ll come back for fixing down the road. David Evans often illustrates this with an analogy of a city bus: A bus can go a lot faster if it doesn’t have to stop to pick up passengers, but it isn’t really doing its job then.
- Workflow and session rules can often be checked only against the user interface layer. But that doesn’t mean that the only option to automate those checks is to launch a browser. Instead of automating the specifications through a browser, several teams developing web applications saved a lot of time and effort going right below the skin of the application—to the HTTP layer.
- Automating executable specifications forces developers to experience what it’s like to use their own system, because they have to use the interfaces designed for clients. If executable specifications are hard to automate, this means that the client APIs aren’t easy to use, which means it’s time to start simplifying the APIs.
- Automation itself isn’t a goal. It’s a tool to exercise the business processes.
- Effective delivery with short iterations or in constant flow requires removing as many expected obstacles as possible so that unexpected issues can be addressed. Adam Geras puts this more eloquently: “Quality is about being prepared for the usual so you have time to tackle the unusual.” Living documentation simply makes common problems go away.
- Find the most annoying thing and fix it, then something else will pop up, and after that something else will pop up. Eventually, if you keep doing this, you will create a stable system that will be really useful.