6 Curious New Tools to Try for Writing Automated Checks for Browser Apps

While I don’t find myself writing a lot of browser-based automated checks these days, I still am on the look out for interesting new tools in that space. The reason: the new tool solves an existing problem I have with setting up such a testing suite from scratch or provides a solution for certain curious use cases I’ve never experienced before. While using Ruby and Watir together in writing tests running through the browser for me is sufficient for common tasks, such a new tool could be a better fit for another project.

Here’s a list of such tools that popped up in my feed in recent months:

  • Cypress. What I like about Cypress, aside from the standalone package installation option and the built-in pretty test report page, is that the pre-defined browser tests that the actual team runs on its own site is included out-of-the-box. This way they made it easy for me to write custom tests; I just had to search for an example of what I wanted to do, copy-pasted it to my own test, and updated the parts that needed changing. Tests are written in Javascript. I have yet to try running the tests via the terminal though, which is important when running tests on a CI server. Using their test runner is free for all projects, however there is a pricing plan for using their dashboard service which helps keep test recordings private.
  • Katalon Studio. This is a full-blown automation solution that is completely free. There’s a pricing plan for business support services. The record-and-playback feature built-in to the tool failed to impress me when I ran it through our legacy apps, but perhaps writing the actual test code through their GUI fares better (using which there will be a high learning curve for people like me who like to use the CLI and personally-configured IDEs).
  • PuppeteerBuild to control Google’s headless Chrome or Chromium browser, running over the DevTools protocol. Tests are written in Javascript. Easy to try and get into using their web playground. Alister Scott has tried it running with Mocha and Circle CI on a demo project.
  • Chromeless. Similar to Puppeteer, but built to automate an army of Chrome browsers running in parallel. It gives us the option to run tests on AWS Lamba too. Again, tests are written in Javascript, which we can try on their demo playground.
  • Laravel Dusk. This gives PHP developers familiar with Laravel the ability to write and run their own browser app tests, using a programming language they’re much accustomed to.
  • Appraise. Similar to BackstopJS, a tool for visually validating browsers apps. Tests are written in Markdown.
Advertisements

Lessons Learned from Richard Bach’s “Curious Lives: Adventures from the Ferret Chronicles”

I always start the new year like most people, spending quality time with friends and family, some reflection and goal setting, with good food, many cheers, and hearty laughs. I also often choose a wonderful book or two to go on an adventure with during the holidays, because it is always worth the while. This time, I got lucky finding a paperback copy of an old Richard Bach collection, titled “Curious Lives: Adventures from the Ferret Chronicles”, was glad to meet up with ferret friends, old and new both, and got valuable reminders about the courtesies and living accordingly to our highest sense of right, along with the fun ride.

Here are my favorite lines from the book:

  • Whatever harm I would do to another, I shall do first to myself. As I respect and am kind to myself, so shall I respect and be kind to peers, to elders, to kits. I claim for others the freedom to live as they wish, to think and believe as they will. I claim that freedom for myself. I shall make each choice and live each day to my highest sense of right.
  • Once, long ago, we changed our minds: end violence. In its place, no matter what: courtesy.
  • If you excel at your craft, there is a good chance that curious ferrets will need to know why, to find out what makes you different.
  • With the adventures we choose and the mysteries we solve we build our own credentials, write our own introduction to others around the world who value adventure and mystery themselves.
  • Trust. There’s a light, when we close our eyes, the light of what we want to do more than anything else in the world. Trust that light. Follow, wherever it leads.
  • Giving our visions and stories and characters to become friends to others lifts not only ourselves but the world and all its futures.
  • “There’s a time to work on a book and you know it,” said the muse. “There’s a time to think about the story, a time to care about your readers, your publisher, about rhythm and timing and grammar and spelling and punctuation, about design and advertising and publicity. But none of those times, Budgeron, is when you’re writing!”
  • Her husband had told her long ago that she didn’t need to please everyone with her stories – if a book pleases only half of one percent of the reading public, though no one else bought a single copy, it will be a massive bestseller.
  • Budgeron Ferret had chosen to be a writer. With his choice came poverty, loneliness, rejection, frustration, despair, perseverance, delight, attention, riches, love, understanding, fulfilment, a life of ideas that mattered to him, shared now and then with kings and kits.
  • How strange, he thought. Find the greatest teachers, ask the hardest questions, they never say, ‘Study philosophy’, or, ‘Get your degree’. They say, ‘You already know’.
  • The mark of true flight is not our altitude but our attitude, not our speed but our joy in the paths we find above the earth.
  • No one taught her, but she knew: more important than talent or gifts or education is the determination to make one’s wish come true.
  • “Vink, if you want to meet the one ferret who can fix any trouble, no matter how bad it is, the one who can bring you happiness when nobody else can do it – why, just look in the mirror and say hello.”

Lessons Learned From Gergely Revay’s “Web Hacking: Become a Pentester” Online Course

The realm of security testing is something I have not explored yet in deep detail not because it’s not an interesting field but because I have always found it to be intimidating, stuffed with jargons and specialized tools to learn. But the curiosity is there, and I’ve decided late last year that I want to get better at it. For that reason I’m glad that Gergely Revay has opened an online course on becoming a web pentester this year. Great timing! And very practical too because I was able to directly apply what I learned on the course at work. 🙂

As with any skill, we master it through practice. But here are some notes about the key ideas I learned from the course:

  • Security testing requires exploratory testing. A tester can only find out where the security vulnerabilities are when such person has good understanding of what risks are present in the application, and one can only know about what the risks are when one has vastly explored application behavior in various scenarios as well as the technology stack where it runs.
  • Using JavaScript to create stored cross-site scripts and running them on a vulnerable app is an easy way to annoy users who frequent a page.
  • We can download or view application data (and more) through a system’s insecure file upload feature. Secret configuration files may not be as safe as we think they are.
  • Kali Linux provides us common word lists that we can use to brute-force attack logins. An account is only as safe as the complexity of its matching password.
  • Getting legitimate users to run a malicious script for an attacker relies on how good the attacker is in manipulating the target person to visit some desired page.
  • It is possible to run operating system or database commands on the server where an application is running.
  • Even if an SQL injection does not provide us details of the query results, as long as the injection works we may still get interesting data from the app through succeeding creative attacks.
  • Applications, as innocent as they may seem, can help an attacker find vulnerabilities through the user experience. Be careful about the hints you provide to users when they fail to authenticate their account, among other possible
  • Because security testing relies so much on a tester’s knowledge of the app under test, security testing is difficult. The deeper the tester know about which features are available and how they work, both in the user interface level and in the background, the better the chances of the tester finding security vulnerabilities.

Takeaways from Margaret Heffernan’s “Willful Blindness”

To answer a question about exploratory testing, Alister Scott recommends testers to read a Margaret Heffernan book, titled “Willful Blindness“. He tells us that we have to be less blind when we’re exploring in order to find bugs in systems under test. We have to keep on looking, we have to continuously question things, we have to choose to know and understand how the system works. Reading Margaret’s book has helped me realize what being willfully blind meant and how we become blind without noticing. It has helped me be more aware of the different ways I can misjudge things, and thus helps me get better. Cognitive limits, biases, division of labor, money, hierarchy, relationships, feelings of belonging or ostracism, all these and more play a part in how we behave in various situations. They affect how we perform our software testing too.

Some takeaways:

  • We can’t notice and know everything: the cognitive limits of our brain simply won’t let us. That means we have to filter or edit what we take in. So what we choose to let through and to leave out is crucial. We mostly admit the information that makes us feel great about ourselves, while conveniently filtering whatever unsettles our fragile egos and most vital beliefs.
  • Most people marry other people very like themselves: similar height, weight, age, background, IQ, nationality, ethnicity. We may think that opposites attract, but they don’t get married. Sociologists and psychologists, who have studied this phenomenon for decades, call it “positive assortative mating” – which really just means that we marry people like ourselves. When it comes to love, we don’t scan a very broad horizon. People may have an interest in people who are different from themselves but they don’t marry them. They’re looking for confirmation, for comfort.
  • All personalization software does the same thing: make our lives easier by reducing overwhelming choice. And software is doing it the same way that our brain does, by searching for matches. This is immensely efficient: It means that the brain can take shortcuts because it is working with what it already knows, not having to start from scratch. When we find what we like, part of our pleasure is the joy of recognition. But the flip side of that satisfaction is that we are rejecting a lot along the way.
  • We like ourselves, not least because we are known and familiar to ourselves. So we like people similar to us – or that we just imagine might have some attributes in common with us. They feel familiar too, and safe. And those feelings of familiarity and security make us like ourselves more because we aren’t anxious. We belong. Our self-esteem rises. We feel happy. Human beings want to feel good about themselves and to feel safe, and being surrounded by familiarity and similarity satisfies those needs very efficiently. The problem with this is that everything outside that warm, safe circle is our blind spot.
  • Bias is pervasive among all of us, whether we think we’re biased or not.
  • The argument for diversity is that if you bring together lots of different kinds of people, with a wide range of education and experience, they can identify more solutions, see more alternatives to problems, than any single person or homogenous group ever could. Groups have the potential, in other words, to be smarter than individuals; that’s the case put forward so compellingly by James Surowiecki in his book, The Wisdom of Crowds. But the problem is that, as our biases keep informing whom we hire and promote, we weed out that diversity and are left with skyscrapers full of people pretty much the same.
  • But while it’s true that all of us now have access to more information than ever before in history, for the most part we don’t use it. Just like newspapers, we read the blogs that we agree with – but there we encounter a virtually infinite echo chamber, as 85 percent of blogs link to other blogs with the same political inclination.
  • Our blindness grows out of the small, daily decisions that we make, which embed us more snugly inside our affirming thoughts and values. And what’s most frightening about this process is that as we see less and less, we feel more comfort and greater certainty. We think we see more – even as the landscape shrinks.
  • Indeed, there seems to be some evidence not only that all love is based on illusion—but that love positively requires illusion in order to endure. When you love someone, he or she may even start to adapt to your illusion of him or her. So there is a kind of virtuous circle: you think better of your beloved who starts to live up to your illusions and so you love him or her more. It sounds a little like a fairy tale, but kissing frogs may make them act like princes or princesses. It is indeed a kind of magic, illusions transforming reality. We don’t have to love people for who they are but for who we think they are, or need them to be. This is something everyone does: overlook the flaws, discount the disappointments, focus on what works. Our love for each other allows us, even compels us, to see the best in each other.
  • One of the many downsides of living in communities in which we are always surrounded by people like ourselves is that we experience very little conflict. That means we don’t develop the tools we need to manage conflict and we lack confidence in our ability to do so. We persuade ourselves that the absence of conflict is the same as happiness, but that trade-off leaves us strangely powerless.
  • Because it takes less brain power to believe than to doubt, we are, when tired or distracted, gullible. Because we are all biased, and biases are quick and effortless, exhaustion makes us favor the information we know and are comfortable with. We’re too tired to do the heavier lifting of examining new or contradictory information, so we fall back on our biases, the opinions and the people we already trust.
  • People stay silent at work—bury their heads in the sand—because they don’t want to provoke conflict by being, or being labeled, troublemakers. They may not like the status quo but, in their silence, they maintain it, believing (but also ensuring) the status quo can’t be shifted.
  • Hierarchies, and the system of behaviors that they require, proliferate in nature and in man-made organizations. For humans, there is a clear evolutionary advantage in hierarchies: a disciplined group can achieve far more than a tumultuous and chaotic crowd. Within the group, acceptance of the differing roles and status of each member ensures internal harmony, while disobedience engenders conflict and friction. The disciplined, peaceful organization is better able to defend itself and advance its interests than is a confused, contentious group that agrees on nothing. The traditional argument in favor of hierarchies and obedience has been that of the social contract: It is worth sacrificing some degree of individuality in order to ensure the safety and privileges achieved only by a group. When the individual is working alone, conscience is brought into play. But when working within a hierarchy, authority replaces individual conscience. This is inevitable, because otherwise the hierarchy just doesn’t work: too many consciences and the advantage of being in a group disappears. Conscience, it seems, doesn’t scale.
  • Human beings hate being left out. We conform because to do so seems to give our life meaning. This is so fundamental a part of our evolutionary makeup that it is strong enough to make us give the wrong answers to  questions, as in Asch’s line experiments, and strong enough to make us disregard the moral lessons we’ve absorbed since childhood. The carrot of belonging and the stick of exclusion are powerful enough to blind us to the consequences of our actions.
  • Independence, it seems, comes at a high cost.
  • The larger the number of people who witness an emergency, the fewer who will intervene. The bystander effect demonstrates the tremendous tension between our social selves and our individual selves. Left on our own, we mostly do the right thing. But in a group, our moral selves and our social selves come into conflict, which is painful. Our fear of embarrassment is the tip of the iceberg that is the ancient fear of exclusion, and it turns out to be astonishingly potent. We are more likely to intervene when we are the sole witness; once there are other witnesses, we become anxious about doing the right thing (whatever that is), about being seen and being judged by the group.
  • It is so human and so common for innovation to fail not through lack of ideas but through lack of courage. Business leaders always claim that innovation is what they want but they’re often paralyzed into inaction by hoping and assuming that someone else, somewhere, will take the risk.
  • The greatest evil always requires large numbers of participants who contribute by their failure to intervene.
  • Technology can maintain relationships but it won’t build them. Conference calls, with teams of executives huddled around speakerphones, fail to convey personality, mood, and nuance. You may start to develop rapport with the person who speaks most—or take an instant dislike to him or her. But you’ll never know why. Nor will you perceive the silent critic scowling a thousand miles away. Videoconferencing distracts all its participants who spend too much time worrying about their hair and whether they’re looking fat, uncomfortable at seeing themselves on screen. The nervous small talk about weather—it’s snowing there? It’s hot and sunny here—betrays anxiety about the vast differences that the technology attempts to mask. We delude ourselves that because so many words are exchanged—e-mail, notes, and reports—somehow a great deal of communication must have taken place. But that requires, in the first instance, that the words be read, that they be understood, and that the recipient know enough to read with discernment and empathy. Relationships—real, face-to-face relationships—change our behavior.
  • The division of labor isn’t designed to keep corporations blind but that is often its effect. The people who manufacture cars aren’t the people who repair them or service them. That means they don’t see the problems inherent in their design unless a special effort is made to show it to them. Software engineers who write code aren’t the same as the ones who fix bugs, who also aren’t the customer-service representatives you call when the program crashes your machine. Companies are now organized—often for good reasons—in ways that can facilitate departments becoming structurally blind to one another.
  • We want money for a very good reason: it makes us feel better. Money does motivate us and it does make us feel better. That’s why companies pay overtime and bonuses. It may not, in and of itself, make us absolutely happy—but, just like cigarettes and chocolate, our wants are not confined to what’s good for us. The pleasure of money is often short-lived, of course. Because there are always newer, bigger, flashier, sweeter products to consume, the things we buy with money never satisfy as fully as they promise. Psychologists call this the hedonic treadmill: the more we consume, the more we want. But we stay on the treadmill, hooked on the pleasures that, at least initially, make us feel so good.
  • Motivation may work in ways similar to cognitive load. Just as there is a hard limit to how much we can focus on at one moment, perhaps we can be motivated by only one perspective at a time. When we care about people, we care less about money, and when we care about money, we care less about people. Our moral capacity may be limited in just the same way that our cognitive capacity is.
  • Money exacerbates and often rewards all the other drivers of willful blindness: our preference for the familiar, our love for individuals and for big ideas, a love of busyness and our dislike of conflict and change, the human instinct to obey and conform, and our skill at displacing and diffusing responsibility. All these operate and collaborate with varying intensities at different moments in our life. The common denominator is that they all make us protect our sense of self-worth, reducing dissonance and conferring a sense of security, however illusory. In some ways, they all act like money: making us feel good at first, with consequences we don’t see. We wouldn’t be so blind if our blindness didn’t deliver the benefit of comfort and ease.
  • Once you are in a leadership position, no one will ever give you the inner circle you need. You have to go out and find it.
  • We make ourselves powerless when we choose not to know. But we give ourselves hope when we insist on looking. The very fact that willful blindness is willed, that it is a product of a rich mix of experience, knowledge, thinking, neurons, and neuroses, is what gives us the capacity to change it. We can learn to see better, not just because our brain changes but because we do. As all wisdom does, seeing starts with simple questions: What could I know, should I know, that I don’t know? Just what am I missing here?

Favorite Talks from Sauce Conference 2017

The European Selenium Conference is underway in Berlin but before the talks for that event get uploaded online I thought I should finish binge-watching this year’s Sauce Conference videos first. Many of the talks in SauceConf 2017 are actually similar in feel to those I’ve watched in previous Selenium Conferences, except that a number of them point out how awesome Sauce Labs‘ service has been for their cross-browser and cross-platform testing needs. Not that I mind nor I disagree, even if I have not used Sauce Labs’ extensively. Test automation in the web application space has been mostly stable in recent years and that’s a good thing. Now we are left with discussing the consequences of the automated systems that we have put in place and focusing on other areas which needs further improvement.

Anyway, here are my favorite talks from the conference:

Three Recommended Paid Software Testing Online Courses

Free online courses are nice, but some paid ones are just better at delivering value especially when instructors take a lot of care about building the best possible content they can provide to their audience. Free is always an option, but the problem with free is that we don’t necessarily have to give anything back in return for a service. We can do nothing and that’s okay. With paid courses however I find myself to be more motivated to take in everything I can. I focus more and I believe that helps me learn better.

Here are three recommended paid software testing courses I’ve taken recently, from which I’ve learned a great deal:

Lessons from Lucius Seneca’s “On The Shortness Of Life”

One interesting topic I’ve come across this year in a few podcasts I listen to is Stoicism. William Irvine introduced the school of thought in an episode of the Art of Manliness podcast and Ryan Holiday wrote a practical guide about it for entrepreneurs on The Tim Ferriss Show. It has practical applications in daily life, and because of that I’d like to eventually learn more about what to practice and its how’s and why’s. For now though, I thought that reading up on Seneca’s essay ‘On the Shortness of Life‘ would be a good starting point.

Here are some notes from the essay:

  • It’s not that we have a short time to live, but that we waste much of it. Life is long enough, and it’s been given to us in generous measure for accomplishing the greatest things, if the whole of it is well invested. But when life is squandered through soft and careless living, and when it’s spent on no worthwhile pursuit, death finally presses and we realize that the life which we didn’t notice passing has passed away. So it is: the life we are given isn’t short but we make it so; we’re not ill provided but we are wasteful of life.
  • Look at those whose prosperity draws crowds: they are choked by their own goods. How many have found their wealth a burden! How many are drained of their blood by their eloquence and their daily preoccupation with showing off their abilities! How many are sickly pale from their incessant pleasures! How many are left with no freedom from the multitude of their besieging clients!
  • What foolish obliviousness to our mortality to put off wise plans to our fiftieth and sixtieth year, and to want to begin life from a point that few have reached!
  • The person who devotes every second of his time to his own needs and who organizes each day as if it were a complete life neither longs for nor is afraid of the next day. For what new kind of pleasure is there that any hour can now bring? Everything has been experienced, everything enjoyed to the full. For the rest, fortune may make arrangements as it wishes; his life has already reached safety. Addition can be made to this life, but nothing taken away from it – and addition made in the way that a man who is already satisfied and full takes a portion of food which he doesn’t crave and yet has room for.
  • I am always astonished when I see people requesting the time of others and receiving a most accommodating response from those they approach. Both sides focus on the object of the request, and neither on time itself; it is requested as if it were nothing, granted as if it were nothing. People trifle with the most precious commodity of all; and it escapes their notice because it’s an immaterial thing that doesn’t appear to the eyes, and for that reason it’s valued very cheaply – or rather, it has practically no value at all.
  • The greatest waste of life lies in postponement: it robs us of each day in turn, and snatches away the present by promising the future. The greatest impediment to living is expectancy, which relies in tomorrow and wastes today. You map out what is in fortune’s hand but let slip what’s in your own hand. What are you aiming at? What’s your goal? All that’s to come lies in uncertainty: live right now.
  • There is a common saying that it was not in our power to choose the parents we were allotted, and that they were given to us by chance; yet we can be born to whomever we wish. There are households of the most distinguished intellects: choose the one into which you’d like to be adopted, and you’ll inherit not just the name but also the actual property, which is not to be hoarded in a miserly or mean spirit: the more people you share it with, the greater it will become.
  • Honors, monuments, all that ostentatious ambition has ordered by decree or erected in stone, are soon destroyed: there’s nothing that the long lapse of time doesn’t demolish and transform. But it cannot harm the works consecrated by wisdom: no age will efface them, no age reduce them at all. The next age and each one after that will only enhance the respect in which they are held, since envy focuses on what is close at hand, but we more freely admire things from a distance.
  • It is nevertheless better – believe me – to know the balance sheet of one’s own life than that of the public grain supply.
  • In this mode of life much that is worth studying awaits you: the love and practice of the virtues, forgetfulness of the passions, knowledge of how to live and to die, and deep repose.