Problem: I want to automatically run unit tests, lint the application code, and check it’s state against team standards every time I try to commit my changes to a project. It would be nice if the commit aborts if any of the existing tests fails or if I did not follow a particular standard that the team agrees to uphold. The commit pushes through if there are no errors. If possible, I don’t have to change anything in my software development workflow.
Solution: Use a Git pre-commit hook. Under the .git/hooks hidden folder in the project directory, create a new file called pre-commit (without any file extension) containing something like the following bash script (for testing PHP code):
stagedFiles=$(git diff-index --cached HEAD | grep ".php" | grep "^:" | sed 's:.*[DAM][ \\''t]*\([^ \\''t]*\):\1:g');
errorMessage="Please correct the errors above. Commit aborted."
printf "Linting and checking code standards ..."
for file in $stagedFiles
php -l $file
if [[ $LINTVAL != 0 ]]
php core/phpcs.phar --colors --standard=phpcs.xml $file
if [[ $STANDVAL != 0 ]]
printf "Running unit tests ..."
core/vendor/bin/phpunit --colors="always" [TESTS_DIRECTORY]
if [[ $TESTSVAL != 0 ]]
- linting and code standard checks only runs for the files you want to commit changes to
- code standard checks are based on a certain phpcs.xml file
- unit tests inside a particular TESTS_DIRECTORY will run
- the commit will abort whenever any of the lints, code standard checks, or unit tests fails
There are various levels of testing in software. The one most people are familiar with (including software testers) is testing done through the user interface, which is basically using the application at the end of a software development cycle and finding out whether it does what it is supposed to do. It’s a practice that is easy to understand and natural to do. Most of us have mobile devices or computers at home and in that sense we all understand how to test apps on the UI at some basic level. We explore the functionalities apps say they deliver and we decide for ourselves whether we think those promises are being kept or not. We feel good when everything works well or we feel bad when it is difficult to use the app (and maybe never use that app again). That said, software testing is not limited to the user interface.
The more experienced testers understand that testing is easier to perform and more valuable when it is done early in the software development process, making sure that we are doing the right things and are doing things right, even though we know we can’t test everything all at once. Bugs found before shipping are cheaper and easier to solve than bugs found later. Quick and early feedback is ideal. But to accomplish testing early in the software development process means that testers actually need to understand how software is built from code, not just the code itself and how various pieces of code integrate with one another but how programmers write code too. Like everyone else, programmers are people and human and are fallible. People make mistakes, and people can continue to make mistakes even if they work on projects carefully, because that’s how people and the things they build grow. That’s why testing needs to happen as early as possible. That means testers working alongside programmers in putting systems in place that tests the application simultaneously while it is still being written, even when there is no user interface to see yet. That means recognizing where and when mistakes happen, whether in code or habits or processes, and making it easy to spot them when they happen again. Testing in the user interface will never disappear but we can do better than restricting ourselves to just testing at the end.
For any software tester or any software professional it matters to be curious about the industry we choose to be a part in, to be knowledgeable about the people we work with and the tools we use to perform our best work. It helps to be aware of existing practices and be in the loop with the news, well, because that’s how we find solutions to problems, and sometimes more problems to solve. We use the software we test. We find answers on the web. We try applications that could maybe make us be more productive. We network with people like us on the internet, and we go online, study, and digest whatever we can. That’s part of how we improve our skills. That’s part of how we grow, and help others along the way.
But learning takes time and effort and energy. It’s not just about reading and watching everything, taking every online course and going to every conference there is. Our minds does not work like that. We have to pace ourselves well. We need to take breaks in between, we need time for details to sink in, we need contemplation and scrutiny to guide us where to move next and why exactly. Some reflection happens in discussions with colleagues, family, and friends, while other realizations occur only when we are truly alone with ourselves.
So.. after about a week or so since I asked permission for read-write access to our application code repository I’m glad to say that I’m almost done with setting up a version of our apps locally on my machine. It is necessary because I first need to check my changes if they work locally before committing those changes. No code commits yet until said local apps have the same stability as our apps in Staging.
But there are no unit tests. How would I know if everything works after cloning the app repository and running the local settings? Only one option: I had to run local versions of my Staging application API tests. They’re slower than unit tests but at least they let me know if the apps work on some good enough level.
Running tests on local applications!
We’re in business! 🙂
Most of the problems that I encountered whilst setting up were database problems. That’s because no one was maintaining a small-but-updated version of the database. As told, I had to manually match which queries to run according to what problems my tests found. Not pretty, though I could say that in retrospect looking at the errors and finding the DB fix on my own was good exercise. Not elegant, but it helped me get familiarized a little bit with our applications as code.
There was also no documentation about the application and how to run them on various machines. Guides are important but README files were mostly left blank. I had to rely on programmer friends for clues about what to do next whenever I got stuck.
Such problems took time and patience to solve. I had to take notes about updating certain pieces too. Sometimes, I had to make changes to the code itself in order for some features to not fail locally. And yes, I need to remember not to accidentally commit those changes to the remote repository.
It would be nice if we can just go to some private repository and download an environment image or two which runs smoothly when integrated with the app repository. Update the code on a local machine and the environment updates automatically. Set up would have been done in a matter of minutes, not days. But, alas, that’s a problem worth solving for another day.
The realm of security testing is something I have not explored yet in deep detail not because it’s not an interesting field but because I have always found it to be intimidating, stuffed with jargons and specialized tools to learn. But the curiosity is there, and I’ve decided late last year that I want to get better at it. For that reason I’m glad that Gergely Revay has opened an online course on becoming a web pentester this year. Great timing! And very practical too because I was able to directly apply what I learned on the course at work. 🙂
As with any skill, we master it through practice. But here are some notes about the key ideas I learned from the course:
- Security testing requires exploratory testing. A tester can only find out where the security vulnerabilities are when such person has good understanding of what risks are present in the application, and one can only know about what the risks are when one has vastly explored application behavior in various scenarios as well as the technology stack where it runs.
- We can download or view application data (and more) through a system’s insecure file upload feature. Secret configuration files may not be as safe as we think they are.
- Kali Linux provides us common word lists that we can use to brute-force attack logins. An account is only as safe as the complexity of its matching password.
- Getting legitimate users to run a malicious script for an attacker relies on how good the attacker is in manipulating the target person to visit some desired page.
- It is possible to run operating system or database commands on the server where an application is running.
- Even if an SQL injection does not provide us details of the query results, as long as the injection works we may still get interesting data from the app through succeeding creative attacks.
- Applications, as innocent as they may seem, can help an attacker find vulnerabilities through the user experience. Be careful about the hints you provide to users when they fail to authenticate their account, among other possible
- Because security testing relies so much on a tester’s knowledge of the app under test, security testing is difficult. The deeper the tester know about which features are available and how they work, both in the user interface level and in the background, the better the chances of the tester finding security vulnerabilities.
Last Wednesday afternoon I anxiously asked my boss for permission to make changes on our application code repository. I said I wanted to try fixing some of the reported bugs listed on our tracking system, if there are no other resources available to pass them to. I made a case about myself not posing any problems because of the code review process built into our repository management tool, that there’s no reason for me to merge any changes without getting feedback from a senior developer first.
He smiled at me and gleefully said “Go ahead. I’m not going to stop you.“, to which I beamed and heartily replied “Thanks, boss!”
This is a turning point in my software testing career, to be able to work on the application code directly as needed. It is actually one of my biggest frustrations – to not be able to find out for myself where the bug lives in the code and fix them if necessary. It’s always a pain to be able to do nothing but wait for a fix, and for a fix to be dependent on the resources available. In my head I think that I’m available and maybe I can do something, but I don’t explicitly have access to the application itself and the code that runs it so I can’t do anything until I have the rights to do so. That’s how it always been. Software testers are often not expected to fiddle with code, at least in my experience, especially in the past where automation was not yet known to be useful as a testing tool. Now that I have the skills and the permission to work on the application repository, I feel that my reach for making an impact on application quality has now expanded remarkably well.
Now bug-fixing is not software testing work in the traditional sense. But I figured there’s no harm in trying to fix bugs and learning the nitty-gritty details of how our legacy applications actually run deep in the code. I believe that learning technical stuff helps me communicate better with programmers. It helps me test applications in a more efficient manner too. Of course I have to consistently remind myself that I am a software-tester-first-programmer-second guy and have to be careful not to fill my days playing with code and forgetting to explore our applications themselves. That said, there are ideas I really want to experiment within our software development process, towards the goal of improving code quality and feedback, and I can only tinker with those ideas inside the application repository itself. Dockerized testing environments, code linting, and unit tests are three things I want to start building for our team, ideas that I consider to be very helpful in writing better code but has not been given enough priority through the years.
I think I’m still testing software, just extending the knowledge and practice of the various ways I perform testing.
To answer a question about exploratory testing, Alister Scott recommends testers to read a Margaret Heffernan book, titled “Willful Blindness“. He tells us that we have to be less blind when we’re exploring in order to find bugs in systems under test. We have to keep on looking, we have to continuously question things, we have to choose to know and understand how the system works. Reading Margaret’s book has helped me realize what being willfully blind meant and how we become blind without noticing. It has helped me be more aware of the different ways I can misjudge things, and thus helps me get better. Cognitive limits, biases, division of labor, money, hierarchy, relationships, feelings of belonging or ostracism, all these and more play a part in how we behave in various situations. They affect how we perform our software testing too.
- We can’t notice and know everything: the cognitive limits of our brain simply won’t let us. That means we have to filter or edit what we take in. So what we choose to let through and to leave out is crucial. We mostly admit the information that makes us feel great about ourselves, while conveniently filtering whatever unsettles our fragile egos and most vital beliefs.
- Most people marry other people very like themselves: similar height, weight, age, background, IQ, nationality, ethnicity. We may think that opposites attract, but they don’t get married. Sociologists and psychologists, who have studied this phenomenon for decades, call it “positive assortative mating” – which really just means that we marry people like ourselves. When it comes to love, we don’t scan a very broad horizon. People may have an interest in people who are different from themselves but they don’t marry them. They’re looking for confirmation, for comfort.
- All personalization software does the same thing: make our lives easier by reducing overwhelming choice. And software is doing it the same way that our brain does, by searching for matches. This is immensely efficient: It means that the brain can take shortcuts because it is working with what it already knows, not having to start from scratch. When we find what we like, part of our pleasure is the joy of recognition. But the flip side of that satisfaction is that we are rejecting a lot along the way.
- We like ourselves, not least because we are known and familiar to ourselves. So we like people similar to us – or that we just imagine might have some attributes in common with us. They feel familiar too, and safe. And those feelings of familiarity and security make us like ourselves more because we aren’t anxious. We belong. Our self-esteem rises. We feel happy. Human beings want to feel good about themselves and to feel safe, and being surrounded by familiarity and similarity satisfies those needs very efficiently. The problem with this is that everything outside that warm, safe circle is our blind spot.
- Bias is pervasive among all of us, whether we think we’re biased or not.
- The argument for diversity is that if you bring together lots of different kinds of people, with a wide range of education and experience, they can identify more solutions, see more alternatives to problems, than any single person or homogenous group ever could. Groups have the potential, in other words, to be smarter than individuals; that’s the case put forward so compellingly by James Surowiecki in his book, The Wisdom of Crowds. But the problem is that, as our biases keep informing whom we hire and promote, we weed out that diversity and are left with skyscrapers full of people pretty much the same.
- But while it’s true that all of us now have access to more information than ever before in history, for the most part we don’t use it. Just like newspapers, we read the blogs that we agree with – but there we encounter a virtually infinite echo chamber, as 85 percent of blogs link to other blogs with the same political inclination.
- Our blindness grows out of the small, daily decisions that we make, which embed us more snugly inside our affirming thoughts and values. And what’s most frightening about this process is that as we see less and less, we feel more comfort and greater certainty. We think we see more – even as the landscape shrinks.
- Indeed, there seems to be some evidence not only that all love is based on illusion—but that love positively requires illusion in order to endure. When you love someone, he or she may even start to adapt to your illusion of him or her. So there is a kind of virtuous circle: you think better of your beloved who starts to live up to your illusions and so you love him or her more. It sounds a little like a fairy tale, but kissing frogs may make them act like princes or princesses. It is indeed a kind of magic, illusions transforming reality. We don’t have to love people for who they are but for who we think they are, or need them to be. This is something everyone does: overlook the flaws, discount the disappointments, focus on what works. Our love for each other allows us, even compels us, to see the best in each other.
- One of the many downsides of living in communities in which we are always surrounded by people like ourselves is that we experience very little conflict. That means we don’t develop the tools we need to manage conflict and we lack confidence in our ability to do so. We persuade ourselves that the absence of conflict is the same as happiness, but that trade-off leaves us strangely powerless.
- Because it takes less brain power to believe than to doubt, we are, when tired or distracted, gullible. Because we are all biased, and biases are quick and effortless, exhaustion makes us favor the information we know and are comfortable with. We’re too tired to do the heavier lifting of examining new or contradictory information, so we fall back on our biases, the opinions and the people we already trust.
- People stay silent at work—bury their heads in the sand—because they don’t want to provoke conflict by being, or being labeled, troublemakers. They may not like the status quo but, in their silence, they maintain it, believing (but also ensuring) the status quo can’t be shifted.
- Hierarchies, and the system of behaviors that they require, proliferate in nature and in man-made organizations. For humans, there is a clear evolutionary advantage in hierarchies: a disciplined group can achieve far more than a tumultuous and chaotic crowd. Within the group, acceptance of the differing roles and status of each member ensures internal harmony, while disobedience engenders conflict and friction. The disciplined, peaceful organization is better able to defend itself and advance its interests than is a confused, contentious group that agrees on nothing. The traditional argument in favor of hierarchies and obedience has been that of the social contract: It is worth sacrificing some degree of individuality in order to ensure the safety and privileges achieved only by a group. When the individual is working alone, conscience is brought into play. But when working within a hierarchy, authority replaces individual conscience. This is inevitable, because otherwise the hierarchy just doesn’t work: too many consciences and the advantage of being in a group disappears. Conscience, it seems, doesn’t scale.
- Human beings hate being left out. We conform because to do so seems to give our life meaning. This is so fundamental a part of our evolutionary makeup that it is strong enough to make us give the wrong answers to questions, as in Asch’s line experiments, and strong enough to make us disregard the moral lessons we’ve absorbed since childhood. The carrot of belonging and the stick of exclusion are powerful enough to blind us to the consequences of our actions.
- Independence, it seems, comes at a high cost.
- The larger the number of people who witness an emergency, the fewer who will intervene. The bystander effect demonstrates the tremendous tension between our social selves and our individual selves. Left on our own, we mostly do the right thing. But in a group, our moral selves and our social selves come into conflict, which is painful. Our fear of embarrassment is the tip of the iceberg that is the ancient fear of exclusion, and it turns out to be astonishingly potent. We are more likely to intervene when we are the sole witness; once there are other witnesses, we become anxious about doing the right thing (whatever that is), about being seen and being judged by the group.
- It is so human and so common for innovation to fail not through lack of ideas but through lack of courage. Business leaders always claim that innovation is what they want but they’re often paralyzed into inaction by hoping and assuming that someone else, somewhere, will take the risk.
- The greatest evil always requires large numbers of participants who contribute by their failure to intervene.
- Technology can maintain relationships but it won’t build them. Conference calls, with teams of executives huddled around speakerphones, fail to convey personality, mood, and nuance. You may start to develop rapport with the person who speaks most—or take an instant dislike to him or her. But you’ll never know why. Nor will you perceive the silent critic scowling a thousand miles away. Videoconferencing distracts all its participants who spend too much time worrying about their hair and whether they’re looking fat, uncomfortable at seeing themselves on screen. The nervous small talk about weather—it’s snowing there? It’s hot and sunny here—betrays anxiety about the vast differences that the technology attempts to mask. We delude ourselves that because so many words are exchanged—e-mail, notes, and reports—somehow a great deal of communication must have taken place. But that requires, in the first instance, that the words be read, that they be understood, and that the recipient know enough to read with discernment and empathy. Relationships—real, face-to-face relationships—change our behavior.
- The division of labor isn’t designed to keep corporations blind but that is often its effect. The people who manufacture cars aren’t the people who repair them or service them. That means they don’t see the problems inherent in their design unless a special effort is made to show it to them. Software engineers who write code aren’t the same as the ones who fix bugs, who also aren’t the customer-service representatives you call when the program crashes your machine. Companies are now organized—often for good reasons—in ways that can facilitate departments becoming structurally blind to one another.
- We want money for a very good reason: it makes us feel better. Money does motivate us and it does make us feel better. That’s why companies pay overtime and bonuses. It may not, in and of itself, make us absolutely happy—but, just like cigarettes and chocolate, our wants are not confined to what’s good for us. The pleasure of money is often short-lived, of course. Because there are always newer, bigger, flashier, sweeter products to consume, the things we buy with money never satisfy as fully as they promise. Psychologists call this the hedonic treadmill: the more we consume, the more we want. But we stay on the treadmill, hooked on the pleasures that, at least initially, make us feel so good.
- Motivation may work in ways similar to cognitive load. Just as there is a hard limit to how much we can focus on at one moment, perhaps we can be motivated by only one perspective at a time. When we care about people, we care less about money, and when we care about money, we care less about people. Our moral capacity may be limited in just the same way that our cognitive capacity is.
- Money exacerbates and often rewards all the other drivers of willful blindness: our preference for the familiar, our love for individuals and for big ideas, a love of busyness and our dislike of conflict and change, the human instinct to obey and conform, and our skill at displacing and diffusing responsibility. All these operate and collaborate with varying intensities at different moments in our life. The common denominator is that they all make us protect our sense of self-worth, reducing dissonance and conferring a sense of security, however illusory. In some ways, they all act like money: making us feel good at first, with consequences we don’t see. We wouldn’t be so blind if our blindness didn’t deliver the benefit of comfort and ease.
- Once you are in a leadership position, no one will ever give you the inner circle you need. You have to go out and find it.
- We make ourselves powerless when we choose not to know. But we give ourselves hope when we insist on looking. The very fact that willful blindness is willed, that it is a product of a rich mix of experience, knowledge, thinking, neurons, and neuroses, is what gives us the capacity to change it. We can learn to see better, not just because our brain changes but because we do. As all wisdom does, seeing starts with simple questions: What could I know, should I know, that I don’t know? Just what am I missing here?