What is this project all about?
Can I set up the project locally? How? What are the dependencies?
How do I know if I installed the project properly on my machine?
How do I contribute? What is the existing team process?
Did I break anything after making changes to the code? Are there any tests I can run to make sure that the project is still stable? What sort of tests do we have?
In an ongoing software development project we are using Makefile tasks to make running long and repetitive commands as easy and as fun as possible to run. They’re like bash aliases, shortcuts to performing recurrent jobs we frequently have to do while writing new code or testing applications. For example, we could define a task that runs unit tests and code standard checks on an application running in a Docker container like so:
echo "Checking application code with PSR2 standards ..."
docker-compose exec -T php phpcs -v --standard=phpcs.xml ./app/src
echo "Running unit tests ..."
docker-compose exec -T php phpunit --colors=always --configuration ./app
and we would run the task with only the following command:
Cool, right? I don’t have to remember all the exact commands to do what I need to do. And even if I forget the right task name (in this case,
make test) I can just run the
make command in the CLI and I’ll be provided a list of the tasks that I can use for the project.
Now Makefile tasks will run on Unix terminals out of the box. For Windows however, we still have to do some setup before Makefile tasks can run. For my machine at work, I did the following:
- Download and install GnuWin32
- Go to the install folder
C:\Program Files (x86)\GnuWin32\bin
- Copy all files inside the bin folder to the root project directory (
- Add the installation bin directory to the system environment variables
There are other tools that we can use to configure Makefile to run on Windows but this is a quick and easy way to do it. After that we can run
make.exe test on the default cmd CLI but on some Unix-like terminals like the Docker Quickstart Terminal we can definitely use
Problem: I want to automatically run unit tests, lint the application code, and check it’s state against team standards every time I try to commit my changes to a project. It would be nice if the commit aborts if any of the existing tests fails or if I did not follow a particular standard that the team agrees to uphold. The commit pushes through if there are no errors. If possible, I don’t have to change anything in my software development workflow.
Solution: Use a Git pre-commit hook. Under the .git/hooks hidden folder in the project directory, create a new file called pre-commit (without any file extension) containing something like the following bash script (for testing PHP code):
stagedFiles=$(git diff-index --cached HEAD | grep ".php" | grep "^:" | sed 's:.*[DAM][ \\''t]*\([^ \\''t]*\):\1:g');
errorMessage="Please correct the errors above. Commit aborted."
printf "Linting and checking code standards ..."
for file in $stagedFiles
php -l $file
if [[ $LINTVAL != 0 ]]
php core/phpcs.phar --colors --standard=phpcs.xml $file
if [[ $STANDVAL != 0 ]]
printf "Running unit tests ..."
core/vendor/bin/phpunit --colors="always" [TESTS_DIRECTORY]
if [[ $TESTSVAL != 0 ]]
- linting and code standard checks only runs for the files you want to commit changes to
- code standard checks are based on a certain phpcs.xml file
- unit tests inside a particular TESTS_DIRECTORY will run
- the commit will abort whenever any of the lints, code standard checks, or unit tests fails
There are various levels of testing in software. The one most people are familiar with (including software testers) is testing done through the user interface, which is basically using the application at the end of a software development cycle and finding out whether it does what it is supposed to do. It’s a practice that is easy to understand and natural to do. Most of us have mobile devices or computers at home and in that sense we all understand how to test apps on the UI at some basic level. We explore the functionalities apps say they deliver and we decide for ourselves whether we think those promises are being kept or not. We feel good when everything works well or we feel bad when it is difficult to use the app (and maybe never use that app again). That said, software testing is not limited to the user interface.
The more experienced testers understand that testing is easier to perform and more valuable when it is done early in the software development process, making sure that we are doing the right things and are doing things right, even though we know we can’t test everything all at once. Bugs found before shipping are cheaper and easier to solve than bugs found later. Quick and early feedback is ideal. But to accomplish testing early in the software development process means that testers actually need to understand how software is built from code, not just the code itself and how various pieces of code integrate with one another but how programmers write code too. Like everyone else, programmers are people and human and are fallible. People make mistakes, and people can continue to make mistakes even if they work on projects carefully, because that’s how people and the things they build grow. That’s why testing needs to happen as early as possible. That means testers working alongside programmers in putting systems in place that tests the application simultaneously while it is still being written, even when there is no user interface to see yet. That means recognizing where and when mistakes happen, whether in code or habits or processes, and making it easy to spot them when they happen again. Testing in the user interface will never disappear but we can do better than restricting ourselves to just testing at the end.
For any software tester or any software professional it matters to be curious about the industry we choose to be a part in, to be knowledgeable about the people we work with and the tools we use to perform our best work. It helps to be aware of existing practices and be in the loop with the news, well, because that’s how we find solutions to problems, and sometimes more problems to solve. We use the software we test. We find answers on the web. We try applications that could maybe make us be more productive. We network with people like us on the internet, and we go online, study, and digest whatever we can. That’s part of how we improve our skills. That’s part of how we grow, and help others along the way.
But learning takes time and effort and energy. It’s not just about reading and watching everything, taking every online course and going to every conference there is. Our minds does not work like that. We have to pace ourselves well. We need to take breaks in between, we need time for details to sink in, we need contemplation and scrutiny to guide us where to move next and why exactly. Some reflection happens in discussions with colleagues, family, and friends, while other realizations occur only when we are truly alone with ourselves.
So.. after about a week or so since I asked permission for read-write access to our application code repository I’m glad to say that I’m almost done with setting up a version of our apps locally on my machine. It is necessary because I first need to check my changes if they work locally before committing those changes. No code commits yet until said local apps have the same stability as our apps in Staging.
But there are no unit tests. How would I know if everything works after cloning the app repository and running the local settings? Only one option: I had to run local versions of my Staging application API tests. They’re slower than unit tests but at least they let me know if the apps work on some good enough level.
Running tests on local applications!
We’re in business! 🙂
Most of the problems that I encountered whilst setting up were database problems. That’s because no one was maintaining a small-but-updated version of the database. As told, I had to manually match which queries to run according to what problems my tests found. Not pretty, though I could say that in retrospect looking at the errors and finding the DB fix on my own was good exercise. Not elegant, but it helped me get familiarized a little bit with our applications as code.
There was also no documentation about the application and how to run them on various machines. Guides are important but README files were mostly left blank. I had to rely on programmer friends for clues about what to do next whenever I got stuck.
Such problems took time and patience to solve. I had to take notes about updating certain pieces too. Sometimes, I had to make changes to the code itself in order for some features to not fail locally. And yes, I need to remember not to accidentally commit those changes to the remote repository.
It would be nice if we can just go to some private repository and download an environment image or two which runs smoothly when integrated with the app repository. Update the code on a local machine and the environment updates automatically. Set up would have been done in a matter of minutes, not days. But, alas, that’s a problem worth solving for another day.
The realm of security testing is something I have not explored yet in deep detail not because it’s not an interesting field but because I have always found it to be intimidating, stuffed with jargons and specialized tools to learn. But the curiosity is there, and I’ve decided late last year that I want to get better at it. For that reason I’m glad that Gergely Revay has opened an online course on becoming a web pentester this year. Great timing! And very practical too because I was able to directly apply what I learned on the course at work. 🙂
As with any skill, we master it through practice. But here are some notes about the key ideas I learned from the course:
- Security testing requires exploratory testing. A tester can only find out where the security vulnerabilities are when such person has good understanding of what risks are present in the application, and one can only know about what the risks are when one has vastly explored application behavior in various scenarios as well as the technology stack where it runs.
- We can download or view application data (and more) through a system’s insecure file upload feature. Secret configuration files may not be as safe as we think they are.
- Kali Linux provides us common word lists that we can use to brute-force attack logins. An account is only as safe as the complexity of its matching password.
- Getting legitimate users to run a malicious script for an attacker relies on how good the attacker is in manipulating the target person to visit some desired page.
- It is possible to run operating system or database commands on the server where an application is running.
- Even if an SQL injection does not provide us details of the query results, as long as the injection works we may still get interesting data from the app through succeeding creative attacks.
- Applications, as innocent as they may seem, can help an attacker find vulnerabilities through the user experience. Be careful about the hints you provide to users when they fail to authenticate their account, among other possible
- Because security testing relies so much on a tester’s knowledge of the app under test, security testing is difficult. The deeper the tester know about which features are available and how they work, both in the user interface level and in the background, the better the chances of the tester finding security vulnerabilities.