I’m currently in the midst of a test code overhaul, a re-writing project of sorts. It started about a week ago and so far I’ve made considerable progress on what I’ve wanted to achieve with the rewrite, which is basically cleaner and more maintainable code, mostly in the sense of test data management and test description language. The number of tests running everyday in our Jenkins system has grown noticeably and I’ve felt that it’s been difficult to add certain tests because of how I structured the test data in the past, which I have not upgraded since then. The two possible avenues for running tests – on the UI and HTTP layers – also adds a bit of complexity and it’d be nice if I can integrate the two smoothly. It’s an interesting development because I did not plan on any re-writing to be done anytime soon but I guess at the back of my mind I knew it’ll happen eventually. And so I decided to take a step back from writing more tests and do some cleanup before it gets tougher to change things. I plan to finish everything in about a month or so.
At the moment, I’m reminded of the phases I’ve gone through in learning to code and writing automated checks in the past few years:
- Early 2014. It all begins with Selenium IDE, with giving the self some time to study the basic Selenese commands for writing automated checks and (more importantly) understand how to properly retrieve the page elements you want to manipulate.
- Mid 2014. Test management in Selenium IDE becomes difficult as the number of tests grow, hence the decision to switch to Selenium WebDriver. The only programming language background I had back then was C++, which was limited to only functions and logical/conditional operators, so I chose Java to work with to lessen the learning curve.
- Late 2014. Familiarized myself with Git, which hooked me on making daily commits and appreciating version control. Along the way I learned the concepts of classes and objects.
- All of 2015 up to Early 2016. I was in a trance, writing code daily and pushing myself to create all the automated checks that I wanted to run for our apps before every release. Tests run on the Eclipse IDE using TestNG and I was happy with what I had, except that those end-to-end tests are really slow. Running everything took overnight to finish, which was okay for my employer but annoying for me personally.
- Mid 2016. Re-writing existing tests in Ruby with Cucumber integration started off (when I found Jeff Morgan’s “Cucumber & Cheese” book online) as a side project for fun and testing my skill level in programming. And I did have buckets of fun! The experiment told me that there’s still a lot I need to practice on if I want to write better code, and it also told me that I can be more productive if I switch programming languages. There’s a bit less code to type in when writing code in Ruby than Java and I liked that, plus all the interesting libraries I can use. I switched to Sublime Text and used both Jenkins and the command-line interface more extensively too.
- Late 2016. As I was looking for ways to speed up end-to-end tests total execution, which by then takes about 4 hours to complete, I ended up exploring testing apps in the HTTP layer instead of in the UI. That took a lot of studying of how our apps actually behave under the hood, what data are being passed around, how images are actually sent, how to view pages without a browser, how redirections work, among other things. After years of testing apps via the user interface, this was such a refreshing and valuable period, and I completely wondered why I never knew such a thing existed until then. It wasn’t being taught extensively to testers, perhaps because it all depends on how the app was structured to run through an API.
And these phases brings me to now, where there’s a healthy dose of API and UI layer tests all checking major app features. It’s all good, just several pieces needing a cleanup, a little parallelization, better test description language, and great documentation. It’s all good, because the lessons in both programming and testing keep piling. The two practices differ in mindset but I think they complement each other, and I think that there’s no reason anyone can’t do both.
Many of the things I’ve learned about software testing in recent years I discovered through reading books and blogs of actual software testers who are masters of their own craft. Some of them record videos too, of tutorials and webinars and anything they find valuable for their readers/viewers, and I try to share those which I’ve found engaging, five at a time.
Here are some interesting recordings to view today, if you’d like:
What is a test description language? How do we use that language to effectively write tests? In a series of blog posts, Jeff Nyman talks about what TDL is and how to write better tests with them. The posts are mostly written in the sense of how Cucumber checks are being used for writing tests for automation, but the lessons can also be applied in writing test cases if ever there is a need to write them for clients.
Some notes from the series of posts:
- It’s very easy to confuse the essence of what you’re doing with the tools that you use.
- Just because I’m using TestLink, Quality Center, QTP, Cucumber, Selenium, and so forth: that’s not testing. Those are tools that are letting me express, manage, and execute some of my tests. The “testing” part is the thinking I did ahead of time in discerning what tests to think about, what tests to write about, and so forth.
- Testable means that it specifically calls out what needs to be tested.
- When examples are the prime drivers, you are using the technique of specification by example and thus example-driven testing. When those examples cover business workflows, you are using the technique of scenario-based testing.
- Requirements, tests, examples — they all talk about the same thing. They talk about how a system will behave once it’s in the hands of users.
- The goal of a TDL (test description language) is to put a little structure and a little rigor around effective test writing, wherein “effective test writing” means tests that communicate intent (which correspond to your scenario title) and describe behavior (the steps of your scenario)
- It is now, and always has been, imperative that we can express what we test in English (or whatever your common language is). This is a key skill of testers and this is a necessity regardless of the test tool solution that you use to manage and/or automate your testing.
- So many tests, and thus so much development, falters due to a failure to do this simple thing: make sure the intent is clear and expressive for a very defined unit of functionality.
- Software development teams are not used to the following ideas: The idea of testing as a design activity. The idea of test specifications that are drivers to development. The idea of effective unit tests that catch bugs at one of the most responsible times: when the code is being written.
.. here’s a list of online courses I took last year in order to learn how to use Selenium Webdriver (an open source tool that automates browsers):
It’s not a comprehensive list but they are enough to get you started. The first two in the list are free courses, the last two are paid (I always wait for a $10 sale before purchasing the more expensive courses since uDemy does that a lot anyway), all of them will of course require you to have a uDemy account to access them. I learned a bunch from these tutorials, even though it can sometimes be difficult to listen to Indian instructors, even though they won’t have all the answers that you need. A lot of the more important lessons in implementing successful automated test suites will come from the experience of using the tool, from debugging test code, from looking for answers on the main Selenium documentation or StackOverflow, and from following blogs of software testers who have used Selenium far longer than we have. But do check them out, especially the free ones, if you are into web testing and you are curious about how Webdriver works. 🙂
I didn’t know there was such a thing as a yearly Selenium conference, after all this time implementing a Selenium test automation project at work, until I came across a recent blog post by the Mozilla Web QA Team, enumerating their thoughts about the event. I wished I was there (and I wish there were events like this here in my country), especially when I saw some of the keynote presentations and talks they were able to film. I must say that they were very interesting. 🙂
Here are links to these videos, re-posted from the Mozilla Web QA blog:
- Selenium: State of the Union. Fun and exciting keynote talk by Simon Stewart of Facebook, about where Selenium has been, where it is now, and where it is moving forward.
- Mobile End-to-End Testing at Scale: Stable, Useful, Easy – Pick Three –. Inspiring talk from Remi Chaintron of Facebook. Yes, end-to-end automated UI testing is hard, but it does work.
- Docker Selenium. I love the way Marek Jastrzebski of HomeAway talks, how it feels very accommodating. Anyway, Docker seems to be very useful. I’ll try to figure out how this works in Windows.
- Distributed Automation using Selenium Grid/AWS/Auto-scaling. I was bored by Ragavan Ambighanantha’s talk (of Expedia), but I wanted to know the results of his experiments with Selenium Grid parallelization combined with Amazon Web Services cloud-based testing.
I started studying test automation using Selenium IDE, which is for the most part is a record-and-play tool. It was pretty easy to use, and all I needed to learn well is how to properly retrieve element locators. My needs changed since then – better test management, test reuse, better reporting, test scheduling – and so I had to learn many other things:
- programming language: Java & Selenium Webdriver
- development environment: Eclipse
- testing and reporting framework: TestNG
- test scheduling: Hudson/Jenkins & Ant
- test distribution: Selenium Grid & VirtualBox
- code repository: GitHub
It certainly has been a lot of work but it’s been fun. And there’s a lot more to learn I haven’t even started studying mobile. 🙂
A year and a half ago, I knew nothing about software test automation. All I had was a problem: I had been increasingly getting flustered over checking all necessary test scenarios for project releases. It seemed that our software systems were getting big. For each project iteration, new tests were being added for regression, some more complicated than others, and testing time was short and finite, and I did not have enough means to process everything to make deadlines. It was starting to get chaotic because I felt I had no way of looking over each test to make sure things are working fine, and that was bad. Then I came across Selenium, and I thought that maybe this could solve my problem, and started using it (with failures and successes), and the rest is history. I got the solution I wanted after a lot of tinkering, and I am glad I pushed myself to learn.