After reading David Evans and Godjko Adzic’s awesome “50 Quick Ideas To Improve Your User Stories” e-book last March, I had high hopes for their book about testing, “50 Quick Ideas To Improve Your Tests” when I purchased it. Their user stories book was awesome so I was thinking that maybe they had a lot of interesting ideas about better testing too. Long story short, it’s a great read too. Here are a few valuable lessons:
- A test that executes multiple actions is fragile and is usually very difficult to maintain. This is true for automated tests, as far as my experience takes me, and is a hard lesson to learn.
- Difficult testing is a symptom, so the software development team needs to be alarmed when this happens. If a team finds it difficult to know the quality of the product during testing, then it will also be difficult to know if the product works well during development or even during discussion on requirements. Godjko Adzic and his team says that this is a modelling problem.
- Manual tests are most valuable when testers do not follow test scripts step-by-step. Automated tests, on the other hand, work well when they are very specific.
- When a software system grows, the number of tests that need to run to evaluate its quality also grow accordingly and at some point, the team needs to understand that running all these tests will eat a huge chunk of time. This means that sometimes the team will have to trade speed of feedback versus comprehensiveness of test results. Someone will also have to decide which set of tests run immediately after commits, which executes overnight, and which only need to run occasionally.
- Testing is never finished. It only stops sometimes.
- It is important to measure the half-life of the stability of tests, to measure the amount of change all tests undergo. If changes frequently happen to our tests, it might mean a problem in test design or it might mean that the system is fragile.
- When designing specifications, the book suggests giving the Given-When-Then pattern a try.