An Experiment on Executable Specifications

What: Create executable specifications for an ongoing project sprint. Executable specifications are written examples of a business need that can be run anytime and acts as a source of truth for how applications behave. It is a living documentation of what our software does, and it helps us focus more on solving the unusual and noteworthy problems instead of wasting time with the ones that we know we shouldn’t have been worrying about.


  1. Join a software development team for one sprint duration.
  2. Discuss project tasks for that sprint with everyone.
  3. Ask about requirements examples and create executable specifications for those tasks as the application is being written.
  4. Refine the specifications when necessary.
  5. Continuously update the specifications until they pass.
  6. Keep the specifications running on a preferred schedule.

Why: To see if writing executable specifications alongside development is feasible during the duration of a sprint.

Limitations: Tester writes the executable specifications, programmers work as is.

Experiment Realizations:

  • Writing executable specifications can be done during actual app development, provided that tester is experienced with the tools for implementing it and understands well why the need for such specifications.
  • It is of course more beneficial if everyone in the team learn how executable specifications work and write/run the specifications, and why the need to implement it.
  • It will take quite a long while before executable specifications becomes a norm in the existing software development process, if ever. This is a function of whether everyone believes that such specifications are actually useful in practice, and then building the skills and habit of including these specifications in the team’s definition of done.

A Daily Standup Reminder

The programmers are the usual suspects in a daily stand-up ritual. Each of them take turns talking about what they did yesterday, what they will be taking on today, and what problems or successes they’ve encountered. It is a must that their voices are heard clearly by the scrum master and the product owner, who are there to monitor the project’s progress, and their co-developers, who most need to be in sync with code changes.

I hope the team scrum master and tester are sharing the things they’ve been busy doing too; as far as I know, they work towards the same goal as the people who write the code. Maybe tasks are not yet ready to be tested, maybe the team is cruising well on the development iteration, still their thoughts and opinions are valued – about ongoing tasks, about current processes, about possible ways of moving the team forward as awesome as possible. Share tests scripts and important use cases, honestly tell the group where they’re at, ask clarifying questions whenever needed, show test results, suggest process changes that could deliver more fun and creativity, say thanks to everyone who’s been working hard.

Fine-Tuning The Software Development Process

Lately I’ve been thinking about the software development process we have at work, the stuff that has been keeping us in shape for the past few years. It’s a system of smaller systems, which in turn makes up a flow – requirements gathering, sprint planning, sprint iteration and monitoring, demo and retrospectives, testing, project release. Good flow helps deliver quality service, bad flow makes it difficult to produce value.

I actually think we’re actually doing good so far on the flow. 🙂

Still, there are some areas in our current process where I believe we could be many times better in the near future, such as tasks estimation, business rules presentation and documentation, sprint data analysis, testing (especially unit testing), and continuous integration. It’ll take a while, of course. The challenge is for everyone in the team to keep seeking the possible small improvements we can easily implement, baby steps that makes our good flow better than ever. Delivering quality has always been a team effort, not a task reserved to testers (nor to any one person) alone.

About Sprint Data Gathering

Looking at a sprint document I designed a few years back (and updated by my scrum master colleagues), what I see are all sorts of data: planned total story points, actual finished story points, planned total task hours, actual finished task hours (total, per developer, team per day), burndown chart, attendance records, retrospective notes, tickets notes, returned tasks, unplanned tasks, underestimated tasks. So many information being gathered and compiled every sprint iteration; it’s not really too difficult to document but it does kind of takes away a good chunk of time to manage. Back then when we were new to scrum I thought that crunching numbers will provide insights as to how we can improve estimates and productivity, I thought that maybe we’d become faster and better at what we do after performing some mathematics.

It turns out that it doesn’t work that way. It turns out that estimates and productive behavior is a lot more complicated than it seems. It turns out that personalities and preferences and work environment and team dynamics and other small stuff we don’t pay attention much to play big roles in being good at software development as a team. Data about these things are not shown in the sprint document, and I don’t have the slightest idea how to measure them. So, looking back at all these stuff we’re trying to analyze, I’m haunted by questions: Which data are important enough to keep taking notes of, sprint after sprint, to keep reviewing? Why are they important? How do these data help me define how my team improves in succeeding iterations? Are they just pointing out problems but no solutions in sight? What if I stop documenting these data, would that help me focus my energy towards other possibly more important to-dos?

The Not Too Obvious Goal

For every new software project or system feature created during the scrum development process, product managers, developers, and software testers all toil to make things work as they should and could be. The goal to satisfy the client’s needs, the need to meet the requirements before the deadline, motivates everyone to produce remarkable work. This is great.

In between great works of software development, meanwhile, there’s another goal that’s not too obvious to everybody. After every sprint, in retrospective meetings, people share whatever it is that went well and issues that they thought delayed their progress. Possible strategies to improve processes are discussed. The aim is to be better at what they do in the long-term, to setup systems that pushes for growth after each project, to become eventual masters, not to settle for anything less excellent.

A Survey On Scrum

My boss recently decided to survey the team about the team’s outlook on the scrum process, asking for comments and/or suggestions. Here’s what I was able to say:

The good thing about having the scrum in the current development process is that it forces everyone in the team to plan and really think about their goals for the next few weeks. This is definitely better than just reacting to desired software modifications from clients all at the same time. What it gives is the time to work on the most important issues first, with focus.

As such, the scrum process, the sprints, will really work well for everyone if the teams (ideally) are able to work on their respective projects with focus, with no or very minimal unplanned side requests. Estimations are easier to assess in this case as well; it is very hard to properly estimate the amount of work a team can finish if they are continuously disrupted by issues outside of the current project.

Other things:
1. Most, if not all, dependencies should be finished before the sprint starts.
2. No overtime work.
3. Add a day or two in the sprint duration, for regression testing and bug fixing. OR reduce the amount of estimated work to be completed in each sprint in order to fill in the tests.
4. Consider enough additional time in developer estimation for proper code reviews.

The Sprint

  • lets the development team discuss with the product owners (or end users) the most pressing and important requirements that need to be delivered after a desired amount of time,
  • allows the development team to share their thoughts about the work at hand – how much they think they can actually finish and what they need to accomplish the feat,
  • forces the product owners to trust the programmers to do the work,
  • gives the development team the responsibility to follow through on what they promised,
  • provides the development team ample time to review their work, to experiment, to redo things in better ways if possible, to be challenged enough, to learn more,
  • supplies the software tester enough time to check if the pieces of the puzzle are working on their own before fitting them together,
  • keeps log of all things that happen as the software development process goes on, and
  • helps everyone get to know their strengths and weaknesses, which eventually shows the team the road to being better.