Tell Me a (User) Story

Tell me what you want. What you really, really want — Spice Girls

If you’re involved in Agile then it’s pretty much guaranteed you’ve come across User Stories. It was a technique that started with one of the founding contributors to the Agile movement, Extreme Programming, but it has become pretty much ubiquitous with Agile itself. I’ve noticed however that for a technique that sits at the heart of modern Agile it often seems to be poorly understood and applied. For some reason a lot of people just ‘do’ User Stories without appearing to do any research or training, picking up the habits of the team they happen to be on, some good but mostly bad seemingly without much self reflection.

I started researching user stories to try and address my own ignorance and to better understand them because they felt important and I could see the problems they caused when not applied well. As I delved into them I began to respect the form even more, I liked how each of the rules and the constraints on behaviour fed back and supported each other and the succinct language forced the writer to reduce fluff and concentrate on what was important.

When properly applied they are the fundamental form of communication of a good Agile team, I like to think of them as a contract or API between the part of the business that wants something done and the part of the business that is doing it. A way of defining what the conditions for success are without being prescriptive of how they will be met.

But what is a User Story, and what makes a good one?

Extreme Programming is a programming process developed by Kent Beck and his team at Chrysler in the late ’90s and one of the precursors to what became Agile. The original definition of user stories centred on small discrete physical cards on a physical board that contained the outline of a piece of work to be completed. They share similarities with the cards on a Kanban board but instead of representing all work in progress they are described as ‘placeholders for a conversation’, a way of describe clear intent and enough specification but leaving room for broader discussion.

The physicalness of the card encouraged manipulation and discussion and aimed to make teams think of discrete atomic and independent units of work, by keeping the cards small it limited the amount of detail that could be added thus focusing whoever was writing the card to keep their description succinct.

The standard rules for a User story follow the formula of INVEST:

Independent of all others

A story is an atomic unit of work not reliant on any other piece of work to be finished before it can be started. Obviously there has to be some order, you can’t play an extra feature for a product you have yet to build, but at any point a piece of software has a number of possible options and of those possible choices any can be played without reliance on the other.

The Independent aspect links to the V of value (which we’ll get to in a minute). If each story represents a single piece of value it is easier to play independently, each story layers on the previous building an accumulation of value that grows over time but importantly each layer is still something useable and valuable to the user.

Where confusion sometimes reigns is how to see the independent pieces of value that can add up to a bigger picture.

A clear example of this is when my team had to deliver a complex filter search for the product we were working on. The vision was for a complex system of filters and searches, but to build the whole thing in one go would have been a huge undertaking so we discussed how we split the delivery up into independent units.

If you looked at this piece of functionality in it’s total it was hard to see how you could split any of the bits into individual pieces as they all seemed to reply on each other. After much discussion the first story was to deliver a simple free text search box so a user could filter the content by its title.

Now importantly a free text search was not part of the final vision of this filter system, it did not appear on any wireframes or design. It was though the simplest way of starting to tackle the actual problem for the user, which was quickly finding the content they were interested in. It was a small independent piece of value which, though it didn’t look like the final idea, was a definite step towards it.

From that seed the feature has grown and is now matching the complexity of the original designs but at each step along the way the user has had a working feature, one that could be considered ‘finished’ at the end of every story, with every subsequent addition coming from feedback on what was delivered before.

Negotiable

Negotiable and value are to me the two most important properties of a User Story. The shorthand definition of a User Story is ‘placeholder for a conversation’, I also like ‘open to discussion’. The worst stories are instead ‘placeholders for my solutions’. These type of stories are full of language like ‘add button to form’ or ‘create form that emails result’.

This is bad in several ways, when presented with these types of stories people quickly stop thinking for themselves and start delivering what is written on the card, whether its the best solution or not. Defining stories this way infantilises the developers on your team, makes them purely a delivery method for the story writers wishes.

To make a story negotiable you must focus on the problem not solution. This means stating clearly and in concise language what is the precise nature of the thing you are trying to solve without defining any implementation detail, no click this or drop-down that.

By focussing on the problem you invite discussion of it, as soon as aspects of a solution creep in then you make it incredibly hard to think past that solution to see other possible alternatives.

Not long ago I was in discussion with another Team Lead and a Product Owner, we were discussing an upcoming story regarding modifying some of our companies more legacy bits of software. The story was defined in such a way that it was calling out that this modification had to appear in a certain place in the legacy software UI.

Now even though I know the pitfalls I got caught up in this, I missed that the UI detail had crept in and very soon I found myself debating some really quite nasty options to make it work in that specific way. It was a day later when I suddenly realised what I had done, I went back and asked the Product Owner to make a tweak to the language and remove the concrete reference to that aspect of UI and instead specify what they actually wanted to achieve.

We went from the solution ‘I want to change this bit of the UI’ to the problem ‘I want to make sure this new type of content can be combined with this old type of content’. As soon as that change was made to the language all these other possibilities started presenting themselves in my mind. It was like the language was a prison and removing it released my thought process from its shackles.

This is just one of countless examples where I’ve seen something similar play out, this is why language is so important in User Stories. The wrong language can actively limit the development teams creativity and force an environment that fosters really poor choices.

Another important and often overlooked aspect of negotiation is the setting of boundaries, a good story is as much about discussing where to stop as much as defining what to achieve.

In my first proper job an inexperienced sales director actually created a contract with the sentence ‘and so on’ in it, this cost the company thousands as the customer essentially had a caveat for asking for feature after feature.

Poorly defined story boundaries cause exactly the same problem, without a clear stopping point then neither the Product Owner or the team really know when enough is enough, for example stories that are essentially placeholders for vague style changes cans snowball into tweak after tweak. This type of poorly defined boundary can quickly lead to feature creep or bloated meandering stories with no clearly understood outcomes, where neither Product Owner or dev team really know when to stop.

Valuable

Single piece value. That is what every story should be, but what does that really mean? It’s sometimes described as a discrete vertical slice but again it’s slightly ambiguous unless you understand what that implies. Essentially what both these statements are saying is that once a story is complete then it should deliver a discreate atomic body of work that either allows a hypothesis to be validated or solves a specific problem.

I say allows a hypothesis to be validated because until a story is released and it’s value realised then it is unknown whether the effort is actually going to achieve anything.

This is why a constant stream of smaller single value stories is more beneficial than a lesser number of bigger multiple value ones. Even if you’ve got market research to your eyeballs that this new search feature or that widget is going to be the bees knees to a user, it is pure fantasy until you have the proof of someone using it. It’s better to find out that the one piece of value you released yesterday is actually not as earth shattering as you thought it would be, than to discover that fifteen of the thirty five bits of value you released after 3 months of development weren’t actually going to be used.

This is also why horizontal slicing doesn’t count, this is where a feature is sliced often because of mistaken ideas of what the S of ‘small’ means, so you have the story for the database bit and the story for the frontend bit etc. It’s easy to look at your board and get a warm feeling from all those lovely story points and little stories, but they are worthless if you have to wait for all 5 to be released before you can actually see any value. This also links back to ‘independent of all others’ as these types of story anti-pattern are not atomic.

The other danger from the horizontal slice is knowing something is done. Once a vertical story is finished a Product Owner can sit down and see the finished feature, ideally they can use it without any dev team intervention and they can see it working end to end (if I add this here then I can see the result there). With a horizontal slice that is never the case, if you’ve ‘finished’ the database schema story and then a week later the person working on the application goes ‘hang on a minute I need that column changed’ it’s not actually finished is it?

Sometimes the value doesn’t mean just doing the same thing but simpler, a lot of times it’s trying to capture the minimum need that requires satisfying and delivering that first. Like the example earlier with the free text search, the User Story is the method for distilling the ‘simplest thing that could possibly work’ into a set of understandable requirements, it is not delivering a set of jigsaw pieces that you have to wait for all to complete before the full picture can be realised.

Estimable

Estimation is a whole other can of worms that needs an entire article to do it justice but I will attempt to give an overview. Estimation is a very loaded term and it can mean wildly different things to different people. Typically Agile estimation has tried to move the conversation from questions around concrete ‘tell me exactly how long this will take’ to the more relative ‘how big is this in relation to other things of that feel like a similar size’.

The favoured method of estimation can also vary between team and organisation, the Fibonacci sequence is the most popular but there’s also ‘ideal days’, the Brodzinki’s estimation scale (one, two f**cking big and no f**cking clue) and my favourite the estimate goat.

When I think of estimable I think of understandable, is the amount of effort and complexity understood and known? What I think you are doing when you estimate is forcing the team to commit and express if they fully understand what this story entails.

On my team we use the Fibonacci sequence but we only use the numbers 1, 2, 3, 5 and 8. We’ll rarely ever use 1 unless it is literally a text change or a colour tweak, 2 and 3 are indicators that the scope of a story is well understood and all factors and complexity are clear. A 5 is something where there is some uncertainty creeping in, we’re reasonably confident but there are some unknown factors that could expand the work. When we get to an 8 we’re essentially saying all bets are off we’re really not sure how long this will take but it’s going to be a while.

The idea here is that we’re aiming for a 2 or 3, things that have a clear scope and are well understood. if we’re suddenly seeing things scored with a 5 or 8 it’s an indicator that things are potentially getting too big, we need to review the stories and check there aren’t multiple bits of value hiding or initial spikes to better understand the unknowns.

If we decide that we still can’t split it smaller then we go in with our eyes open and understand this is a bigger piece of work with a level of uncertainty, if we start seeing lots of stories this size it’s an indicator that our refinement or story process is off and we need to correct. We never go above an 8 because from our experience anything above that number and the estimation essentially becomes meaningless.

Small

Classically this is defined as ‘can fit in an iteration’, the problem with that statement is people will often sacrifice the V of value to make it true. I’ve challenged people on stories before where they are horizontally slicing work and the come back is ‘well if we don’t do that it won’t fit in an iteration’.

Small really means ‘smallest possible piece of value’ and I prefer to concentrate on that and not to be too precious about the 1 iteration rule. If you focus on the honing of the value part of a story then most of the time the stories naturally stay small and stay in an iteration, if you do get the occasional one that stretches over more than one sprint it’s normally for good reason.

However if you find that stories are constantly ballooning and running over sprint after sprint, then the smell here is that they either they contain multiple pieces of value or your boundaries are not clear enough in the acceptance criteria and not that they need to be artificially chopped up to make them smaller.

Remember ‘smallest possible piece of value’ does not always mean cutting a feature into smaller chunks, it could mean reimagining a feature into a simpler version of itself (as with the text search example earlier) or finding other angles to achieve the same results.

Testable

A story should be testable is several ways:

• Automated tests, good acceptance criteria should be directly translatable into a series of automated tests that will verify if the code is exhibiting the behaviour the story requires. The tests don’t have to map to the acceptance criteria word for word (though that can sometimes be the case) rather a developer can point to specific suites of tests that verify each part.
• The completed story should be verifiable. If you focus on the end to end value then a completed story should be a completed feature or piece of functionality, this means a product owner or subject matter expert can sit down and see that piece of value end to end and verify it independently against the acceptance criteria without the intervention of a developer. There will be cases where this is sometimes not possible but that should be the exception not the norm.
• Finally the hypothesis the story sets out to prove should also be verifiable. This is where things like logging and reporting are invaluable, did the feature we just released achieve the aims we set out in the story? Is anyone actually using it? How much? And by whom? All of this allows you to verify if the expected value is realised and if it is not where to go next.

All of the above can be summed up in one word, feedback. Automated tests give you the feedback that you are meeting the expectations of the agreed behaviour. A verifiable story allows feedback that the feature was what was expected. Validating the hypothesis allows feedback from the end user which can be fed back into the next cycle of stories.

Conclusion

INVEST is the DNA of a User Story, not the specific language or structure but a checklist of essential underlying qualities.

Each of these qualities reinforces the others, a well discussed story that focuses on a single bit of value is normally small, independent and verifiable by its nature and producing a backlog of stories with these qualities tends to reduce the variance between them which makes them more estimable.

Breaking these rules or overemphasising one at the expense of others can lead to lot of common story issues. Multiple pieces of value normally equals confusion and story bloat, not negotiable and the story becomes a task to be given as opposed to a discussion to be had. Focus on small at the expense of other qualities and you will lose the value and independence as your stories become a web of dependence. If you find yourself plagued with these issues then the INVEST formula gives you a handy checklist to measure against and correct.

INVEST gives us the framework, but what about the stories themselves? What is the formula for a well written story and what are the pitfalls to avoid? Part 2 of this series will seek to address these questions as we look at the art of writing a good User Story.