Backlog grooming in practice

Over the last few years while working in numerous scrum teams, I have come to realize one thing that all of us seem to struggle with - completing user stories within sprint boundaries. The most common reason is the long list of tasks we have at hand. Tasks that need to be completed so that a user story can be accepted. Delay in any one of the tasks usually leads to the story spilling over to the next sprint.

After a detailed introspection of the scrum, I often find that the only way to avoid delays is to diminish the unknown factors in a sprint and have a better plan. In other words carry out effective backlog grooming. Unknown factors can range from UX layout issues to crystalizing requirements for setting up a test bed.

Let us see a sample of the problem at hand through a typical 'tale of a sprint'.

Figure 1: Typical tale of a Scrum

Here a sprint with seemingly easy user stories became a struggle in the end. This was attributed to the unknowns discovered during the sprint.

Effective backlog grooming

One of the techniques that we evolved and mastered in our agile projects is something I like to call the "Building Man" technique. This technique has proved to be a major differentiator in scrums improving their velocities and reducing rejection rates during story acceptance.

The Building Man technique breaks down backlog grooming into three steps and takes a systematic approach to the problem of discovering the unknown factors.

Figure 2: "The Building Man" technique

Step 1: Strawman Grooming (20% Confidence)

• What ?

Strawman grooming provides the Product Owner (PO) and other executive stakeholders just enough details to size the epics with respect to each other. It is crucial in carrying out a cost benefit analysis of various epics.

• When ?

Whenever a new item is added to the product backlog.

• How ?

Strawman usually begins with a discussion between the PO and the architect. For example, as a shopper on XYZ e-commerce website, I should be able to login using my Google credentials, and I should not be prompted to create a separate user-ID/password for purchases.

Acceptance Criteria

• Application should work on all browsers including mobile platforms
• No more than one click should be required to login into the application if a user is using Google credentials.

In this step, the description of what needs to be done is laid out at a high level and no low level technical details are mentioned. In order to groom the epic, there must be a conversation between the Architects and the Product Owner (PO). Architects will ideally have scores of queries, response to which will refine and define the epic further.

Your user story may look like this after you get inputs from the architect:

• 30% of users are expected to use this form for authentication
• Mobile browser will be for Android 2.2+
• OAuth will be the authentication scheme

Decision to use OAuth is an input from the Architect and should not impact user behavior in any way. Thus, this can be captured in a section called Scrum notes in a user story. Based on the newly gathered information, architects begin sizing the epic. All the epics in the product backlog ideally must be at least sized at 20% confidence level.

Epic	Points	Estimate Type
Epic - 1	30	Strawman
Epic - 2	180	Strawman
Epic - 3	10	Strawman
Epic - 4	40	Strawman
Epic - 5	70	Strawman
Epic - 6	80	Strawman
Epic - 7	130	Strawman
Epic - 8	90	Strawman
Epic - 9	120	Strawman
Epic - 10	10	Strawman
Epic - 11	90	Strawman
Epic - 12	80	Strawman
Epic - 13	50	Strawman
Epic - 14	20	Strawman

Prioritized Product Backlog

Epic - 1	30
Epic - 2	180
Epic - 3	10
Epic - 4	40

Prioritized Release Backlog (~300 points)

Figure 3: A typical sizing output

Top prioritized items from Strawman can be directly pulled into the release depending on the capacity available for the release. For example, post Strawman grooming, the PO will have the data to prove that not doing Epic-2 can mean doing at least 2 more epics in the next release. This is how we can make informed decisions. But remember Strawman estimates have 20% probability of being accurate, which is very low.

• What we know now ?
• Top level use cases for an epic
• High level technical components

• Relative sizing of one epic in relation to others at 20% confidence level

Step 2: Woodman Grooming (50% Confidence)

• What ?

Woodman grooming helps divide the bigger epics into vertical slices with the technical experts in the team. It ensures scrum teams are focused on one feature for grooming and execution.

• When ?

Before initiating a release or before any commitment is made to the customer.

• How ?

As the name suggests, Woodman is more rigid and precise. It is carried out for all stories which are to be executed in a sprint. The result is vertical slicing of the stories which can then be used by POs to prioritize stories within epics.

Vertical slicing cannot be done at this level by POs in isolation, they need technical inputs from time to time.

We need not involve every member of the scrum team at this point in grooming. Typical participants in Woodman grooming include Architects, SMEs (Subject Matter Experts) and the PO. Most of the work to reach this level is done offline. POs and Architects usually try to find answers to following questions:

• Which technologies will be used to implement this feature?
• Do we need to use third party components to complete the epic?
• What would a typical user experience be like?
• Is this epic dependent on any other epic?
• What are the vertical slices at this level and are they dependent on each other?

Once these questions are answered, epics can be sliced as follows.

Another round of estimation can be done with the estimates here marked at 50% confidence. Now, these sub epics are more or less independent of each other and we should have a fair idea of workflows as each of the sub epics shall have a UX element as well. These sub-epics should not be confused with leaf user stories.

What we know now ?

• UX flows
• Technology involved
• Modules expected to change
• Scale testing methodology
• Minimum viable option to ship the feature
• Dependencies between sub-epics (if any)

In case, any of the queries cannot be answered without carrying out prototyping, then we can go ahead with prototyping and call it a mock-up during this step. However, like feature stories, mock-up stories should also have a defined acceptance criteria.

Step 3: Steelman Grooming (80% Confidence)

Steelman grooming has the most details and helps scrum teams pick up stories directly from the boards for execution.

When ?
Ideally it should be carried out before sprint planning but maximum 2-3 sprints prior to when stories are planned to be executed.

How ?

The last stage of grooming, Steelman is where scrums take sub-epics and break them down to user stories with the help of POs and Architects. Each scrum is assigned one sub-epic at a time. These independent sub-epics are picked up by scrums and sized to the next level through scrum sizing techniques like planning poker, relative sizing etc.

Following are the typical items that should be answered to complete Steelman grooming and for scrum sizing to begin.

• How will UX function?
• What data model and business rules are to be used?
• What all will be automated?
• How will scale testing be done?
• How will a user story be demoed?
• Are there any extra tools/hardware needed to complete the user story?

After these questions are answered by POs, Architects or Scrums, we move on to scrum sizing. In case, the size of the user story is more than what is executable in one sprint, it is broken down further.

For example,following are possible options to breaksub epic -1 (Allow user to login with Google credentials)

• What we know now ?
  • - A clear and defined acceptance criteria
  • - Required UX changes
  • - DB schema changes (if any)
  • - Business rules required
  • - Scope of automation
  • - Details on how scale testing will be done
  • - Information on demoing a user story
  • - Tools required to complete a user story
  • - Any exceptions in the ‘Definition of Done’
  • - Plan to address technical debt (if any)
  • - Dependencies between user stories (if any)

Definition of Done (DOD) is a simple checklist of important activities required to produce software. These include writing code, coding comments, unit testing, integration testing, release notes, design documents, etc.

• Scrums spend requisite amount of time in grooming
• Collective expertise of the team is utilized while grooming
• Scrums have stories appropriately groomed before they are available to be picked up in a planning meeting

All this in turn ensures less spill over due to unknowns factors resulting in quality and timely deliveries, increased customer satisfaction and happier scrums.