Scrum Product Backlog: What Works and What Does Not

A Scrum Product Backlog is the backbone of any product that is to be developed using Agile Scrum methodology. It is the master list of all desired functionalities and features that the product should incorporate. It may also contain a list of enhancements to existing functionalities or list of problems found during the earlier development iterations. Some Agilists have gone to the extent of recommending the addition of project themes, epics (large user stories), probable risks and issues to the Product Backlog, but whether a Product Backlog is a place for these elements is debatable.

Post-It notes, whiteboard, spreadsheets (MS Excel®) or web-based Scrum management tools – one can find a Product Backlog in any of these forms! When an Agile team is collocated and works in close proximity, Post-It notes, whiteboard or spreadsheets work great to manage and track user-stories. However, if the teams are working in a distributed environment, there may be a need for online Scrum management tools such as ScrumWorks® Pro or Scrum Time.

As collaboration is one of the fundamental principles of Agile, all stakeholders are encouraged to contribute their ideas towards the betterment of the product being developed. Yet, the Product Backlog is the property of the Product Owner who understands the business and values that the end-result would deliver.

Over a period of last 5 years, I have discovered and studied what works and what doesn’t when compiling and managing a Product Backlog. Though not comprehensive, this is my attempt to share what I have learnt from my experience about the Scrum Product Backlog, and it may prove nifty to anyone passionate about Agile.

What works...
1. Assigning priorities to Product Backlog entries based on their complexity and impact

When creating an entry in the Product Backlog, a user-story or requirement should hold a Complexity Factor and a User-Impact Factor. The Complexity Factor, on a very broad scale tells how much effort is involved in developing the particular requirement. The User-Impact Factor describes how important the functionality is for the end-user of the product while keeping in mind the ultimate goals of the product/service being developed. A matrix composed of complexity and impact for each Product Backlog item helps in deciding the priority of the requirement. E.g. A low complexity, high impact requirement would be at the top of the priority list, a high complexity, high impact somewhere in the middle, and high complexity, low impact right at the bottom. However, only the Product Owner can prioritize the Product Backlog. Also, items with highest priority are placed at the top of the Product Backlog.

Determining the complexity based on effort-estimate should be a team activity. Techniques such as PERT and Points System using Fibonacci Numbers can be used to compute effort-estimate, but their discussion is beyond the scope of this article.

At least one person who understands the business and can ably represent the end-users should be part of the team deciding the User-Impact of each item.

2. Ongoing maintenance of the Product Backlog

Regularly reviewing, and adding detailed definitions to the items that are at the top of the Product Backlog ensures that there is a well defined, prioritised and estimated Product Backlog for the upcoming few Sprint iterations. It is the Product Owner’s responsibility to ensure that the Product Backlog can continuously provide a stream of items for the Sprint Backlog.

What does not work...
1. Everyone has “Full Access” to the Product Backlog

Duplication, requirements not qualified according to the original goals of the product/service, and irrelevant or vague demands are some of the side-effects of giving everyone full control of the Product Backlog. E.g. before I came on board on one of the projects that I was handling, everyone including the sales team, senior management, customer support, business analysts and developers had access to the Product Backlog. As a result, the Product Backlog looked more like a dumping- ground or a place-holder for people’s thoughts rather than a collection of user-stories and probable requirements for the product being developed. Several entries in the Backlog were duplicated or did not at all support the goals of the product. Eventually, we had to put in several hours worth of work to filter out the irrelevant requirements and consolidate the duplicate ones. After filtering and consolidating (and using up 15 person days), the number of entries in the backlog came down from 600+ to under 450!

Although creating a Product Backlog should be a group-activity, it’s best not to relinquish control of the backlog to anyone and everyone in the team. All users’ demands and requirements should be discussed using techniques such as brainstorming, interviewing and survey (Delphi), but entries should be moderated by a select few, preferably by the Product Owner and business analysts. At no point should the moderators lose focus of the ultimate goals of the product/service being developed and every new probable item should be scrutinized to see if it fits these goals. Restricting the number of people creating entries in the backlog will reduce the chance of duplication and thus decrease waste of time and effort.

2. User-stories entered in the Product Backlog are too brief

One-liners in the Product Backlog leave too much room for speculation and imagination about what the user and the requirement actually demand. If such requirements somehow slip through into the Sprint Backlog, there is a good chance that someone misinterprets the one-liner and converts it into a functional specification far from the customer’s actual needs. This can cause considerable rework, schedule slippage and budget overshoot. E.g. consider a one-line requirement “Showroom manager would like to see a report of used cars sold.” The requirement does not specify anything about which columns the report should contain, the dates between which the records should be picked, or how they should be sorted.

User-stories sitting very low in terms of priority and without any plausible chance of making it to a Sprint Backlog in the near future can be exceptional cases, and can be sketchy at the most.

Moderators can reduce problems arising from brief user-stories by deciding on some ground-rules and format for writing the stories. Along with a broad description of what the requirement demands, the following details should also be added to the user-stories: effort-guesstimate, impact on product’s functionality, priority, and name of end-user or customer who made the demand. Wherever possible, the requirements should be quantifiable, e.g. “Screen for ‘All Vehicle Details’ is taking more than 8s to load – should load in less than 4s.” All this will reduce the scope of misinterpretation, need to redefine the requirements, and thus reduce chances of wrongly developing the functionality.

Following these simple guidelines when managing a Scrum Product Backlog should steer most new Agilists clear of the pitfalls that we fell in when we started off on this path. After all, we don’t want to reinvent the wheel, do we?

Agile Testing - "Develop n Test n-1" Approach

While Agile emphasizes an approach to software development that is more adaptive to changing requirements and enables more interaction between the development team and clients or stakeholders, it goes without saying that even software testing has to shape up along these Agile principles. Therefore, unlike Waterfall, where integration testing, functional testing and user acceptance testing (UAT) take place only when the deliverable service/product starts getting a demonstrable form, which typically could be as late as the last quarter of the entire project lifecycle, Agile proposes all aspects of testing (unit, functional, integration and user-acceptance) and the involvement of the customer at a very early stage in the project lifecycle.

But, have you ever been in a situation where you adopted Agile development methodology and your project schedule was slipping because the customer-representatives were not performing UAT or providing feedback fast enough? If yes, then the remainder of this passage might provide you with a probable solution.

Some critics might blame the schedule slippage because of delayed action from customer-representatives on lack of commitment to Agile on the part of involved stakeholders. Some might even say that it is lack of understanding of Agile methodology on the part of the project manager. But from my understanding such situations may arise because it’s not always possible for the customer to provide dedicated resources for testing and collaborating with the development team, and yet the client wants to go the Agile way. Often the client ends-up providing subject matter experts (SMEs) who have to take up the responsibilities of testing and providing feedback in addition to their day-to-day activities. To keep up with the true spirit of Agile, it will only be apt if the continuous user testing and feedback process can be adapted so that it doesn’t overload the customer representatives.

To illustrate the solution for such a problem let us consider a typical Agile scenario. Let us assume that the development team is working in 2-week iterations. So these 2 weeks (10 business days) will be packed with design, development, unit testing, integration testing and UAT. Let us say that last 2 of the 10 days are assigned to UAT. If the customer representatives fail to complete their testing and provide their input within these 2 days, the development team cannot have a complete plan for the next iteration. Thus the delay from the previous iteration has overflowed into the upcoming iteration. If this happens a few more times, it can snowball into unmanageable schedule slippage. And the project manager will soon have to start the juggling act of balancing the schedule, the scope, the resources, etc.

To counter this problem, we had devised the Develop n Test n-1 approach. While the development team is developing iteration n, the customer would be carrying out UAT on iteration n-1. Going back to our earlier example, our 2-week iterations would now only contain design, development, unit testing and integration testing. UAT is isolated and moved to the next iteration. Assume that the development team has just released the 1^st iteration. When the development team commences work on iteration 2, the customer commences testing of iteration 1. When the development team is ready with the release of iteration 2, the UAT is complete on iteration 1. The results and feedback from testing of iteration 1 along with the originally time-boxed requirements for iteration 3 become the new requirements for iteration 3. As the development team commences work on iteration 3, the customer starts testing iteration 2. The progress can be depicted in a chart as shown below.

 

This approach has two significant advantages:

1.       The customer gets the entire span of iteration for UAT.

2.       The development team does not have to wait on customer feedback to start planning and development of the subsequent iteration.

Working within the constraints and overcoming limitations can be possible with such simple, yet innovative and adaptive changes in various phases of software development.  After all, continuous improvement, adaptation, increased interaction with customer, and simplicity are some of the key principles of the Agile manifesto.

Adapting Agile for Fixed-Price Contracts

Agile development methodology is growing quickly in today’s software world due to features like:

·         Flexibility on the part of the software vendor to implement client’s changing requirements during the development phase.

·         Continuous intermediate releases (sprints) allowing the customer to get a look ’n feel of the end-product/service early in the development cycle.

·         Customer collaboration to generate feedback on intermediate releases and use it positively.

·         Increased usability of delivered end-product/service.

But who should absorb the cost of change if the product or service is being developed under a fixed-price contract? Implementing Agile in fixed-price contracts can be a tricky if the customer demands changes to originally signed-off requirements as per the statement-of-work, but is not willing to pay for them.  It’s highly unlikely that the software vendor would bear the increased cost, and thus lose on its profit-margin. This, in-turn, almost always affects the end-product, decreasing its user-friendliness, maintainability and overall product-life.

One option that allows Agile to be used in fixed-price projects is to factor in a buffer into the total cost of development at the beginning. What this means is that the software vendor provides the customer the cost of developing the project as per the original requirements, and then adds a buffer, let’s call it Agile Cost Buffer, for any changes/additions that the customer may suggest during the development phase. The software vendor not only declares the Agile Cost Buffer in the contract with the customer, but also clarifies the amount worth of work (perhaps in person-days) that this buffer would cover. 

E.g. Customer C1 wants to develop a product with a set of original requirements;

Software Vendor S1 studies the requirements and based on its analysis comes up with a total effort estimate of P1 person-days and development cost of $T1;

S1 has determined that the Agile Cost Buffer should be b% and they can deliver w% more work of the originally estimated effort.

In the contract between C1 and S1, the original development cost should then be mentioned as $T1, and the total development cost including Agile Cost Buffer should be mentioned as $T1 + b%. This additional cost would cover all existing requirements plus any changes/additions worth w% of P1 person-days.

If the entire Agile Cost Buffer is not used, provision can be made in the contract to adjust it in the final payment to the vendor.

This will not only safeguard the margins of the software vendor, but will also protect the customer from the exorbitant rates that some vendors charge for change requests. Above all, it will fulfil the primary principles on which Agile is based, which are increased customer satisfaction and increased usability of the delivered software.

Why is Waterfall not Agile?

For years now, Waterfall has been the most common development methodology in Information Technology sector. There has been a bigger faction of IT project managers who believe in this rigid set of management philosophy, which states that for a project to be successful it has to be distributed into phases of requirements gathering, design, development, testing and deployment, or the Waterfall method in Software Development Life Cycle (SDLC). Breaking away from these norms and rituals was going to be tricky and difficult, if not blasphemous! At the beginning of this century though, a group of revolutionary software developers penned a public declaration of their opinions and ideas about how to develop a more adaptable, client-centric and usable software. This, they called the Agile Manifesto.

Agile in its various forms (Extreme Programming, Pair Programming, Scrum, Adaptive Software Development, etc.) is still catching up in terms of its number of supporters against those of Waterfall. Primarily there are two main reasons for resistance to adoption of Agile as a preferred methodology. Firstly, till 90’s, we as Computer Science students were only taught about Waterfall method of software development as part of our curriculum. Therefore, these students who form today’s bulk of software architects or technical managers are more familiar and feel comfortable working in Waterfall. Secondly, it’s the relative ease of implementation - once requirements for a project are frozen in the early stages, project managers can concentrate on their project plan and tasks at hand without having to worry about any further changes in specifications from stakeholders. Typically, these managers do not feel the need of breaking the shackles to meet the customers’ additional or changed requirements after the project costing is negotiated and contract signed. Even their customers have got used to the notion that after requirements are finalized on paper, there can be little done to change the eventual output of the project. The cost involved in making the change plays an important role in their decision to not opt for the change (refer fig. 1). Therefore, change-requests, with their often steep rates, almost seem like a tool to deter the customer from demanding changes to the original signed-off requirements.

Fig. 1 Cost of Change and Adaptability to Change plotted against Time to develop software

Waterfall v/s Agile:

Waterfall methodology has based itself on the strict reigns that to build any software successfully, all requirements should be documented before the design is initiated, there has to be a plan, and the plan should be followed to the last dot. The customer should be thorough in his requirement specifications, and be able to imagine and notate every minute detail of the final product. The architects should be able to draw up a plan for development, and the team sticks to this plan throughout the execution. The managers have to monitor every bit of the plan and apply corrective procedures for the smallest hiccup. Agile understands that changes are imminent in practical software development. So the Manifesto advocates less on concentrating on documentation or following a plan, and more on delivering working software and responding to change. Software being developed using Agile adapt to the changing needs of the customer, allowing progressive elaboration on his/her requirements.

Another shortfall of Waterfall methodology is the time in development when the customer gets to see his/her developed product. Almost always, customers never see the developed product till it has some demonstrable functionality and user-interface to it. This often is after system-integration stage, which can be very late in the development cycle. Till this time the software is running a high risk of failure (refer: fig. 2) arising from having misinterpreted the initial requirements or change in requirements. Statistics say that 66% of the customers do not see what they are expecting on their first glimpse of the software when developed using Waterfall! And I believe that the project managers of other 34% projects are smart enough to have started the project with either a prototype that gives the customer a fair idea of what to expect. Agile can reduce the percentage of “shocked customers after the first glimpse of the software” to negligible. That is due to a combination of the iterative development method, increased interaction and customer collaboration emphasized by Agile. With every iteration the customer can see and feel a more complete version of the software, understand how various aspects of his/her requirements are being fulfilled, and provide feedback. An early glimpse of the evolving software gives the customer a better judgement of how the end-product can be made more efficient and usable, and also drastically reduces the risk of failure.

Fig. 2 Comparative Risks of Failure when Waterfall and Agile are adopted for the same project

Several other factors in the last decade or so have prompted the need to adapt software development to customers’ changing requirements. Reasons common to most changes in customers’ requirements are:

· End-users’ ever growing appetite for additional information;
· Relative decline in the cost of software development;
· Ability to develop software quicker;
· Fast-paced changes in technology that can provide new features, more processing power and faster applications;
· Market-competition.

A new breed of software-service vendors is surfacing, the one who understand that to stay in competition they have to be creative in their development strategy and adapt to the customer’s evolving needs. Business analysts, architects and managers are getting rid of the blinkers that forced them on the path of set norms and rituals of Waterfall; they are embracing Agile. They are now more receptive to customer’s feedback during development. They are flexible and tolerant to changes and pick up newest technology released in the market even during the development phase. The software that they then deliver is more closely aligned to the customer’s needs, more usable, has longer product-life and cheaper in the long-run. This is the essence of Agile.