What are Software Bugs and What Do They Want?

Setting the Stage

In this article, we explore and discuss differing thoughts on the subject of bugs and their inevitable impact on your project. Can you, in a Scrum model, add them to your sprint with no estimating or prioritizing? How can you estimate a bug? How do you determine in your application which issue types take precedence: new features or defects?

Just like any good debate, we have an interviewer posing questions and 2 colleagues from the Product/Project Manager team arguing their solutions. Let’s do this!

Debate the Issues

Question One: Problems, issues, broken features, oh my! There are always bugs in software. Bugs represent work required to fix an issue and improve quality of the product. If we assert that all work needs prioritization, are bugs then considered part of a product backlog?

 Absolutely! Bugs are synonymous to planned work not done, work that slipped through our quality assurance guards, or work impacted by recent changes.

• They should be recorded, reviewed, and prioritized with the product manager.
• Each sprint can have new features and a few rolled over prioritized bugs (scoped/unscoped).
• Within a sprint, keep checking with the product manager for re-prioritization between stories and any blocker production bugs and their impact on the sprint.

 Defects are a part of software development. There should be processes and tools — unit and integration tests — that identify defects, and developers should address them immediately upon discovery. Adding these types of defect tickets to the backlog cause too much overhead and increases management time. Consistently entering these bugs would not accurately reflect the overall status of the project. In the time it might take to write up and triage the defect, it might just as easily be resolved.

• However, bugs found during a phase like UAT are added to the team’s backlog to be triaged for priority and clarification.
• Not all “bugs” are the same. All too often a “bug” is not a functional issue, but rather an unclear user experience or a needed change to scope. People can be too quick to enter an issue as a bug when they might be better classified as a feature change — to be scoped and implemented at the appropriate time.

Question Two: Do we estimate the work required to fix and test a bug that was discovered? Work is work, right? Or are there other reasons why we don’t estimate?

 I strongly believe the delivery team should take accountability for the bugs (i.e. work not done, or incorrectly done).

• “Same sprint feature bugs” – should not be estimated additionally
• “Backlog bugs” – Ideally, I would not estimate bugs. But if they are older unclear issues, it’s a good idea to consider adding a spike to investigate and determine the effort needed to fix. It could help the product manager prioritize them based on their effort. This would also help in planning/reporting, to convey how many more sprints we would approximately need to complete the project.

I would be sensitive and balance expectations before assigning effort to bugs:

• “Is the team casual about the bugs being reported since bugs are counting towards velocity (hence, no incentive to mitigate the bugs)?”
• “Will the client be concerned about bugs reducing the velocity for feature development?”

There is nothing wrong with estimating bugs, as long as these expectations are clear, and it works best for the team and the client. A word of caution, be mindful of the team velocity and ensure it is being maintained.

 We should take the same approach for prioritizing and estimating bugs as any other piece of development. Here is why:

• Code presumably will need refactoring. Which, for all intents and purposes, is writing code. You’ve estimated code development for a user story, why wouldn’t you do the same for this new code?
• Unit and integration tests will need implementation or updating.
• This new work will need to be scoped, prioritized, agreed to, and assigned. Again, just like you do with a user story.
• The sprint impact will be the same as a user story — there is a functional change or addition to the previous build.

Question Three: Regarding work effort for bugs and commitment to fix a certain number of bugs within a sprint, how do you figure out what the team can commit to if a bug isn’t estimated?

 As mentioned previously, bugs are development overhead and it is the team’s responsibility to address them as soon as possible.

• In sprint planning, it would be good to check with team members on prioritized bugs. Ask the team if they understand what the issue is. Do they know what the overall solution is? Can they commit to fixing the bug, considering other new features being delivered?
• Even if the team has no recorded estimates, this method still takes into consideration the impact on time and if that issue can be addressed.
• Teams need to put in more time to address the work not completed (i.e. bugs).
• That being said, any development-heavy fixes or large impact bugs should be analyzed, discussed for solutions and estimated, because they can affect the velocity to a greater extent.

 It is critical for a team to make commitments, define what done means, and project with certainty their ability to deliver. Without these triggers in place, the team will certainly misalign their sprint when reviewing the defect, either by under-committing or over-committing. Either way, you have lost the ability to accurately predict the effort required for the sprint. I agree that bugs can in many projects become overhead — I have witnessed it on many teams. However, they become overhead when they are not treated with the same weight and consideration as any other work effort during the SDLC. In regard to committing to “fix a certain number of bugs” a project lead should never fill a sprint with the full established velocity. Try to leave 10% to 20% space to allow for any number of impediments the team might encounter. Bugs (refactoring) are key among these impediments. Committing to resolve a fixed N of yet unknown bugs in a sprint is a futile effort. Do allow for them as a general rule but recognize you cannot plan for an unknown count.

Question Four: Let’s dive into the struggle between overall quality and the business value added by new feature development. Please discuss influencing product owners to prioritize bugs within a sprint over new features. In other words, how do you assign business value to bugs?

 Business value for our clients is measured based on “happy customers.” This means the product should have good quality, be usable, and have new features important to end customers. So there needs to be a balance.

• Consider a periodic “bug review” meeting with the product owner. This is a good option for those prioritization discussions. Product owners are willing to prioritize functional defects and address technical debt over building new features if the bug’s impact is understood. As project leads, we are there to help guide decisions and foster a balance to sprint work and to keep things moving in the right direction.
• Handle with care. Technical debt is expensive. Interdependent features and reusable components with issues/bugs that are being extended, or having new features built on top in a coming sprint, are more costly. The longer you wait, the larger the negative impact to your budget and product quality.
• Users become easily frustrated having to deal with product issues. If you have UX teammates, seek out user experience recordings to demonstrate these user frustrations to the PO. This will help plead the case for bug priority over features.

 Overall quality is a business value. To take that to an extreme, you cannot make the statement that all MVP features were delivered on time if 25% of those features are faulty. Bugs that impact overall functional use in an application have the same value as the feature itself. Scrum is an agile process. By definition, in a successful sprint, you deliver potentially shippable feature sets incrementally. An analogy used here is known as the Train Model. The train departs the station on a known, reliable schedule, as determined according to the cadence established. If a feature misses a train, it can catch the next one. However, it is not currently shippable. When project leads fall out of this model or it is simply ignored, technical debt is incurred. This is why you imbue identified bugs with the similar business value.

Question Five: The dedicated bug sprint. What happens when there’s “never enough time” to fix all the bugs. Quality is suffering. The support team is overwhelmed. We’ve all been there. Are bugs estimated in this situation of a bug bashing sprint?

 When there is a conflict of interest between the product team and support team (i.e. new features vs. bug fixes), scoping defects can help make decisions regarding priority.

• Scope the defects, address solutions, and prioritize the bugs based on business value.
• After a go-live release, many teams decide to take a break from new feature development and plan a “bug fix sprint”. This could be shorter than a standard sprint depending on the needs of the project.

 In short, yes. However, this leads to a point where you would ask yourself, and the team, how did we get to the point that we need an entire sprint dedicated to bug fixing? If you find yourself in the position that you have accumulated enough defects worth addressing for an entire sprint, you should go through the processes you’ve established. Whether the bug requires a quick fix or significant investigation, it will require some code development. As mentioned, measure your velocity and capability to achieve a successful, effective and value-add sprint. In short: yes, estimate the work effort.

Conclusion

Every team has their own approach to managing, estimating and prioritizing bugs. Remember, there’s no one right way to manage bugs or conduct a sprint.

Here are some takeaways from our discussion.

• Acknowledge bugs exist and will always exist in software. Plan for bugs but work with your team to mitigate them.
• Categorize reported issues into bugs, new features and change requests.
• Prioritize bugs against new features using a combination of effort and business value.
• Respect your team – work with them to find the best solutions and give praise for their battle against bugs.
• Be mindful of harboring technical debt; it will eventually sink your (product) ship
• Don’t be afraid of a Bug Bashing sprint – make it fun for the team!
• Read more on estimating and predicting your future with confidence.

A special thank you to Dave Collins and Gauri Mohile for accepting this challenge!