In the 18th episode of my blog series on “20 Controversial Topics of Debate in Agile,” the debate topic is how to handle defects in Agile (aka bugs). I always try to hammer out whenever I start working with a new Scrum Team because every team I have been on seems to do something different.
What is a bug, anyway?
Defects, otherwise known as “bugs,” are issues found during testing. However, there is a subtle distinction between broken functionality versus new (or missed) requirements.
Something is broken
When a system throws an error, returns incorrect or invalid data, bombs out, or otherwise doesn’t work as intended – it’s broken. If a function doesn’t meet all documented Acceptance Criteria or pass all Test Cases, then you’ve got a defect. These are things that clearly cause a system to be unusable, and they must be fixed – period.
Missed or New Requirements
Here’s a dirty little secret of Business Analysis: there’s no such thing as perfect requirements – something is usually missed or overlooked. When testing, it’s common to discover missed, hidden, implied, or new requirements. And that’s okay. What’s not acceptable is treating these as bugs – because they’re not. When this happens, the items should be added as new User Stories – not addressed as issues with the original User Story.
Treat bugs at the Story level or as tasks under a User Story?
Depending on what type of tool you are using to manage your Product and Sprint Backlogs, you may (or may not) have the option to configure how bugs are handled. My favorite tool for managing requirements is Microsoft’s Azure DevOps. That said, you might be constrained by your tool and unable to change how bugs are treated.
User Story Level
My preference is to keep bugs at the same level as User Stories. The advantage of this method is that it allows the Product Owner to prioritize the Product Backlog. Defects are considered along with everything else when making decisions about priorities. It also improves visibility into the amount of technical debt that exists in a system.
Task Level
Alternatively, you may treat bugs as children (rather than peers) of User Stories. This directly relates the defect to the source, providing better traceability. However, having bugs at this level may prevent you from closing stories if they have any remaining bugs. Some bugs are not worth fixing. My team’s “Definition of Done” states that no critical or high bugs may remain before acceptance. However, if there are lower severity bugs, this will quickly cause problems, and the bugs may get lost.
To log a bug, or not to log a bug?
When you find an issue while testing, do you ask yourself:
“If I find something wrong, should I log a bug? Or not?”
My typical consultant answer is:
“It depends.”
It’s a quick fix, so don’t log a bug
If it’s something minor, it doesn’t seem worth the overhead of creating a bug with steps to reproduce. In those instances, I go straight to the Developer and ask if they can make a quick change. If it takes longer than 15 minutes to fix, it justifies the work; but if it’s something simple, it is just not worth it.
It’s going to take some time, so log a bug
If my Developer tells me the issue needs to be investigated, s/he can’t get to it right away, or it seems like a more serious issue than just a quick fix, then go ahead and log a bug. Capture the steps to reproduce, screenshots, system configuration, version, etc. – anything you need to find the root cause of the issue so you can correct it.
Which backlog should defects go in?
Your Product Backlog feeds your Sprint Backlog. But if you find a defect during a Sprint, where should that bug be logged?
The current Sprint Backlog
My rule of thumb is that if a defect is found that impacts a User Story in the current Sprint, it should go in the Sprint Backlog. Typically, it would be best to fix these things right away versus burying them in the Product Backlog. This just leads to the accumulation of technical debt. However, if it’s a low or medium severity issue that could be deferred, the Scrum Team could put it in the Product Backlog instead.
The Product Backlog
If a defect is found in a production system, but it’s not critical or high in severity, it should probably be logged in the Product Backlog versus the Sprint in progress. These are called “escaped defects,” and they are yet another form of technical debt that you should eventually address. However, if the bug is impacting a production system, it should be fixed immediately. There are also often service level agreements for these types of defects that need to be met.
Should bugs be assigned Story Points?
My opinion on this is: NO. Story Points are realized for valuable work items that are produced, not for problems or issues found. You should still estimate tasks for fixing bugs to account for the team’s capacity, but you shouldn’t get credit for them. They should have been done correctly in the first place.
Final Thoughts…
Defects or bugs can be pesky to deal with, and each Scrum Team handles bugs somewhat differently. There are many options, which can lead to confusion; my advice is to figure this out as soon as your team is formed. You can always adjust later if you find something isn’t working.
Now it’s your turn to tell me what you think! I’m curious to know how your team handles defects. Please share your thoughts with me in the Comments section below!
Next up in this blog series:
- Can you use a Sprint 0 in Agile?
- Do you need Documentation in Agile?
- Is there a “right” way to write User Stories?
- What’s the best way to write Acceptance Criteria?
- How long should your Agile sprints be?
- Should all your Agile teams be run the “same” way?
- Which “flavor” of Agile is best?
- Can Agile co-exist with Waterfall?
- Story Size – What’s an Epic, Theme, Feature…?
- Can the Scrum Master be a team member, too?
- Can distributed Agile teams work?
- How should you estimate in Scrum & Agile?
- Is it OK to add items to the current Sprint?
- How should you manage your Agile backlog?
- Can an Agile team have more than one Product?
- What is the optimal size for Agile teams?
- Should Agile teams stay together?
- How should you handle defects in Agile?
- Can a hybrid of Waterfall and Agile succeed?
- What are the official roles on an Agile Team?
Ummm. . . . Last time I said this I started a minor ruckus, but this is the internet, I can’t help myself, and no one knows where I live, so I’m going to say it anyway. Scrum has no concept of “bugs”. The term is never mentioned in the scrum guide, not once. The part that starts the ruckus is when people assume this means they shouldn’t use bugs….but that’s not the intent. The team is always free to use whatever PBI types provide value be it user stories, bugs, tasks, issues, or something else. That being said, although I’d use them I’ve never cared for the label because most of the time it doesn’t change anything. i.e. say Donald Duck can’t do his job because the software crashes. If I write it as a bug it will have an issue/objective and a set of steps. If I write it as a user story it will have an objective (App shouldn’t crash on Donald) and some criteria (the set of steps). Either way it will get prioritized, test cases written, estimated, etc. The value I’ve found in labeling something as a bug is for tracking improvement purposes. But then we get into scenarios where Donald has a user ID with a “:” in it that caused the crash. Should that be a bug because the dev team should know the app shouldn’t crash, a new user story because the original user story (assuming you can find it) doesn’t say anything about supporting non-alpha-numeric characters, or do we just flag it as rework so the team can talk about it at a retrospective. Add to that I may not know if it’s a true “bug” until after I’ve confirmed the “:” was really the cause of the issue by fixing it. (TFS doesn’t allow changing the PBI type without recreating it which I would argue is waste if the issue has been solved, changes checked in, etc.) But most people I’ve worked with seem to feel better if they can label stuff as bugs…so I usually appease them…. After all, It does put a cool icon in TFS.
My 2 cents
“Don’t just fix the bugs; fix whatever permitted the bugs in the first place.” – Anonymous