Posts Tagged ‘Bug Tracking’

30 January

Bug Tracking 3: Minimal Bug Report Information

Bug reports vary greatly in length and complexity depending on your product, corporate culture and needs but there are some fields that I believe are a must for any bug report. A bare minimum, if you will. This happens to be my own list and other testers will have different fields they consider mandatory but I tend toward a bit of minimalism.

Title
A clear and concise title helps everyone. I make it a practice of putting unique information in the title like error codes or exe names. This makes it easier to later search the bug tracking system for duplicates or regressions rather than file a new bug.

Environment
A summary of the environment the bug occurs in as it relates to what you are testing. It could be as simple as the operating system or as complex as the os, browser, version of flash, etc.

Priority
How important do you think this bug is to fix? This should be important to the overall product and it may be higher priority if it’s blocking other testing or progress.

Product or Feature Area
What product (in a shared tracking system) or what feature area this bug occurred in. This serves to help when deciding who to assign bugs to or what areas of the product need concentrated attention.

Assigned To
This is the current owner of this bug and who is responsible for the next action with it.

Repro Steps
These are the steps, in order, required to reproduce this bug. They should be complete enough that someone unfamiliar with that bug can accurately reproduce it. I recommend a numbered list. If an additional file or script is needed to reproduce the bug, the location of that should be noted here or the file or script attached if the bug tracking system supports it.

Summary
This is a clear statement of what the issue is and why this is a bug. This is the place to make a case for this bug to be fixed so customer impact as well as expected behavior should be listed. If the bug is not high priority, that should be listed here as well. This is not a place to try to game the system and artificially cry wolf on bugs that really don’t deserve it.

Opened By
Who opened the bug. This allows questioning of the person, if necessary.

Opened Date
Date the bug was opened. This is useful for determining bug age.

Opened Version
This is the version of the product the bug is being reported for. It tracks what version was tested, no matter the date the bug was filed.

Fixed By
Who fixed the bug.

Fixed Date
Date the bug was fixed.

Fixed Version
Version of the product the fix will appear in. This allows the person verifying the fix to know when to expect it to be fixed instead of guessing.

Closed By
Who verified the bug was fixed.

Closed Date
Date the bug fix was verified.

29 January

Bug Tracking 2: Keep the Focus on the Product

The basic purpose of creating and tracking bugs is to improve the product you are testing. It seems really simple, almost too simple, but every other purpose bugs and bug tracking is made to serve in addition to this one can dilute this focus and create problems. You have to take each choice to use the bugs and your bug tracking system and consider what that choice can mean to the goal of product improvement.

Here are several real-world examples of situations where a secondary use for the bugs and bug tracking system affected the primary focus:

  • Zero Bug Mandates
    There’s a very common rule on teams I’ve worked on that the product cannot be shipped if there are any open bugs. Obvious and makes sense, no? It does make sense in theory, but what I’ve seen happen in practice is that the team focus shifts from making the right decision for the product to making sure that bug count is at zero and stays there. Good bugs are discarded with inadequate investigation or review. Bugs are closed instead of deferred. Bugs aren’t filed because the tester doesn’t believe they will receive the correct attention and instead they are forgotten or scrawled on a notepad so the tester might file them later-if they remember to do so.

    If your project has a Zero Bug Mandate, you have to make sure you have a process that supports bug retention, addressing the bugs that don’t make the cut for the current release but which do need to be addressed. You also need to encourage people to still file the bugs they find right away. Depending on how you track bugs, you may want to create a new database or release for the next iteration of the project for these bugs. If you are using an agile process, this are part of the product backlog.

  • Bug Metrics as Indicators of Individual Performance
    Sometimes a decision is made to use bug metrics as a way to measure the performance of team members. The typical logic is that bug activity correlates to “work” on the part of the various team members and there should be a way to use these concrete bug metrics to judge how much work any team member is doing relative to other team members. It sounds good, but what it does not take into account is that all bugs are not created equal in terms of severity, work to find, work to fix or effort to find or regress. The effort moves from a focus on finding and solving bugs as expeditiously as possible to finding and fixing a large quantity of bugs, regardless of their quality or the quality of the fix.

    If you are using bug metrics as a measure of individual performance, you need to insure it’s not the main way performance is judged. Bugs should be assessed for root cause and similarity and duplicates or related bugs identified as such to prevent cluttering the bug tracking system and unduly influencing your measurement process. Honestly, bugs are a very poor measurement of performance and I never advocate using them for that purpose for individual team members.

Stay tuned for Bug Report Basics!

15 January

Bug Tracking 1: Potential Red Flags

A discussion on a group I belong to on LinkedIn got me thinking about bug/defect tracking and the different systems I’ve used for it over the years along with the relative successes or failures of these systems.

Tracking bugs is a core need in software development. Bugs can be generated by the development team (all disciplines within the team), by customers, by product support, etc. This insures bugs that are found are followed, even if not fixed, and metrics from bug tracking help determine things like when a version of the product is ready to release, when a QFE or patch may be needed, etc.

For such a simple task, bug tracking can quickly become a complex and sometimes frustrating chore and this is often due to one or more of the following:

  • Tracking More Than Bugs
    This tends to lead to field creep – an explosion in the fields in the bug form – in order to accommodate things like specifications, work items, customer issues, etc.
  • Tracking Every Available Piece of Information
    Without a basic agreement on what information MUST be in a bug report, the tendency is to track every piece of available information so nothing is missed. This is another contributor to the issue of field creep.
  • Lots of Required Fields
    If fields are made “required” when they are not always applicable, people will put junk in them to get past the requirement. At the point this starts to happen, the usefulness of that field’s information disappears. The same thing happens when the list of required fields is large – people will put anything in just to be done with the bug report.
  • Field Overlap
    If there is more than one place to put the same exact same piece of information, users will begin to make mistakes, omit that information, or enter contradictory information.
  • Lack of User Training and Documentation
    Users of the bug system should be trained in the way its implemented and used and what is required of them along with a good idea of the process used to address bugs and the bug lifecycle in use in their company. When the first solution to lack of compliance is to set up the bug tracking system to enforce rules, it can cause worse behavior because the users still don’t know the rules, instead they work at getting around the automatic enforcement.
  • Shared Bug Tracking
    Sharing bug tracking among multiple projects, sometimes throughout a company, can lead to huge bug forms with a lot of redundancy or fields that are meaningless to all but one project.
  • No Central System Control
    Lack of a single person or core v-team that is responsible for managing the bug tracking system can lead to both having no one in charge of fixing issues that inevitably arise, but also to people just randomly adding fields or field contents without a master plan or vision.
  • Off the Shelf Configuration
    Using a bug tracking solution off the shelf, without reviewing it and possibly customizing it for the product, the development process and the needs of the team is akin to the square peg, round hole problem.
  • Bug Tracking not Periodically Reviewed
    Using the same bug tracking solution and setup over and over without reviewing it periodically to insure it’s still applicable and optimized for the needs of the company, team and product can eventually erode its usability and usefulness.

These are the potential red flags from my own list and years of managing bug databases and bug tracking systems. I’m sure there are more and you can leave those in comments.