Finding twelve problems does not automatically mean twelve development tasks. First establish what happened, who was affected, what repeats, and what deserves attention.
A solution test can uncover defects, usability observations, new needs, doubts, requests, opportunities, and risks. If every item becomes a feature in the backlog, the team loses context and may fix symptoms rather than causes. The challenge is turning findings into traceable decisions. GOV.UK user-research guidance recommends feeding findings into planning and prioritization [R01]; Nielsen Norman Group suggests examining frequency, impact, and persistence when assessing the severity of usability problems [R02].
1. Distinguish observations, findings, defects, and requests
| Term | Meaning | Example |
|---|---|---|
| Observation | Recorded event without assumed cause | Three participants could not find “Save” |
| Finding | Evidence-based interpretation | The confirmation may be insufficiently visible |
| Defect | Reproducible failure against expected behavior | Selected time disappears after saving |
| Incident | Event disrupting an execution or service | Reservations unavailable for a period |
| Request | Change someone would like | “Add color-coded calendars” |
| Risk | Possible future event with consequences | Duplicate confirmation after a sync failure |
| Opportunity | Plausible improvement needing evaluation | Reduce steps in a frequent change |
A user's statement alone does not prove that everyone needs a feature. A technical defect does not necessarily explain its cause. Keep original evidence, causal hypotheses, and decisions in separate fields. Mark insufficiently contextualized reports as incomplete rather than treating them as actionable facts.
When a finding comes from an experiment, also preserve the original results and evidence log. An observation from a commercial test does not automatically equal a functional defect.
2. Create a record that supports reproduction and understanding
Each entry should have an ID, date, solution version, source, actor or segment, context, observed steps, expected and actual results, available evidence, known impact, and investigation owner. Add device, batch, shift, materials, elapsed time, rework cost, and human interventions where relevant.
A cosmetic issue and a security incident do not need identical form lengths, but both need traceability. Store video, screenshots, and logs responsibly with minimal personal data and appropriate permissions. If someone reports a problem no one has reproduced, mark it “not yet reproduced”—neither false nor confirmed.
3. Group by problem, not by the first label you see
A slow screen, delayed notification, and duplicate booking might share a synchronization issue, or have three unrelated causes. Group by flow, affected outcome, component, evidence type, and suspected origin, but do not merge findings without investigation.
Distinguish a duplicate report (two descriptions of one event), a recurring pattern (several people experience the same problem), and related symptoms (different problems that may share a cause). Preserve links from source records to consolidated findings. Do not delete contradictory evidence to make the story simpler.
Grouping informs investigation and correction; it does not prove root cause.
4. Separate severity, priority, frequency, and impact
Severity describes how serious the problem is when it occurs. Priority describes how soon the team should address it, considering severity, risk, obligations, value, and capacity. Frequency describes observed occurrences in a stated context and denominator. Impact describes consequences for people, operations, costs, or outcomes.
A rare defect exposing sensitive records can be both severe and immediately prioritized. A frequent cosmetic annoyance may be tolerable until a later iteration. A blocking defect affecting a smaller segment should not be ignored simply because it was observed less often.
Nielsen Norman Group considers frequency, impact, and persistence when classifying usability problems [R02]. This is domain-specific guidance, not a universal formula for physical hazards, legal duties, or cybersecurity.
5. Classify before ordering the backlog
First mark mandatory blockers: safety, compliance, significant data loss, or inability to deliver core value. These should not compete with decorative changes through arbitrary weighted scores.
Then distinguish defects requiring repair; experience issues needing redesign and testing; requests needing value evidence; uncertainties needing research; technical or process debt; and deferrable improvements. GOV.UK's agile manual notes that backlogs contain more than new features [R03].
Use labels such as blocking, important, investigate, later, but include reasons. “Not in this version” is not a synonym for “does not matter.”
6. Prioritize using evidence, consequences, and effort
Ask for each finding: What outcome does it affect? Who is affected? How serious is the consequence? How often has it been observed and under what conditions? How confident are we about the cause? What effort is needed? What other work depends on it? What happens if it remains open?
Do not assign arbitrary numbers merely to generate a ranking. Use ranges or qualitative assessments when data is insufficient. For critical problems, impact and obligations outweigh team convenience. If two issues appear similar, consider reversibility, cost of delay, and what further learning a correction would enable.
A request from a prominent customer may deserve examination, but it is not automatically the highest priority without considering other users and commitments.
7. Run a decision-oriented triage review
A triage meeting can be short, but it should produce decisions:
- Check that each report has enough context and evidence.
- Identify duplicates, relationships, and possible causes without claiming certainty.
- Assess severity and impact, marking blockers explicitly.
- Choose fix, investigate, reproduce, defer, explicitly accept permissible risk, or reject with a reason.
- Assign owner, review trigger, and target version without inventing an unapproved deadline.
- Define how to check whether a fix solved the problem.
- Communicate the decision appropriately to people who reported it.
Risk acceptance is limited by actual obligations: teams cannot make mandatory safeguards optional. “Rejected” does not mean the observation never occurred. It can simply mean the requested change is not justified now.
8. Keep a decision log separate from the backlog
The backlog lists possible work. A decision log preserves the reasoning. A useful entry includes ID, date, problem, version, evidence, alternatives, costs and risks, decision, owner, residual uncertainty, and the signal that would trigger reconsideration.
For example: “We will not add color-coded booking types in V1 because the current evidence points to confusion about confirmation, not visual identification. We will review if observable cases emerge.” This is more informative than “no time” or “we do not like it.” Retain changing criteria and meaningful disagreements; do not overwrite earlier reasoning.
If a finding shows that the specified condition was insufficient, return to requirements and acceptance criteria to revise expectations with traceability. Passing an earlier criterion does not make a real usability problem irrelevant.
9. Verify the fix and retain the learning
“Ticket closed” is not proof the problem ended. Test the original condition, reproduce the failing scenario, rerun affected checks, and look for new issues. A UI adjustment may fix mistaken clicks but create accessibility barriers; a technical correction may change an integration.
If the problem remains unreproducible or results are inconclusive, record limits and the next review trigger. A favorable screenshot should not be used to fabricate confidence.
Worked example: twelve findings from a booking app
A fictional test produces 12 entries:
| Initial group | Entries | Proposed treatment |
|---|---|---|
| Two reports of the same save failure | 2 | Consolidate as one defect while retaining both reports |
| Confusing confirmation across screens | 3 | Group as a UX pattern; investigate context |
| Unauthorized access to unrelated records | 1 | Block release; review security |
| Requests for colors and filters | 3 | Research value, postpone decision |
| Inconsistent error messages | 2 | Check for common cause and criteria |
| One isolated cosmetic request | 1 | Record for later review |
These are instructional numbers, not real data. Twelve entries do not equal twelve features. Triage flags unauthorized access as a blocker, saving as a critical defect, confirmation as a UX investigation, and color preferences as requests without adequate evidence. Error messages may share a cause, but that remains a hypothesis.
The team prepares a backlog of actual actions, not just labels, and a decision log explaining exclusions and future triggers. After changes, it reruns original cases and related tests.
A decision to fix a problem must be linked to the version where the change will be made and checked. Without that link, a closed issue does not show which users actually received the fix.
Common mistakes and limitations
Prioritizing the most dramatic anecdote; calling every preference a defect; treating severity as identical to priority; counting duplicate reports as independent people; claiming causes without evidence; deleting negative findings; turning research directly into features; ignoring obligations when prioritizing; closing without retesting; maintaining a backlog without owners or exit conditions.
Triage organizes decisions under uncertainty. It does not replace root-cause investigation or qualified review when significant risks are involved.
What you should have when finished
A traceable findings register, a justified backlog, and a decision log specifying what to fix, investigate, or defer. Each correction should have verification criteria, an owner, and a link to the version where its outcome will be checked.
References
- [R01] UK Government Digital Service. Plan user research for your service (feeding findings into decisions). https://www.gov.uk/service-manual/user-research/plan-user-research-for-your-service
- [R02] Nielsen Norman Group, Jakob Nielsen. Severity Ratings for Usability Problems. https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/
- [R03] UK Government Digital Service. Deciding on priorities. https://www.gov.uk/service-manual/agile-delivery/deciding-on-priorities