The difficult question is not “What did I find that confirms my idea?” but “What else could explain what I am seeing, and what evidence would help me tell the explanations apart?”
Why good research can lead to a bad conclusion
Research does not improve simply by accumulating interviews, tables, or screenshots. Sources can be incomplete, participants may have been selected with bias, and our interpretations may reflect pre-existing wishes. The goal is not absolute certainty: it is making decisions based on an honest account of what was observed and what remains uncertain.
GOV.UK recommends turning research observations into findings through early, collaborative analysis to reduce the influence of individual interpretations [R07]. This article proposes simple editorial tools to do that across businesses and projects of different sizes.
First, separate five types of statement
Direct observation: “During the session, the manager checked three channels to confirm a reservation.” Statement: “The manager said some messages get lost.” Secondary record: “The support system includes fourteen inquiries tagged as reservations in one month.” Interpretation: “Fragmented channels may increase effort.” Hypothesis or decision: “We should investigate whether centralizing information would reduce errors.”
All may be useful, but they do not mean the same thing. An observation describes a limited situation; a statement captures someone's perspective; a record depends on how it was measured; an interpretation offers an explanation; a hypothesis proposes something to investigate. Mixing them leads to publishing conclusions as though they were facts.
How to seek evidence that contradicts an opportunity
Reverse your assumptions: if you believe all establishments lose reservations, look for cases where none are lost. If you think a service is too slow, study who considers it fast enough. Ask about recent decisions not to switch alternatives and why. Investigate whether the severity of a problem decreases when a business adopts a simple process already used successfully elsewhere.
Organize a four-column table:
| Belief | Supporting evidence | Contradicting evidence | What still needs clarification |
|---|---|---|---|
| “The notebook fails at busy times” | One establishment described duplicate bookings | Two establishments handle the same volume successfully | What distinguishes their workflows? |
| “Customers want online reservations” | Some ask for a booking link | Others prefer calling and receiving personal confirmation | Under which circumstances does channel preference change? |
Do not pretend both columns carry equal weight. A detailed observation of a critical failure may matter more than several vague opinions; a properly sampled dataset may be more useful for estimating prevalence than an anecdote. Always explain why you trust a particular piece of evidence.
Four biases worth watching
Confirmation bias. Paying more attention to information supporting the idea, asking questions that invite positive answers, or discarding inconvenient cases.
Selection bias. Studying only easily reached people—friends, followers, loyal customers, or businesses in one particular area. Indifferent users and negative experiences may be excluded.
Recall and social desirability bias. Participants may misremember how frequently or how long something happened, or respond in ways they believe are polite and expected. Where appropriate, compare their accounts with recent examples and authorized observation.
Survivorship bias and missing data. Studying only surviving businesses or customers who stayed may hide reasons for leaving. Available records do not necessarily represent undocumented events either.
These categories summarize practical research risks; they do not establish that every study containing one is worthless. Better research questions, recruitment, and cross-checking methods can reduce their impact [R03, R06, R07].
How strong is a source?
Assess it explicitly: Who produced the data, and for what purpose? When and where? Which population does it include or exclude? What exactly does the indicator mean? How was it collected? Are there alternative explanations? Can you trace it to its origin? An industry study may describe broad trends well yet provide weak evidence for a problem in one district.
Do not confuse an official source with automatic applicability: a public statistic may be perfectly calculated but describe a different population from yours. Nor should you equate a large sample with a well-selected sample: thousands of volunteer answers do not necessarily remove systematic bias.
What to do with small samples
In qualitative research, a small sample can provide deep accounts of behavior and reveal variations in a problem. On its own, it cannot accurately estimate population percentages. GOV.UK distinguishes small qualitative rounds from quantitative research requiring different sample sizes and study designs [R03].
For example, if six of eight interviewees mention delays, report “six of the eight people interviewed” and describe their characteristics. Do not claim that 75% of restaurants experience delays. The frequency inside a sample does not automatically become a market statistic.
If your main sources are individual conversations, revisit how to interview without leading participants before interpreting patterns.
Triangulation and alternative explanations
Combine interviews, observation, records, and secondary data when they answer different parts of the same question. Suppose employees say “we are always running out of products,” but inventory records indicate shortages are concentrated just before weekly deliveries. This discrepancy does not mean you must choose one source and discard the other: it may reveal different definitions of “shortage” or a timing issue.
If you are unsure which approach could distinguish two competing explanations, revisit research methods beyond interviews before treating that difference as a settled conclusion.
Seek competing explanations. If sales fell, possible causes include lower demand, the end of a promotion, shorter opening hours, a new competitor, or recording changes. Without an appropriate design, do not attribute causation to the first plausible explanation. If you need to test an intervention's effect, the experimentation pillar will cover how to design a test and its decision criteria [R10].
When the evidence comes from tests run by your team, also review experiment-specific bias and results analysis.
Build a traceable evidence record
| Field | Example |
|---|---|
| Claim | “Reservation changes through messages interrupt customer service” |
| Source | Contextual observation during two authorized shifts |
| Date and setting | Week and type of establishment |
| Sample | Two establishments selected for convenience |
| Record | Incident notes without personal data |
| Limitation | Other shifts and establishments were not observed |
| Contradicting evidence | One shift with no meaningful interruptions |
| Implication | Investigate when the pattern occurs |
The record separates documentation from interpretation and makes it possible to correct findings as new information appears. Protect participant data and use only material collected with appropriate consent or authorization [R08, R09].
Example: a home-service technician platform
Five interviewees say it is difficult to trust unknown technicians. The temptation is to conclude “the market wants verified reviews.” But two interviewees hired successfully through recommendations; one cannot remember a bad experience; and the others had punctuality problems rather than workmanship problems. The issue may not be “missing reviews” but different ways of reducing risk before a visit.
A responsible conclusion documents the incidents, separates trust, punctuality, and price, and proposes examining real hiring decisions. The eventual solution might be a platform, a guarantee, a procedure, or something not yet considered.
How to communicate uncertainty without losing usefulness
You do not need to write “perhaps” in every sentence. Be specific: “We observed three incidents across five shifts,” “Interviewees were recruited through convenience sampling,” “We did not estimate prevalence,” or “This interpretation needs checking in a different type of establishment.” Explicit uncertainties tell the next researcher what remains unknown instead of allowing them to assume it has been established.
The next step is to use those explicit limitations to decide whether the opportunity is worth pursuing.
For more detail, see how to assess business model viability with scenarios.
Common mistakes
Changing criteria after seeing results without documenting the change; choosing only favorable testimonials; averaging incompatible categories; presenting secondary data without dates or definitions; treating correlation as causation; ignoring contradictory cases; and sharing identifiable quotes or recordings beyond the permissions obtained.
What you should have when you finish
A record of traceable findings, their sources and limitations; a table of supporting and contradicting evidence; alternative explanations; and open questions that matter for the decision. You are not building a collection of arguments in favor of your project: you are creating a basis for correcting it when necessary.
Main sources: [R03] GOV.UK, research planning; [R06] participants; [R07] analysis; [R08] consent; [R09] privacy; [R10] Strategyzer. Examples are hypothetical.
Bibliography and references
- [R07] GOV.UK — Analyse a research session. https://www.gov.uk/service-manual/user-research/analyse-a-research-session — Separar datos de investigación, observar patrones, incorporar perspectivas.
- [R03] GOV.UK — Plan user research for your service. https://www.gov.uk/service-manual/user-research/plan-user-research-for-your-service — Preguntas de investigación, actividades, reclutamiento y planificación de rondas.
- [R06] GOV.UK — Finding participants for user research. https://www.gov.uk/service-manual/user-research/find-user-research-participants — Criterios de reclutamiento y diversidad de participantes.
- [R10] Strategyzer — How to test your idea: start with the most critical hypotheses. https://www.strategyzer.com/library/how-to-test-your-idea-start-with-the-most-critical-hypotheses — Supuestos críticos y dimensiones de riesgo: deseabilidad, factibilidad y viabilidad; la experimentación detallada corresponde al pilar 3.
- [R08] GOV.UK — Getting informed consent for user research. https://www.gov.uk/service-manual/user-research/getting-users-consent-for-research — Consentimiento y explicación de propósito/uso de los registros.
- [R09] GOV.UK — Managing user research data and participant privacy. https://www.gov.uk/service-manual/user-research/managing-user-research-data-participant-privacy — Gestión segura de datos y limitación de finalidad. La normativa explicada es británica; no debe atribuirse automáticamente a Perú ni a otros países.