An opportunity may begin with something genuinely observed and gradually turn into an invented story without anyone noticing. Good experimentation requires separating what happened, what we believe happened, and what remains untested. That distinction prevents us from designing tests that merely confirm our expectations [R07][R09].

Eight concepts that mean different things

  • Data: a record of an event, response, or measurement with a source and unit. “Four reservations were cancelled on Tuesday” is data if there is a reliable record.
  • Observation: a description of something seen or heard in context. “The manager wrote a reservation in a notebook and then spent three minutes finding it” does not yet explain why.
  • Statement: what someone says about their situation. It is evidence that the statement was made, not automatic proof that the behavior described occurred.
  • Interpretation: the meaning we assign to facts: “The notebook causes delays.” It may be plausible, but it is still an explanation.
  • Assumption: a claim we temporarily rely on without adequate support: “Every restaurant has this problem.”
  • Hypothesis: an assumption expressed precisely enough to test: “Among small restaurants using manual booking logs, at least half of managers will report weekly update problems.”
  • Evidence: relevant, assessable information that supports, qualifies, or contradicts a hypothesis; its strength depends on the method and context.
  • Conclusion: a judgment proportionate to that evidence, accompanied by remaining uncertainty.

Digital records can also contain mistakes, and careful observation is not universal truth. Documenting source, date, and method is part of the work [R19].

Sequence from observation and interpretation to testable hypotheses and evidence.
Explanatory diagram: Separate what happened from interpretation before testing. This is an original Atlas teaching illustration; the relationships shown do not replace an actual study.

One event can have several explanations

Imagine a restaurant taking unusually long to confirm a reservation. The delay could be caused by a disorganized notebook, a manager busy serving tables, a weak connection, or a customer changing the time. Writing “They need an app” immediately jumps from the event to a preferred solution.

Use four columns: observed, possible interpretation, other explanations, and what we would check. A follow-up interview may clarify the circumstances; estimating the true frequency may require reviewing logs and times. Exploratory methods appear in research methods beyond interviews. Once a claim can be examined, move to writing testable hypotheses.

Initial statement Classification Next step
“Five duplicate reservations were recorded yesterday” Reported data Request the log, time period, and definition of duplicate
“Customers hate calling” Unverified interpretation Observe behavior and existing alternatives
“They will pay for automated reminders” Business assumption State a hypothesis and test payment behavior
“Three of eight interviewees mentioned errors” Finding from this sample Record recruitment and avoid generalizing percentages

From a finding to a researchable question

A practical sequence is to describe the event without judgment, record its source, propose at least two explanations, identify which decision would change under each, and formulate the hypothesis worth testing. This is an editorial synthesis of research principles, not a single official procedure [R07][R09].

“Is an app useful?” is often too broad. “How many reservations are lost because of coordination errors in restaurants using manual processes, and at what cost?” identifies behavior, context, and consequences. Even so, an estimate from a few restaurants does not necessarily describe the whole market.

Once the claims are separated, classify hypotheses using desirability, feasibility, viability, and adaptability. Identifying the type of risk helps determine the evidence needed without turning one observation into a broad conclusion.

What makes evidence stronger

Relevance to the hypothesis matters as much as volume. A click establishes that an interaction occurred under particular conditions; it does not establish willingness to pay. An actual payment provides different information, but still does not establish future retention. Consider the source, who was excluded, potential pressure, measurement timing, and alternative explanations. For a broader assessment, read evaluating evidence without bias and documenting experiment results.

A minimal claims register

Each record can contain an ID, exact claim, claim type, source, date, scope, supporting evidence, contradicting evidence, uncertainty, and next question. Do not treat “we do not know yet” as “false”: inconclusive is a distinct category.

If this register contains many unresolved claims, prioritize the riskiest hypotheses by the consequences of being wrong and the evidence available before designing a test for each one.

What you should have by the end: a table separating observations, interpretations, and hypotheses before spending money on experiments. Main sources: [R07], [R09], [R19], [R02].

Bibliography and references