A test with no reliable record forces the team to decide from memory. Documentation is not about polishing a slide deck: it preserves what was planned, what actually happened, and what was inferred afterward. Strategyzer's Learning Card distinguishes hypotheses, observations, insights, and actions [R05].

Before the experiment: preserve the starting version

Record hypothesis ID and version, date, owner, protocol, conditions, intended participants, method, primary metric, and decision criterion. It should be clear what was defined before seeing outcomes. If you change a price or message, save the original version; do not silently overwrite it. The experiment design guide explains these fields.

During the experiment: record events, not just opinions

Use pseudonymous identifiers where possible. Note time, event, actor, source, outcome, incident, and interventions. For instance: “R03 received an offer on day 4; agreed verbally; did not complete enrollment; stated reason: no time.” Do not rewrite this as “R03 was very interested.” GOV.UK recommends notes based on observations and attention to consent [R19].

ID Observable event Related statement Incident
R01 Activated log and recorded 8 bookings “The change was easy” None
R02 Accepted demonstration; never activated “I could not make time for it” No usage recorded
R03 Activated and stopped on day 5 “Alerts double my work” Early exit

These are invented teaching data, not research results. A sound record distinguishes observed action from a participant's explanation.

Document denominators and missing values

Count how many contacts were assessed, eligible, invited, started, and completed. Do not equate cancellations with refusals or unreachable people with people who declined. If a record is lost, mark it missing and explain how it affects the metric; inventing a zero changes the data.

Incidents and protocol changes

Maintain a change log: date, modification, reason, approval, and which observations occurred under different conditions. If the system was down for six hours, do not exclude that period from availability calculations without disclosure. If a question led participants, correct it for future sessions while preserving evidence of the original wording.

To identify these distortions before the next session, see how to prevent bias and errors when running experiments. Keep the change versioned: documenting a bias does not show that its effects have disappeared.

From observation to learning: one card per hypothesis

Based on the four fields of the Learning Card [R05], our editorial record includes:

  1. We believed: original hypothesis and criterion.
  2. We observed: facts, figures, sources, and contradictions.
  3. We infer: possible explanations, confidence, and limitations.
  4. We decided: next test or change, owner, and date.

This expanded format is an Atlas adaptation, not an official reproduction of the Strategyzer card. Negative observations can be just as useful as favorable ones: if nine accounts activate but nobody uses them again, “strong adoption” is a misleading learning statement.

If the card blurs recorded events and interpretation, revisit the distinction between observations, assumptions, hypotheses, and evidence. A plausible explanation should not be presented as an event that was directly observed.

Four fields of the editorial learning record.
Explanatory diagram: Inspired by the conceptual fields of the Learning Card. This is an original Atlas teaching illustration; the relationships shown do not replace an actual study.

Privacy and traceability

Minimize identifiable details and allow consent records to be matched with study records when needed. Limit access, define purpose and retention, and do not publish names or audio without permission. Specific duties depend on jurisdiction [R10][R16].

What to deliver after a trial

An evidence package contains the original Test Card, versioned protocol, observed data, incidents, Learning Card, metrics calculation, limitations, and a dated decision. Analysis of rates and patterns follows in analyzing experiment data. If results open several alternatives, use a decision tree.

What you should have by the end: a traceable record a second reviewer can examine without asking you to reconstruct events. Main sources: [R05], [R09], [R16], [R19].

Bibliography and references