A test with no reliable record forces the team to decide from memory. Documentation is not about polishing a slide deck: it preserves what was planned, what actually happened, and what was inferred afterward. Strategyzer's Learning Card distinguishes hypotheses, observations, insights, and actions [R05].
Before the experiment: preserve the starting version
Record hypothesis ID and version, date, owner, protocol, conditions, intended participants, method, primary metric, and decision criterion. It should be clear what was defined before seeing outcomes. If you change a price or message, save the original version; do not silently overwrite it. The experiment design guide explains these fields.
During the experiment: record events, not just opinions
Use pseudonymous identifiers where possible. Note time, event, actor, source, outcome, incident, and interventions. For instance: “R03 received an offer on day 4; agreed verbally; did not complete enrollment; stated reason: no time.” Do not rewrite this as “R03 was very interested.” GOV.UK recommends notes based on observations and attention to consent [R19].
| ID | Observable event | Related statement | Incident |
|---|---|---|---|
| R01 | Activated log and recorded 8 bookings | “The change was easy” | None |
| R02 | Accepted demonstration; never activated | “I could not make time for it” | No usage recorded |
| R03 | Activated and stopped on day 5 | “Alerts double my work” | Early exit |
These are invented teaching data, not research results. A sound record distinguishes observed action from a participant's explanation.
Document denominators and missing values
Count how many contacts were assessed, eligible, invited, started, and completed. Do not equate cancellations with refusals or unreachable people with people who declined. If a record is lost, mark it missing and explain how it affects the metric; inventing a zero changes the data.
Incidents and protocol changes
Maintain a change log: date, modification, reason, approval, and which observations occurred under different conditions. If the system was down for six hours, do not exclude that period from availability calculations without disclosure. If a question led participants, correct it for future sessions while preserving evidence of the original wording.
To identify these distortions before the next session, see how to prevent bias and errors when running experiments. Keep the change versioned: documenting a bias does not show that its effects have disappeared.
From observation to learning: one card per hypothesis
Based on the four fields of the Learning Card [R05], our editorial record includes:
- We believed: original hypothesis and criterion.
- We observed: facts, figures, sources, and contradictions.
- We infer: possible explanations, confidence, and limitations.
- We decided: next test or change, owner, and date.
This expanded format is an Atlas adaptation, not an official reproduction of the Strategyzer card. Negative observations can be just as useful as favorable ones: if nine accounts activate but nobody uses them again, “strong adoption” is a misleading learning statement.
If the card blurs recorded events and interpretation, revisit the distinction between observations, assumptions, hypotheses, and evidence. A plausible explanation should not be presented as an event that was directly observed.
Privacy and traceability
Minimize identifiable details and allow consent records to be matched with study records when needed. Limit access, define purpose and retention, and do not publish names or audio without permission. Specific duties depend on jurisdiction [R10][R16].
What to deliver after a trial
An evidence package contains the original Test Card, versioned protocol, observed data, incidents, Learning Card, metrics calculation, limitations, and a dated decision. Analysis of rates and patterns follows in analyzing experiment data. If results open several alternatives, use a decision tree.
What you should have by the end: a traceable record a second reviewer can examine without asking you to reconstruct events. Main sources: [R05], [R09], [R16], [R19].
Bibliography and references
- [R05] Strategyzer. Capture (Customer) Insights and Actions with the Learning Card. https://www.strategyzer.com/library/capture-customer-insights-and-actions-with-the-learning-card — Resultados, aprendizaje y acciones.
- [R19] GOV.UK Service Manual. Taking notes and recording user research sessions. https://www.gov.uk/service-manual/user-research/taking-notes-and-recording-user-research-sessions — Registros de observaciones y trazabilidad.
- [R10] GOV.UK Service Manual. Getting informed consent for user research. https://www.gov.uk/service-manual/user-research/getting-users-consent-for-research — Consentimiento y transparencia en investigación.
- [R16] GOV.UK Service Manual. Managing user research data and participant privacy. https://www.gov.uk/service-manual/user-research/managing-user-research-data-participant-privacy — Protección y retención de datos.
- [R09] GOV.UK Service Manual. Analyse a research session. https://www.gov.uk/service-manual/user-research/analyse-a-research-session — Diferenciar observaciones e interpretaciones.