A useful experiment begins well before meeting the first participant. Establish what you need to learn, from whom, under what conditions, and which finding would change the decision. Strategyzer's Test Card keeps hypotheses, tests, measures, and thresholds explicit. GOV.UK recommends planning objectives, recruitment, materials, and practice sessions [R03][R20].
1. Define the decision and a bounded hypothesis
Write a question that matters now: “Will eligible managers use a booking log for fourteen days?” State what investment depends on the answer. Link the claim to the hypothesis writing guide. Before testing, decide what evidence would support, weaken, or leave the claim unresolved.
If you have not yet chosen a way to gather that evidence, see how to select the right experiment based on the decision, cost, time, and strength of the expected signal. Choose the method before detailing its protocol.
2. Recruit the people who matter
Define population, inclusion and exclusion criteria, location, and contact method. A restaurant booking service needs people who manage reservations, not only owners who never use the system. Document whether recruiting is convenient, experience-based, or probability-based. Convenience samples can produce useful insights but seldom justify sweeping population claims [R07][R08].
There is no magic participant count. Qualitative interviews, usability studies, surveys, and A/B tests have different objectives; statistical inference calls for a defined design and precision. GOV.UK provides guidance on methods and participant numbers but no universal sample size for all decisions [R07].
3. Design tasks, materials, and conditions
Specify the offer, steps participants see, device, instructions, facilitator, and what may vary. For comparative trials, make conditions comparable; consider group or task-order randomization when appropriate. NIST covers factors, randomization, and confounding in formal experimental design, but this does not mean every exploratory interview needs to become a laboratory trial [R11].
Before finalizing the conditions, review how to recognize and reduce experimental bias: participant selection, leading instructions, and variables changed together can distort comparisons. Safeguards should fit the question and the method.
If using a prototype, verify it supports the relevant tasks and make clear what is simulated versus functional.
4. Predefine metrics, denominators, and criteria
“Measure interest” is not specific enough. Define an event: request submitted, session completed, booking recorded, critical error, actual payment, or cancellation. Keep denominators: of 20 people contacted, how many qualified, received the offer, accepted, and completed? Document the period, measurement method, and decision to be made under different outcomes.
| Element in a hypothetical pilot | Planned definition |
|---|---|
| Hypothesis | Five of ten eligible restaurants will accept a pilot |
| Population | Managers responsible for telephone reservations |
| Procedure | Standard demonstration and identical fourteen-day offer |
| Primary metric | Acceptances / offers delivered to eligible contacts |
| Secondary metrics | Weekly use, errors, booking time, support |
| Threshold | 5/10, an example requiring business justification |
| Inconclusive outcome | Insufficient recruitment or changed conditions |
The threshold is hypothetical; acceptance by 5/10 does not mean half of all restaurants will buy. Accepting a free pilot also does not establish future payment.
5. Plan budget, duration, and responsibilities
Allow time to recruit, confirm consent, prepare the artifact, train staff, execute, analyze, and document. Specify a decision owner and, where possible, a separate recorder. Define stopping criteria for serious technical failures, participant risks, or commercial promises that cannot be fulfilled.
The protocol's budget, duration, and owners should be carried into a realistic schedule with dependencies, availability, and float. A test requiring unavailable equipment, approvals, or participants is not ready simply because a start date has been entered.
6. Practice before the real test
Run through the protocol; check timing, instructions, logging, and artifact functionality. A pilot may reveal leading questions or tasks that fail to measure the hypothesis [R20][R18]. If you change a quantitative protocol after starting, version the change and avoid pooling incompatible observations as though they were identical.
7. Consent and ethical limits
Explain who is researching, purpose, voluntary participation, recording, data usage, and retention. Do not present a presale as an available product if you cannot fulfill it; explain relevant conditions and refunds. Specific legal duties vary by jurisdiction; GOV.UK guidance offers research practices rather than universal legal advice [R10][R16].
If the test involves personal data, payments, preorders, or a regulated activity, use the guide to identifying applicable legal and operational requirements before running it. Do not infer specific legal duties from guidance written for another jurisdiction.
Protocol template
ID · version · blocked decision · hypothesis · risk category · recruitment criteria · method · tasks · materials · metrics/denominators · thresholds · duration · consent · owners · incidents · stopping conditions · expected limitations. Compare the finished record with objective results documentation to complete the cycle.
What you should have by the end: a protocol another person can understand and run without redefining the objectives. Main sources: [R03], [R07], [R08], [R10], [R11], [R18], [R20].
Bibliography and references
- [R03] Strategyzer. Validate Your Ideas with The Test Card. https://www.strategyzer.com/library/validate-your-ideas-with-the-test-card — Hipótesis, prueba, métrica y umbral.
- [R20] GOV.UK Service Manual. Plan a round of user research. https://www.gov.uk/service-manual/user-research/plan-round-of-user-research — Prácticas piloto y planificación de sesiones.
- [R07] GOV.UK Service Manual. Plan user research for your service. https://www.gov.uk/service-manual/user-research/plan-user-research-for-your-service — Preguntas, métodos y selección de participantes.
- [R08] GOV.UK Service Manual. Finding participants for user research. https://www.gov.uk/service-manual/user-research/find-user-research-participants — Reclutamiento, pertinencia y sesgos de selección.
- [R11] NIST/SEMATECH e-Handbook. Choosing an experimental design. https://itl.nist.gov/div898/handbook/pri/section3/pri3.htm — Variables, aleatorización, factores de confusión y diseños comparativos.
- [R18] Strategyzer. Designing Strong Experiments. https://www.strategyzer.com/library/designing-strong-experiments — Precisión de hipótesis, artefactos, participantes.
- [R10] GOV.UK Service Manual. Getting informed consent for user research. https://www.gov.uk/service-manual/user-research/getting-users-consent-for-research — Consentimiento y transparencia en investigación.
- [R16] GOV.UK Service Manual. Managing user research data and participant privacy. https://www.gov.uk/service-manual/user-research/managing-user-research-data-participant-privacy — Protección y retención de datos.