Product Task
Define the release success signal
The team can ship the change, but it cannot yet tell whether the intended product outcome improved or regressed.
“This is costing me analysis time while the release still lacks evidence for a clear product decision.”
Outcome
The result to aim for
Define the intended outcome, baseline, observable measure, decision window, and guardrail before release.
Best practices
- Bring the release decision, intended outcome, current baseline evidence, known risks, and data governance owner.
- Write the decision the team will make after release, the intended outcome, affected group, baseline, review window, and guardrail; then mark causal assumptions as assumptions.
- Have product, engineering, and data owners check the event implementation and known gaps; mark inputs Official only after the tracking-plan owner approves them.
- Confirm that the metric links to one named decision.
- Complete privacy, consent, retention, and access review before collection.
Related sources
Further information
- Documented guidance: GOV.UK Service Manual: Using performance data to improve your service: an introduction
- Related guidance: Hacker News: Are analytics good?