measure-experiment-results
Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations. Use after experiments conclude to communicate findings, inform decisions, and build organizational knowledge.
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-14
- Stars
- ★ 805
- Forks
- 102
01 前言 Front Matter
- name
- measure-experiment-results
- license
- Apache-2.0
- metadata
- { "phase": "measure", "author": "product-on-purpose", "updated": 1781049600000, "version": "2.1.0", "category": "reflection", "frameworks": [ "triple-diamond", "lean-startup", "design-thinking" ] }
- description
- Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations. Use after experiments conclude to communicate findings, inform decisions, and build organizational knowledge.
02 资源文件 4
- 参考文档evals/trigger-fixtures.json4.3 KB
- 参考文档HISTORY.md733 B
- 参考文档references/EXAMPLE.md10.3 KB
- 参考文档references/TEMPLATE.md4.2 KB
03 SKILL.md 正文
<!-- PM-Skills | https://github.com/product-on-purpose/pm-skills | Apache 2.0 -->
Experiment Results
An experiment results document captures what happened when you tested a hypothesis, including statistical outcomes, segment analysis, learnings, and clear recommendations. Good results documentation turns individual experiments into organizational knowledge that improves future decision-making.
When to Use
- After an A/B test or experiment reaches statistical significance
- When an experiment is ended early (for any reason)
- To communicate findings to stakeholders who weren't involved
- During decision-making about whether to ship, iterate, or kill a feature
- To build a repository of learnings that inform future experiments
When NOT to Use
- The experiment is not designed or run yet -> use
measure-experiment-design - The results demand a direction decision -> use
iterate-pivot-decision; this skill reports the evidence, that one decides - You want the transferable learning banked for the organization -> follow up with
iterate-lessons-log - Your data is survey responses, not a controlled experiment -> use
measure-survey-analysis
Instructions
When asked to document experiment results, follow these steps:
-
Summarize the Experiment Provide context: what was tested, when it ran, how much traffic it received. Link to the original experiment design document if one exists.
-
Restate the Hypothesis Remind readers what you believed would happen and why. This frames the results interpretation.
-
Present Primary Results Show the primary metric outcome clearly: what were the values for control and treatment? Include statistical significance (p-value), confidence intervals, and sample sizes. Be honest about whether results are conclusive.
-
Analyze Secondary Metrics Present guardrail metrics that ensure you didn't cause unintended harm. Note any secondary metrics that moved unexpectedly.both positive and negative.
-
Segment the Data Look for differential effects across user segments (platform, tenure, plan type, etc.). Sometimes overall results mask important segment-level insights.
-
Extract Learnings What did you learn beyond the numbers? Include surprising findings, questions raised, and implications for the product hypothesis. Negative results are valuable learnings.
-
Make a Recommendation Be clear: should we ship, iterate, or kill? Support the recommendation with the evidence. If the decision is nuanced, explain the trade-offs.
-
Define Next Steps Specify what happens now.engineering work to ship, follow-up experiments, metrics to continue monitoring, or documentation to update.
Output Format
Use the template in references/TEMPLATE.md to structure the output. A complete readout fills every template section: Summary; Hypothesis Recap; Results; Segment Analysis; Visualization; Learnings; Recommendation; Next Steps; and Appendix.
Quality Checklist
Before finalizing, verify:
- [ ] Statistical methods and significance are clearly stated
- [ ] Confidence intervals are included (not just p-values)
- [ ] Segment analysis checked for differential effects
- [ ] Secondary/guardrail metrics are reported
- [ ] Learnings go beyond just the numbers
- [ ] Recommendation is clear and actionable
- [ ] Negative or inconclusive results are reported honestly
Examples
See references/EXAMPLE.md for a completed example.
04 同仓库更多 Skills
utility-update-pm-skills
Checks for newer pm-skills releases, compares local vs. latest version, previews what would change, and updates local files after user confirmation. Generates a structured update report documenting changed files, new capabilities, and the value delta between versions. Use when you want to bring a local pm-skills installation up to date.
utility-slideshow-creator
Generates professional presentations from a JSON deck specification using 18 slide types with dark/light variants, content-to-layout decision logic, and calibrated character limits. Ships with a default professional theme.
utility-pm-workflow-orchestrator
Run an ordered sequence of pm-skills against one input, pausing for go/no-go and stopping on a failed or empty step. Accepts a saved prioritized action plan (Mode A) or an ad-hoc named chain (Mode B; the chain command routes here). Explicit invocation only; run --dry-run first while the native path is EXPERIMENTAL. To author a durable workflow instead, use utility-pm-workflow-builder.
utility-pm-workflow-builder
Guides a contributor from a workflow idea to a complete Workflow Implementation Packet (draft workflow file, draft workflow command, cross-cutting update checklist) in a staging area for review. Runs overlap analysis against the existing workflows with a Why Gate, then helps select and sequence skills with authored handoffs. Use when creating a new multi-skill workflow or promoting a repeated ad-hoc chain into a durable one. To build a single skill instead, use utility-pm-skill-builder; to run a sequence without authoring anything, use the chain command or utility-pm-workflow-orchestrator.
utility-pm-skill-validate
Audits an existing pm-skills skill against structural conventions and quality criteria. Produces a structured validation report with pass/fail checks, severity-graded findings, and actionable recommendations. Use when checking whether a skill meets repo standards before shipping or after making changes.
utility-pm-skill-iterate
Applies targeted improvements to an existing pm-skills skill based on feedback, validation reports, or convention changes. Reads current files, previews proposed changes, writes on confirmation, and suggests a version bump. Use when improving a skill after validation or feedback.