AGENT SKILL

measure-experiment-results

Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations. Use after experiments conclude to communicate findings, inform decisions, and build organizational knowledge.

License 许可
Apache-2.0
最近更新
2026-11-14
Stars
★ 805
Forks
102

01 前言 Front Matter

name
measure-experiment-results
license
Apache-2.0
metadata
{ "phase": "measure", "author": "product-on-purpose", "updated": 1781049600000, "version": "2.1.0", "category": "reflection", "frameworks": [ "triple-diamond", "lean-startup", "design-thinking" ] }
description
Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations. Use after experiments conclude to communicate findings, inform decisions, and build organizational knowledge.

02 资源文件 4

03 SKILL.md 正文

<!-- PM-Skills | https://github.com/product-on-purpose/pm-skills | Apache 2.0 -->

Experiment Results

An experiment results document captures what happened when you tested a hypothesis, including statistical outcomes, segment analysis, learnings, and clear recommendations. Good results documentation turns individual experiments into organizational knowledge that improves future decision-making.

When to Use

  • After an A/B test or experiment reaches statistical significance
  • When an experiment is ended early (for any reason)
  • To communicate findings to stakeholders who weren't involved
  • During decision-making about whether to ship, iterate, or kill a feature
  • To build a repository of learnings that inform future experiments

When NOT to Use

  • The experiment is not designed or run yet -> use measure-experiment-design
  • The results demand a direction decision -> use iterate-pivot-decision; this skill reports the evidence, that one decides
  • You want the transferable learning banked for the organization -> follow up with iterate-lessons-log
  • Your data is survey responses, not a controlled experiment -> use measure-survey-analysis

Instructions

When asked to document experiment results, follow these steps:

  1. Summarize the Experiment Provide context: what was tested, when it ran, how much traffic it received. Link to the original experiment design document if one exists.

  2. Restate the Hypothesis Remind readers what you believed would happen and why. This frames the results interpretation.

  3. Present Primary Results Show the primary metric outcome clearly: what were the values for control and treatment? Include statistical significance (p-value), confidence intervals, and sample sizes. Be honest about whether results are conclusive.

  4. Analyze Secondary Metrics Present guardrail metrics that ensure you didn't cause unintended harm. Note any secondary metrics that moved unexpectedly.both positive and negative.

  5. Segment the Data Look for differential effects across user segments (platform, tenure, plan type, etc.). Sometimes overall results mask important segment-level insights.

  6. Extract Learnings What did you learn beyond the numbers? Include surprising findings, questions raised, and implications for the product hypothesis. Negative results are valuable learnings.

  7. Make a Recommendation Be clear: should we ship, iterate, or kill? Support the recommendation with the evidence. If the decision is nuanced, explain the trade-offs.

  8. Define Next Steps Specify what happens now.engineering work to ship, follow-up experiments, metrics to continue monitoring, or documentation to update.

Output Format

Use the template in references/TEMPLATE.md to structure the output. A complete readout fills every template section: Summary; Hypothesis Recap; Results; Segment Analysis; Visualization; Learnings; Recommendation; Next Steps; and Appendix.

Quality Checklist

Before finalizing, verify:

  • [ ] Statistical methods and significance are clearly stated
  • [ ] Confidence intervals are included (not just p-values)
  • [ ] Segment analysis checked for differential effects
  • [ ] Secondary/guardrail metrics are reported
  • [ ] Learnings go beyond just the numbers
  • [ ] Recommendation is clear and actionable
  • [ ] Negative or inconclusive results are reported honestly

Examples

See references/EXAMPLE.md for a completed example.

04 同仓库更多 Skills

product-on-purpose
product-on-purpose/pm-skills

utility-update-pm-skills

Checks for newer pm-skills releases, compares local vs. latest version, previews what would change, and updates local files after user confirmation. Generates a structured update report documenting changed files, new capabilities, and the value delta between versions. Use when you want to bring a local pm-skills installation up to date.

★ 805
product-on-purpose
product-on-purpose/pm-skills

utility-slideshow-creator

Generates professional presentations from a JSON deck specification using 18 slide types with dark/light variants, content-to-layout decision logic, and calibrated character limits. Ships with a default professional theme.

★ 805
product-on-purpose
product-on-purpose/pm-skills

utility-pm-workflow-orchestrator

Run an ordered sequence of pm-skills against one input, pausing for go/no-go and stopping on a failed or empty step. Accepts a saved prioritized action plan (Mode A) or an ad-hoc named chain (Mode B; the chain command routes here). Explicit invocation only; run --dry-run first while the native path is EXPERIMENTAL. To author a durable workflow instead, use utility-pm-workflow-builder.

★ 805
product-on-purpose
product-on-purpose/pm-skills

utility-pm-workflow-builder

Guides a contributor from a workflow idea to a complete Workflow Implementation Packet (draft workflow file, draft workflow command, cross-cutting update checklist) in a staging area for review. Runs overlap analysis against the existing workflows with a Why Gate, then helps select and sequence skills with authored handoffs. Use when creating a new multi-skill workflow or promoting a repeated ad-hoc chain into a durable one. To build a single skill instead, use utility-pm-skill-builder; to run a sequence without authoring anything, use the chain command or utility-pm-workflow-orchestrator.

★ 805
product-on-purpose
product-on-purpose/pm-skills

utility-pm-skill-validate

Audits an existing pm-skills skill against structural conventions and quality criteria. Produces a structured validation report with pass/fail checks, severity-graded findings, and actionable recommendations. Use when checking whether a skill meets repo standards before shipping or after making changes.

★ 805
product-on-purpose
product-on-purpose/pm-skills

utility-pm-skill-iterate

Applies targeted improvements to an existing pm-skills skill based on feedback, validation reports, or convention changes. Reads current files, previews proposed changes, writes on confirmation, and suggests a version bump. Use when improving a skill after validation or feedback.

★ 805