AGENT SKILL

ab-test-results-readout

Analyze and communicate A/B test results with metric readouts, subgroup analysis, data-quality checks, ad hoc investigation, visualization, and launch recommendations. Use when interpreting experiment results, preparing an A/B test report, explaining flat or mixed results, checking guardrails, segmenting test/control data, or turning experiment data into a product decision.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 1.6k
Forks
491

01 前言 Front Matter

name
ab-test-results-readout
license
MIT
metadata
{ "tags": "practical-ab-testing,ab-testing,experimentation,analytics,data-visualization", "version": "0.1.0", "category": "Product Management", "displayName": "A/B Test Results Readout" }
description
Analyze and communicate A/B test results with metric readouts, subgroup analysis, data-quality checks, ad hoc investigation, visualization, and launch recommendations. Use when interpreting experiment results, preparing an A/B test report, explaining flat or mixed results, checking guardrails, segmenting test/control data, or turning experiment data into a product decision.
compatibility
Codex, Claude Code, and other Agent Skills-compatible clients.

02 资源文件 5

03 SKILL.md 正文

A/B Test Results Readout

Use this skill to turn experiment data into a clear decision. It emphasizes metric interpretation, data-quality checks, subgroup analysis, ad hoc analysis, visualization, and launch recommendations.

Source Traceability

Primary source: Practical A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from chapter 4 lines 2950-3742 and chapter 1 lines 572-718. Metric tradeoff context comes from chapter 2 lines 1296-1472.

Related Advanced Skills

  • trustworthy-experiment-insights: use when the readout needs false positive, false negative, power, replication, meta-analysis, or suspicious-lift review.
  • experiment-verification-monitoring: use when result interpretation depends on whether assignment, exposure, metrics, canaries, or active monitoring were healthy.
  • long-term-impact-evaluation: use when short-term readout is not enough to decide durable product or business impact.

Reference Routing

Need Read
Readout concepts references/core/knowledge.md
Analysis and reporting rules references/core/rules.md
Example readouts references/core/examples.md
Step-by-step report workflow workflows/prepare-results-readout.md

Workflow

  1. Reconstruct the test design: hypothesis, variants, population, and metrics.
  2. Verify data quality and whether exposure/eligibility match the brief.
  3. Compare primary and guardrail metrics against baseline and decision rules.
  4. Run subgroup analysis when averages obscure meaningful differences.
  5. Investigate outliers, missing data, or surprising movement.
  6. Visualize results so stakeholders can compare control, test, and segments.
  7. Recommend ship, stop, iterate, or investigate with caveats.

Output Format

# A/B Test Results Readout

## Executive Decision
[Ship | Stop | Iterate | Investigate] because [reason].

## Test Summary
- Hypothesis:
- Population:
- Control:
- Test:
- Run window:

## Metric Results
| Metric | Role | Control | Test | Change | Interpretation |
|--------|------|---------|------|--------|----------------|

## Segment Findings
| Segment | What changed | Decision impact |
|---------|--------------|-----------------|

## Data Quality Notes
- Eligibility/exposure:
- Missing data:
- Outliers:
- Instrumentation concerns:

## Recommendation
- Decision:
- Rollout conditions:
- Follow-up analysis:

Quality Bar

  • Do not hide guardrail regressions behind a primary-metric win.
  • Do not overstate subgroup findings; label them exploratory when not pre-planned.
  • Do not show only averages when the product decision depends on user groups.
  • Use charts to clarify comparisons, not to decorate the readout.

04 同仓库更多 Skills

hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-review-with-annotations

Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-find-and-synthesize-context

Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-update-docs

Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-records

Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-interactive-html

Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-collaborate-threads

Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.

★ 1.6k