AGENT SKILL

ab-test-design-brief

Build product A/B test briefs with hypotheses, success metrics, guardrails, baselines, proxy metrics, eligibility, variants, randomization, confidence, and launch criteria. Use when planning an A/B test from a product idea, writing an experiment spec, defining test/control variants, choosing metrics, or checking whether an experiment is ready to run.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 1.6k
Forks
491

01 前言 Front Matter

name
ab-test-design-brief
license
MIT
metadata
{ "tags": "practical-ab-testing,ab-testing,experimentation,product-analytics,metrics", "version": "0.1.0", "category": "Product Management", "displayName": "A/B Test Design Brief" }
description
Build product A/B test briefs with hypotheses, success metrics, guardrails, baselines, proxy metrics, eligibility, variants, randomization, confidence, and launch criteria. Use when planning an A/B test from a product idea, writing an experiment spec, defining test/control variants, choosing metrics, or checking whether an experiment is ready to run.
compatibility
Codex, Claude Code, and other Agent Skills-compatible clients.

02 资源文件 5

03 SKILL.md 正文

A/B Test Design Brief

Use this skill to turn a product change into a decision-ready A/B test brief. It focuses on experiment anatomy: hypothesis, metrics, baselines, variants, eligibility, randomization, confidence, and launch criteria.

Source Traceability

Primary source: Practical A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from chapter 2, especially "Creating a Clear Hypothesis" through "Summarizing the For You A/B Test" in the working text analysis at lines 1203-1942. Related motivation and variant examples come from chapter 1 lines 394-718.

Related Advanced Skills

  • experiment-sensitivity-optimization: use when the brief is blocked by MDE, sample size, noisy metrics, CUPED, capping, or too many variants.
  • experiment-verification-monitoring: use when the brief needs prelaunch QA, canaries, exposure validation, or active experiment health checks.
  • long-term-impact-evaluation: use when the brief needs delayed or sustained impact measurement beyond the initial test window.

Reference Routing

Need Read
Concepts and terminology references/core/knowledge.md
Design rules and readiness checks references/core/rules.md
Brief examples and anti-examples references/core/examples.md
Step-by-step brief creation workflows/create-ab-test-brief.md

Workflow

  1. State the product decision the test must inform.
  2. Write a hypothesis with observation, predicted change, audience, and metrics.
  3. Choose one primary success metric plus guardrail metrics.
  4. Establish the baseline or explain why a proxy metric is being used.
  5. Define eligibility, exposure, test variant, and control variant.
  6. Choose the randomization unit that preserves a coherent user experience.
  7. Record confidence requirements, sample-size assumptions, and launch criteria.

Output Format

# A/B Test Brief

## Decision
[What decision this test will support.]

## Hypothesis
Because [observation], we believe [change] will cause [outcome] for [audience].
We will know this is true when [primary metric] changes without harming [guardrails].

## Metrics
| Metric | Role | Baseline | Target or Concern | Data Source |
|--------|------|----------|-------------------|-------------|

## Variants and Eligibility
- Population:
- Eligibility criteria:
- Exposure event:
- Control:
- Test:
- Randomization unit:

## Confidence Plan
- Minimum detectable effect:
- Sample size or duration:
- Risks to validity:

## Launch Criteria
- Ship if:
- Do not ship if:
- Investigate if:

Quality Bar

  • Do not accept a vague "see what happens" experiment.
  • Do not let proxy metrics hide missing instrumentation; name the compromise.
  • Do not generalize beyond the population that was eligible and exposed.
  • Keep variants interpretable: if many things change, the learning becomes weak.

04 同仓库更多 Skills

hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-review-with-annotations

Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-find-and-synthesize-context

Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-update-docs

Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-records

Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-interactive-html

Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-collaborate-threads

Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.

★ 1.6k