AGENT SKILL

adaptive-experimentation-strategy

Plan adaptive experimentation strategies beyond fixed-horizon A/B tests. Use when evaluating sequential testing, early stopping, multi-armed bandits, Thompson sampling, contextual bandits, dynamic traffic allocation, exploration/exploitation tradeoffs, or readiness for adaptive testing infrastructure.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 1.6k
Forks
491

01 前言 Front Matter

name
adaptive-experimentation-strategy
license
MIT
metadata
{ "tags": "practical-ab-testing,next-level-ab-testing,ab-testing,experimentation,adaptive-testing", "version": "0.1.0", "category": "Product Management", "displayName": "Adaptive Experimentation Strategy" }
description
Plan adaptive experimentation strategies beyond fixed-horizon A/B tests. Use when evaluating sequential testing, early stopping, multi-armed bandits, Thompson sampling, contextual bandits, dynamic traffic allocation, exploration/exploitation tradeoffs, or readiness for adaptive testing infrastructure.
compatibility
Codex, Claude Code, and other Agent Skills-compatible clients.

02 资源文件 5

03 SKILL.md 正文

Adaptive Experimentation Strategy

Use this skill to decide whether and how to use adaptive testing strategies instead of fixed-horizon A/B tests. It covers sequential testing, multi-armed bandits, Thompson sampling, contextual bandits, exploration/exploitation, and the data and engineering readiness required to operate them.

Source Traceability

Primary source: Next-Level A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from Chapter 7 on adaptive testing, sequential testing, multi-armed bandits, Thompson sampling, contextual bandits, and engineering requirements.

Related skills:

  • experiment-type-selection for choosing simpler experiment types first.
  • ml-experiment-evaluation for model and ranking evaluation paths.
  • experiment-verification-monitoring for operational health and alerting.

Reference Routing

Need Read
Adaptive testing concepts references/core/knowledge.md
Readiness and strategy rules references/core/rules.md
Scenario examples references/core/examples.md
Step-by-step adaptive readiness plan workflows/evaluate-adaptive-strategy.md

Workflow

  1. State the decision and why fixed-horizon A/B testing may not be enough.
  2. Decide whether the need is early stopping, reward maximization, or personalization.
  3. Check data freshness, reward definition, dashboards, on-call ownership, and rollback controls.
  4. Choose sequential testing, bandits, Thompson sampling, contextual bandits, or a simpler alternative.
  5. Document exploration/exploitation tradeoffs and user/business risk.
  6. Define rollout, monitoring, and adoption requirements.

Output Format

# Adaptive Experimentation Recommendation

## Use Case
[What decision or allocation problem motivates adaptive testing.]

## Recommended Strategy
[Do not use adaptive testing | Sequential | Bandit | Thompson sampling | Contextual bandit]

## Readiness
| Requirement | Status | Gap |
|-------------|--------|-----|

## Tradeoffs
- Reward:
- Exploration cost:
- Data freshness:
- Operational risk:

## Rollout Plan
1. [Step]
2. [Step]
3. [Step]

Quality Bar

  • Do not recommend adaptive testing just because it is advanced.
  • Do not use bandits when the real need is a clean causal estimate.
  • Do not use contextual bandits without reliable context features and reward measurement.
  • Do not ignore production requirements: stale data, bad allocation, and alert ownership can break adaptive systems.

04 同仓库更多 Skills

hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-review-with-annotations

Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-find-and-synthesize-context

Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-update-docs

Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-records

Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-interactive-html

Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-collaborate-threads

Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.

★ 1.6k