quality-gate
When a change owes an e2e test, and the gates that enforce it — pre-push e2e+smoke to master plus the ->review, ->merge and ->done gates. Use when changing the CLI, a shell script, the install flow or any installed hook, and when handing over, merging or closing a card.
- License 许可
- MIT
- 所属仓库
- muratovv/ai-hats
- 最近更新
- 2026-11-05
- Stars
- ★ 3
- Forks
- 1
01 前言 Front Matter
- name
- quality-gate
- ai_hats
- { "git_hooks": { "pre-push": [ "git_hooks/pre-push-e2e-master.sh" ] } }
- license
- MIT
- description
- When a change owes an e2e test, and the gates that enforce it — pre-push e2e+smoke to master plus the ->review, ->merge and ->done gates. Use when changing the CLI, a shell script, the install flow or any installed hook, and when handing over, merging or closing a card.
02 资源文件 6
- 脚本bin/runcheck.sh2.9 KB
- 脚本git_hooks/pre-push-e2e-master.sh5.6 KB
- 脚本hooks/done-gate.sh1.1 KB
- 脚本hooks/merge-gate.sh482 B
- 脚本hooks/review-gate.sh397 B
- 脚本lib/gate.sh18.6 KB
03 SKILL.md 正文
quality-gate
This directory is machinery, not prose. It carries the gates that guard the
ai-hats codebase — one thin script per edge in hooks/, and the git one in
git_hooks/ — and the role binds each by path (ai-hats-dev/roles/maintainer/config.yaml).
Delete it and the gates stop being installed.
A gate is a few lines: the stages it requires, and a call into lib/gate.sh.
hooks/done-gate.sh --stages prints the list. How a stage runs, and how a marker
is earned for it, is the project's scripts/gates.sh — the gate only hands the
list over. Nothing here runs a stage itself (ADR-0023 D6/D7).
| gate | where it fires | who is at the refusal | asks |
|---|---|---|---|
review-gate |
every road into review |
the agent, alone | is this work fit for a reviewer |
merge-gate |
wt:pre-merge |
the agent, alone | is this branch fit for master |
done-gate |
every road into done |
the supervisor | is master green after this card |
push-gate |
git push to master |
a human | is the pushed tree green |
Below the gates sits the policy they enforce: which changes owe an e2e test at all. That half was a standing rule until it folded in here — one home for the question and the machine that asks it.
When an e2e test is mandatory
A task may not reach done if it changed any of the trigger surface below
without adding at least one e2e test under tests/e2e/ that exercises the real
command chain.
The invariant: the gate fires on any change to code that runs outside the Python process — an entry point, a shell script, the install/venv flow, or any script the assembler installs as a hook. That is where the contract lives at a subprocess boundary the unit suite cannot observe. The list below names the current surfaces; a path that is not on it but crosses the same boundary fires the gate all the same.
src/ai_hats/cli/**/*.py— click commands, command nesting, CLI args/flags.scripts/*.sh— shell scripts (install-launcher.sh,bootstrap.sh, …).src/ai_hats/_bootstrap.py,src/ai_hats/cli/maintenance.py— pip install / launcher / venv flow.[project.scripts]in any workspacepackages/*/pyproject.toml— new or renamed entry points (the rootpyproject.tomldeclares none).packages/ai-hats-library/**/hooks/**and**/git_hooks/**— PreToolUse / PostToolUse hook scripts and the installed git hooks (git_hooksis a distinct segment: a**/hooks/**glob does not match it). Highest blast radius in the repo: a hook gates every tool call of every role composing it, so a four-line change can disable every agent. The test must drive the composed chain (all hooks on the matcher, in order) viatests/e2e/_helpers/hook_chain.py, never one script in isolation — a single-hook test cannot observe a second hook overriding its verdict, which is how a blanket deny once shipped past a green suite.- Anything else crossing an external contract: PEP 508 URL forms, click nesting, shell quoting, venv invocation.
Does not fire: internal Python modules (storage, parsing, business logic), docs, tests-only changes, version bumps.
What counts as an e2e test
All of these must hold:
- Lives under
tests/e2e/(the dedicated real-subprocess CLI layer — seetests/README.md). - Marked
@pytest.mark.integration. - Spawns a real subprocess chain: real
bash, realpip install, realai-hatsbinary. NoMagicMock, nomonkeypatchonsubprocess.Popen, noCliRunner.invoke(). - Asserts observable end-to-end side effects (exit codes, files on disk, captured output) — not internal call counts.
In-process CliRunner tests do not satisfy this, regardless of marker.
Neither do pipeline-integration tests.
Plan and review
The plan names the test by path and by what it asserts — "will add e2e coverage"
is not a name. The reviewer runs it, and checks the one thing a green run cannot
show: that it fails when the change is reverted. If it does not, it lives
beside the change rather than exercising it, and the card returns to execute.
Origin: PROP-031 — two production bugs shipped past done because the unit
suite stubbed the very contracts the change broke.
Passing a gate
-
Transition.
rack transition <ID> review/done, orai-hats wt merge. Run nothing first: the gate reads markers inside the lock and, when they are missing, refuses with the command that earns them. -
Refused? The refusal ends with that command — run it:
Missing: <stage> <stage> Run the gate on that exact content, then retry: cd <worktree> && make done-gate(
make done-gate REV=<sha>for a card already merged; the stages a gate requires are<gate>.sh --stages, never a list in this file.) It runs only the stages the tree has not earned, in order, stops at the first red, and stamps each green one.A stage the gate does not declare —
e2e-rack, say, absent from<gate>.sh --stages— is a zone: an area of the codebase and the tests that assert it. The gates BEFORE the merge (review,merge) require what they declare PLUS the zones your diff touches, so the set depends on what you changed and cannot be declared in advance;->donenever asks for them, so you pay for your zone once —<gate>.sh --zonessays which gate asks.bash scripts/gates.sh touchedprints what your change adds andbash scripts/gates.sh zonesthe whole table;make <gate>earns them like any other stage. Nothing is wrong with the gate — read it as the tests your own area owns, asked for where you can still fix them alone. -
Green — the run's first line is
RESULT …, green. Its last says what the transition after this one will additionally demand, so you can earn it now instead of being refused for it later:[gates] next: done-gate also needs <stage>, <stage>next: nothingmeans no gate on the road is short of anything. Transition. -
Red — the run's block, verdict first:
[gates] RESULT tree 5f225a1b (2ac163eb): 11 cached, 1 ran, FAILED unit (rc=1) [gates] fix unit, then: make done-gate [gates] re-run just these: /path/to/repo/.venv/bin/python -m pytest tests/test_x.py::test_y [gates] unit said: tests/test_x.py:12: AssertionError: … 1 failed, 6024 passed, 2 skipped in 40.1s [gates] lint: 1203 files already formatted … [gates] cached (11): e2e-catalog … [gates] 2ac163eb (tree 5f225a1b) in place: <worktree> [gates] transcript: /tmp/gates-run.Xy12abIn order of importance: the verdict, what to do, what the red stage said (a failing test per line, or a finding per line), one line per green stage, the cached ones, the subject, and the dir holding every stage's full output.
fix <stage>assumes the failure is yours, and the gate cannot tell. When the red is in something this change did not touch, attribute before you fix or ask for anything: skill red-attribution. Cheapest first — a stale base is a more common cause than a broken master.Triage the red stage:
- a red pytest stage hands you the narrow command already — paste the
re-run just theseline. Do not rebuild it: the interpreter in it is the one this checkout answers for, and another imports someone else's sources; - the stage alone, bare, exactly what CI runs:
bash scripts/gates.sh unit(bash scripts/gates.sh listsays what each stage checks); - a checker: the tool itself,
ruff check <path>orpython scripts/check_<name>.py; - fix, commit,
make <gate>again. A fix is a new tree, and a new tree earns every stamp afresh: the checkers take seconds,unitabout a minute — do not reach for--freshor a narrower gate to save it.
the desk is dirty, under the verdict? A run with uncommitted changes happens in a scratch checkout of the commit, so it judges what is COMMITTED — a red one against a fix that was never there, a green one vouching for a fix that never ran. Commit, then run again. - a red pytest stage hands you the narrow command already — paste the
-
Touched what the gate does not name? No card gate runs the whole tier under one name:
->mergeasks for the zones your diff touches,->donefor the half no zone claims (done-gate.sh --stages), and never for your zones again. What that leaves out is a zone your change breaks WITHOUT touching its prefix, and nothing refuses there. Run the tier yourself and say so. It outlives a foreground call, so run it in the background through the wrapper, which exits with the tier's own status:timeout 1800 bash packages/ai-hats-library/src/ai_hats_library/ai-hats-dev/skills/quality-gate/bin/runcheck.sh \ --log /tmp/e2e.log -- bash scripts/gates.sh e2eReport the number in
/tmp/e2e.log.rc, never the completion notice: that notice reports the whole command, so a hand-rolled; echo $?is what it announces — two red tiers arrived asexit code 0that way. What a gate requires of every tree:bash packages/ai-hats-library/src/ai_hats_library/ai-hats-dev/skills/quality-gate/hooks/done-gate.sh --stages
What a stamp saves
A marker names the TREE a stage ran on, per stage. It spares the re-run of an unchanged tree, never of a fixed one:
- the transition after a green run — instant;
make done-gateaftermake review-gate— pays the difference, not twice;- a merge commit whose tree equals the branch tip — a fast-forward, or a
merge into a master that moved nothing else —
REV=<merge sha>finds every stamp already there; - a stage marked green that you want to see run anyway:
<gate>.sh --run --fresh(no make target).
Where it runs, and what that costs
The subject is a commit — HEAD, or REV=<sha>. The run happens in the
checkout only when it is clean and at that commit; otherwise in a one-shot
scratch checkout of the commit under the shared git dir, with a venv of its
own. Measured 2026-09-03 with a warm uv cache: +2.5 s on lint, +5 s on a
35 s unit. A dirty desk therefore never taints a marker and never blocks
one, and REV= is not a cost worth working around.
Either way the stages run under that checkout's .venv, and the gate leads
PATH with it: a bare python3 or pytest a test spawns is the same
interpreter. Nothing to activate or prefix in your shell first.
Pushing master
scripts/run-e2e-gate.sh # out of band, BEFORE the push
git push origin master # the hook then passes instantly
Status is printed, not documented
Do not come back to this file to find out what happened. Read what the gate said. Every branch narrates itself with a reason and a remedy — the refusals, the green, and every quiet pass-through alike. The one thing it never prints is the transcript, which lands beside the card; glob it, never hand-build the name:
<tasks_dir>/<ID>/.checks/*done-gate.sh*.log
Why the push gate runs out of band
Running the suite inside the pre-push hook cannot work. Git opens the SSH
connection for ref-advertisement before running the hook, then runs the hook,
then sends the pack. GitHub closes the idle connection after ~30s, so a hook
taking minutes dies with Connection closed by remote host → SIGPIPE (exit 141)
and the pack is never sent.
Do not re-attempt: client-side ServerAliveInterval at 60 and 15 did not
fix it, and the 2022 community keepalive workaround no longer works (paid for twice).
No bypass
There is no AI_HATS_E2E_SKIP, no --ack and no knob. git push --no-verify
and forging a marker are deliberate local acts by the trusted maintainer, never
an accidental skip.
Master's own CI verdict is not a card's to pass or to wave through, so no card
gate asks for it. The push road prints it before a push to master
(scripts/check_master_ci.py --notice, never blocking: the push is how master
gets fixed), and bash scripts/gates.sh master-ci asks by hand — green only on
a completed green run; a run still going or a verdict it cannot read is unknown,
and unknown is not green. If a red is not yours, the move is a classification,
not a request for a flag: skill red-attribution.
04 同仓库更多 Skills
tool-evaluation-protocol
Time-bounded protocol for evaluating new tools, libraries, or methodologies before adoption. Use when evaluating a candidate dependency for the project, researching a methodology (e.g. ML/LLM frameworks, testing approaches), or following up on a hype-driven recommendation.
systematic-debugging
4-phase bug-fix protocol (evidence, pattern, hypothesis, verify). Use when investigating any error, failure, or unexpected behavior, handling bug reports from users or monitoring, or diagnosing test failures with non-obvious causes.
skill-template
Canonical template and validation guide for ai-hats skills. Use when creating a new skill, reviewing an existing one for structural compliance, or deciding which pattern fits a new behavior.
skill-optimization
Audit and refactor library components to eliminate redundancy and staleness. Use during periodic library maintenance, after adding multiple new skills or rules, when a role's token budget is too high, or after retrospectives identify component issues.
silent-fallback-gate
PreToolUse hook — refuses an Edit or Write that adds an inert broad except. Use when a Python edit was blocked for a silent fallback.
retro-to-framework
Convert an observation about agent behavior — from a retrospective or from a single incident — into a framework-level change (hook, rule, skill, skill update). Use after a retrospective identifies problems that are not project-specific, after an incident where an agent misapplied a component it had read, when CLAUDE.md band-aids accumulate (more than 3 per-project rules that could be generic), or when the same problem recurs across multiple projects.