test-audit
Write or review Windows regression tests for observable contracts, independent expectations, faithful fixtures, and distinct coverage without weakening native proof or architecture guards.
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 2.2k
- Forks
- 349
01 前言 Front Matter
- name
- test-audit
- description
- Write or review Windows regression tests for observable contracts, independent expectations, faithful fixtures, and distinct coverage without weakening native proof or architecture guards.
03 SKILL.md 正文
Windows test audit
Use when adding, changing, or reviewing tests. A review does not authorize test deletion or production refactoring.
Authoring gate
Answer these questions:
- What observable behavior or independent contract does it protect?
- What credible regression makes it fail, and does it fail for that reason rather than a broken fixture?
- Which production owner is exercised? Why does existing coverage miss this risk?
- Are the expected values independent of the implementation under test?
- Does another layer add a distinct transport, persistence, lifecycle, permission, or rendering risk, rather than repeat the same assertion?
- Is this the cheapest faithful boundary, or does it add a process, desktop, WSL, installer, Gateway, or native capability boot that a smaller owner test can cover?
Prefer an existing table, fixture, or owner regression over another process boot. Avoid mocks supplying the behavior being asserted, production helpers computing their own expected values, and test-only exports without a genuine production boundary.
Read regression authoring and use the existing fixture guide when the behavior crosses the application boundary.
Choose the production owner
Keep one primary regression at the strongest faithful owner boundary:
| Boundary | Primary project or lane |
|---|---|
| Gateway registry, credentials, connection lifecycle, migration admission/records and persisted Local AI runtime state | OpenClaw.Connection.Tests |
| Protocol, MCP, capabilities, command policy and shared models | OpenClaw.Shared.Tests |
| Tray policy, settings, onboarding helpers, installer source contracts and update checks/backends | OpenClaw.Tray.Tests |
| Setup engine, WSL installation and Local AI setup/recovery policy | OpenClaw.SetupEngine.Tests |
Shipped winnode arguments, JSON output and command documentation |
OpenClaw.WinNode.Cli.Tests |
| Functional UI element graphs, factories and hook state without mounted native controls | OpenClawTray.FunctionalUI.Tests |
| Mounted native WinUI controls, UI Automation, accessibility and focused input | OpenClaw.Tray.UITests |
| Clean install, upgrade, uninstall and Store migration runtime behavior | The applicable formal validation path or proof pool |
| Real process, Gateway-mediated setup/connect or MXC behavior | Integration, E2E or named proof lane |
Add a second layer only when it proves a different failure mode. Local MCP output does not prove Gateway pairing. A state snapshot is not a rendered UI assertion. Synthetic Gateway proof is not live provider, MXC, installer, Store, or production credential-resolution proof.
Reject low-value proof
Reject or rewrite tests that rely on:
- self-comparisons, copied inventories, subject-generated expected values, or the same helper on both sides of an assertion;
- implementation names, private call shape, or exact delegation counts unless that ownership/delegation boundary is itself the protected contract;
- a mock or fake that supplies the result, ordering, persistence receipt, or policy decision being asserted;
- another test at every crossed layer without a distinct transport, persistence, permission, lifecycle, packaging, or rendering risk;
- production flags or hooks that bypass owner logic, or exports/visibility added only to expose internal behavior without a genuine production boundary;
- a broad source grep that pins incidental identifiers instead of a concrete prohibited pattern or independent shipped inventory.
Negative controls must fail at the intended guard, not during fixture setup. Test names must describe the behavior actually exercised. If a production seam exists only for obsolete tests and has no production-boundary or architecture role, remove it with the tests when the task authorizes production changes; otherwise report it rather than expanding it. Do not reject a narrow adapter over process, WSL, OS, network, clock, or filesystem boundaries merely because tests supply its only alternate implementation. Existing test assembly visibility and ledger-recorded seams remain governed by their real contracts and retirement conditions.
Timing and determinism checklist
For native UI, loopback, process, and fixture tests, check each item against the timing rules:
- Readiness waits on an observable condition, not a delay, sleep, or fixed settle count.
- UI Automation elements are acquired inside the wait and used from that
capture, not looked up once with
!or a throwing helper. - Geometry is asserted only after bounds stop moving, with an explicit tolerance.
- A polling predicate keeps polling while ownership is unproven and fails fast only on positive foreign evidence.
- Fixture data sets every value the assertion depends on; ports and other inputs are deterministic.
- Hang guards are named, environment-overridable constants with headroom for a loaded runner. Latency-bounded work uses a dedicated thread, not the shared thread pool.
- A surviving flake is investigated as a possible product race before the wait is changed.
The TestTimingPatternGuard ratchet rejects new literal short hang guards,
fixed delays, and one-shot UI Automation lookups. Do not raise its baseline to
admit new code; use the named helper or a justified
// test-timing-exception: <reason> comment. A rename or move may carry an
existing baseline entry to the new file when the rule totals stay unchanged.
When a change removes occurrences, lower the baseline in the same change with
.\scripts\validate-test-timing-patterns.ps1 -LowerBaseline.
Preserve valuable contracts
- Keep independent protocol, CLI, storage, migration, defaults, packaging, update, security, architecture, generated-resource, and user-visible ordering contracts.
- Static does not mean disposable. Source-shape guards are appropriate for a
concrete prohibited pattern with the
ledger's retirement condition.
Installer source assertions remain distinct from clean install, upgrade, and
uninstall proof. The shipped
winnodecommand-to-registry contract must remain covered. - Use isolated settings, loopback listeners, owned processes, and existing
TempDirectory,EnvironmentScope,CliHarness, and Gateway fixture helpers. Fakes may provide external input or record calls, but must not implement the behavior under test. - Do not introduce arbitrary sleeps, blind polling, redundant process boots, or parallel native input tests on a shared desktop. Reuse suite-level fixtures when they preserve the same contract and isolation. Use a fresh process when startup, process-global state, ownership/termination, or an independent profile is the boundary under test.
- Do not make a failure green through weaker assertions, hidden skips, blind retries, broadened mocks, or an unrelated baseline update.
Audit workflow
Audit coherent owner batches, not deletion counts. Before deleting or consolidating proof:
- Read the full test, production owner, callers, fixture and relevant history.
- Identify the observable contract and every distinct remaining proof layer.
- Confirm the candidate is duplicate, implementation-coupled, obsolete, or unable to fail for the claimed regression.
- Preserve independent protocol, persistence, packaging, security, architecture, rendering and native-runtime contracts.
- Remove obsolete test-only production seams in the same authorized change.
Baseline failures may identify product bugs. Do not delete or weaken the test until the failure is classified.
Cost and validation
For a bug fix, demonstrate the intended pre-fix failure and post-fix pass, or report why baseline reproduction was unavailable. For a refactor, preserve the contract without pinning incidental implementation names.
Run focused owner tests, then the required validation. Respect explicit skip policies: an allowed ordinary CI skip is not complete custom proof. A seconds-long test must protect a boundary that a cheaper layer cannot. Measure wall time when cost or performance is the claim, but do not import JavaScript timing budgets, worker counts, or release-only rules into native Windows suites.
Report protected behavior, why the owner/layer is necessary, fixture limits, exact commands, execution counts, measured cost when relevant, and remaining proof gaps. Do not claim tests ran from an exit code alone.
Source and maintenance
Windows-owned adaptation of concepts from the MIT-licensed upstream test-audit workflow and test-authoring guide. Windows retains its mandatory local closeout, native proof, and legitimate source guards. This file does not inherit upstream publication permissions.
04 同仓库更多 Skills
network-handoff
Use before generating, changing, or reviewing networking, IPC, or cross-process handoffs, including requests, retries, deadlines, reconnect, process launch, configuration apply, cancellation, and cleanup. Establish confirmation, identity, authority, reconciliation, and producer compatibility before implementation.
retrigger-flaky-ci
Triage failed CI on a pull request or the active branch, decide whether the failures come from its changes, and rerun only failures that are unrelated. Use when asked to retrigger, rerun, or kick CI on a PR with failing checks.
technical-documentation
Write, review, or restructure Windows node documentation and agent instructions using current source, task-focused explanations, and explicit fact preservation.
pr-simplify
Simplify your own Windows-node change to its smallest complete fix while preserving distinct regressions, platform contracts, and required proof. Use after implementation before PR preparation, after large repairs, or when asked to shrink a change or remove slop.
openclaw-testing
Choose proportional OpenClaw Windows node tests, builds, packaging checks, and runtime proof, then diagnose failures without weakening required gates.
openclaw-proof-validation
Plan and collect OpenClaw Windows validation/proof: tests, rubber-duck review, UI evidence, MCP output, and gateway runtime proof.