write-vibe-tests

Write Vibe Tests

Use this skill to make Vibe architecture testable, not just well-shaped. Tests should protect observable behavior while allowing internals to move.

Core Principle

Test behavior through stable boundaries. Do not couple tests to private methods, internal call choreography, or temporary structure.

If a refactor changes no observable behavior but breaks many tests, the tests are probably coupled to implementation details.

Vibe Test Boundaries

Code area	Preferred test boundary
Core use cases and services	Public function/class API, typed events, model outputs, persisted state
Tools	Owning app-server capability contract, typed invocation/results, permissions and user-facing errors; BaseTool.invoke/run for retained BaseTool implementations
LLM/backend orchestration	Native Unified Runtime through serialized app-server RPCs, with deterministic fake completion/discovery services
Config	Pydantic model validation, layer merge outputs, migration results
Sessions	Runtime session operations, durability/recovery and app-server projections; JSONL/metadata only for the supported read-only historical importer
Terminal UI (vibe/cli-rust)	Rust unit tests for pure logic, client-e2e scenarios and goldens for rendering and interaction (see vibe/cli-rust/AGENTS.md)
ACP	ACP session updates and protocol-facing content, not core internals

Prefer Fakes Over Mocks

Prefer in-memory implementations and fake adapters that implement real contracts.

• Put reusable test doubles in tests/stubs/ and name them Fake*.
• Make fakes small and behavior-oriented.
• Mock only at hard process, network, time, or third-party boundaries when a fake would be more complex than the behavior under test.
• Avoid assertions like "method X was called with Y" unless the call itself is the observable contract.

Legacy Or Refactor Workflow

When code is hard to test:

1. Find the smallest seam: function boundary, constructor dependency, protocol/port, wrapper, composition root, feature flag, or module boundary.
2. Add characterization tests through the nearest public entry point.
3. Capture current observable behavior, even if awkward.
4. Refactor behind the seam in small steps.
5. Replace broad characterization checks with clearer behavior/spec tests as the design improves.

Prefer "make it testable" refactors first: isolate I/O, extract pure functions, introduce ports/adapters where useful, or move construction out of business logic.

Test Shape

• Python stack: pytest + pytest-asyncio + respx.
• Use descriptive test names; do not add test docstrings. Pytest displays docstrings instead of node IDs when present, which hurts.
• Arrange, act, and assert clearly, but optimize for readability over ceremony.
• Keep tests deterministic, fast, and explicit about failure.
• Never let a test depend on wall-clock timing (sleeps, pauses, time windows, mtime ticks). Timing-dependent tests flake on loaded CI runners.
• Use autouse fixtures from tests/conftest.py (config_dir, tmp_working_directory) for config/home/working-directory isolation.
• Mark async tests with @pytest.mark.asyncio.
• Mock outbound HTTP with respx.
• Use the narrowest relevant test first, then broaden when shared contracts are touched.
• Tests are exempt from the ANN and PLR ruff rules (see per-file-ignores).
• Changes to managed shell tools, managed shell backends, shell prompts, or shell dependencies require focused POSIX/common validation and passing Windows shell CI. If Windows shell CI is unavailable, the PR notes must include manual native Windows commands and results.
• After app-server session behavior, backend adapters, or shared contract-fixture changes, run the Unified backend contract suite (uv run pytest tests/app_server/backend_contract) and the affected app-server/ACP tests. Preserve original snapshot scenarios when adapting fixtures to the Unified Runtime.

Avoid

• Testing private methods as the primary coverage for behavior.
• Asserting intermediate internal state when user-visible output, emitted events, files, or return values can be asserted.
• Building mocks that mirror the implementation.
• Adding abstractions only to satisfy a test.
• Writing tests that require a specific internal file split or call order when the domain behavior is unchanged.

Verification

After Python test/code changes, run:

uv run ruff format .
uv run ruff check --fix .

Then run the targeted tests:

uv run pytest <test-path-or-node-id>

Run uv run pyright when signatures, models, protocols, or shared contracts changed.