typescript-test-standards

<objective> The TypeScript-specific test standards every TypeScript skill enforces. </objective>

<repo_local_overlay> When another skill loads this reference inside a repository, it must also check for spx/local/typescript-tests.md at the repository root. Read that file after this reference if it exists and apply it as repo-local routing to the product's governing specs and decisions. A local overlay supplements skill behavior; it does not declare product truth. </repo_local_overlay>

<core_model> Every TypeScript test file name encodes three independent axes:

Axis	Tokens	Meaning
Evidence	scenario, mapping, conformance, property, compliance	What kind of evidence the test provides
Level	l1, l2, l3	How painful the test is to run
Runner	optional token such as playwright	Which non-default runner executes the file

Evidence, level, and runner are orthogonal:

• A Playwright test can be l2 or l3
• A filesystem test can be l1 when it uses cheap temp dirs
• A scenario test can run at any level
• A runner token appears only when the runner is not the default

</core_model>

<file_naming> Use this canonical TypeScript pattern:

<subject>.<evidence>.<level>[.<runner>].test.ts

Examples:

Purpose	File
Cheap behavior scenario	config-loader.scenario.l1.test.ts
Deterministic input-output mapping	route-parser.mapping.l1.test.ts
Local browser flow through Playwright	checkout.scenario.l2.playwright.test.ts
Live webhook contract	stripe-webhook.conformance.l3.test.ts
Safety boundary	pii-redaction.compliance.l1.test.ts
Generated invariant	slug-roundtrip.property.l1.test.ts

Do not use legacy file suffixes such as .unit.test.ts, .integration.test.ts, .e2e.test.ts, or .spec.ts as the signal for evidence, level, or runner. </file_naming>

<level_tooling> Choose the level from execution pain and dependency availability:

Level	Infrastructure	Default runner	Typical runtime
l1	Node.js stdlib, temp dirs, repo-required dev tools	Vitest	milliseconds
l2	Docker, browsers, dev servers, product binaries	Vitest	seconds
l3	Remote services, credentials, shared environments	Vitest	seconds/minutes

Use playwright as the runner token when Playwright is the non-default runner:

• browser-menu.scenario.l2.playwright.test.ts for a local browser flow
• production-login.scenario.l3.playwright.test.ts for a credentialed remote flow

Keep runner configuration aligned with the filename pattern:

// vitest.config.ts
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    include: ["spx/**/*.test.ts"],
    exclude: ["**/*.playwright.test.ts"],
  },
});

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  testDir: "./spx",
  testMatch: "**/*.playwright.test.ts",
});

</level_tooling>

<router_mapping> After /test chooses the evidence and level, implement it with these TypeScript patterns:

Router Decision	TypeScript implementation
Stage 2 -> l1	Vitest, test harnesses, typed factories, temp dirs
Stage 2 -> l2	Vitest or Playwright with locally available real APIs
Stage 2 -> l3	Vitest or Playwright with credentials or remote real APIs
Stage 3A: pure computation	Direct tests of typed pure functions
Stage 3B: extract pure part	Pure function at l1, boundary at outer level
Stage 5 exception 1: failure simulation	Interface implementation that throws/errors
Stage 5 exception 2: interaction protocols	Spy function or class with typed call recording
Stage 5 exception 3: time/concurrency	Injected clock or vi.useFakeTimers()
Stage 5 exception 4: safety	Function or class that records intent without side effects
Stage 5 exception 5: combinatorial cost	Configurable fake with real-shaped behavior
Stage 5 exception 6: observability	Spy that captures hidden boundary details
Stage 5 exception 7: contract probes	Stub validated against the contract schema

</router_mapping>

<l1_patterns> Pure computation and filesystem tests at l1 use direct function calls, typed factories, and Node.js temp dirs. Read ${CLAUDE_SKILL_DIR}/references/l1-patterns.md for full examples of each pattern. </l1_patterns>

<exception_implementations> When /test routes to Stage 5, implement the exception in TypeScript using the pattern from <router_mapping>. Read ${CLAUDE_SKILL_DIR}/references/exception-implementations.md for full TypeScript examples of exceptions 1–4 and 6.

Exception 5 (combinatorial cost: configurable fake with real-shaped behavior) is listed in <router_mapping> but does not yet have a TypeScript example. </exception_implementations>

<l2_patterns> Use typed harness factories when tests require real infrastructure (Docker, browsers, product binaries). Verify the binary is available at harness construction time, not inside each test. Throw with an installation hint so the developer knows immediately what is missing. Read ${CLAUDE_SKILL_DIR}/references/l2-patterns.md for the full harness factory pattern. </l2_patterns>

<l3_patterns> l3 tests require real credentials or remote services.

Credential policy: always fail loudly. Skipping is forbidden.

it.skip, it.skipIf, test.skip, and any other skip mechanism are forbidden for credentialed tests. A skipped test silently passes the suite while hiding missing evidence. When credentials are absent, the test must throw with a clear diagnostic.

/**
 * l3 tests require these environment variables:
 *
 *   LHCI_SERVER_URL  - LHCI server base URL
 *   LHCI_TOKEN       - build token
 *
 * Where to find: 1Password "Engineering/Test Credentials"
 * Setup: cp .env.test.example .env.test and fill in values
 */

import { createLhciUploadRequest } from "@testing/harnesses/lhci/upload-request";

describe("LHCI", () => {
  it("uploads audit results to server", async () => {
    await expect(uploadAuditResults(createLhciUploadRequest())).resolves.toMatchObject({ success: true });
  });
});

</l3_patterns>

<dependency_injection> Tests verify behavior through real code paths. Avoid framework-level replacement of the dependency under test.

Forbidden patterns:

• vi.mock(...) replacing the module that should provide evidence
• jest.mock(...) replacing the module that should provide evidence
• vi.spyOn(...).mockReturnValue(...) replacing behavior that the test claims to verify

Allowed doubles are explicit objects or classes passed through dependency injection and mapped to a /test Stage 5 exception, see <router_mapping> above. Define controlled implementations and recording collaborators in @testing/harnesses/*; the linked test imports their observations and owns the assertion.

import { createPaymentRecordingContext } from "@testing/harnesses/payments";

test("records charge requests", async () => {
  const { processor, request, recordingGateway } = createPaymentRecordingContext(PaymentProcessor);

  await processor.charge(request);

  expect(recordingGateway.recordedCharges).toContainEqual(request);
});

</dependency_injection>

<predicate_and_oracle_litmus>

Apply every question in /test-evidence-standards <common_litmus_questions>, every per-assertion-type source-and-oracle rule in its <assertion_type_litmus>, every mutation in its <mutation_litmus>, and every field of its <assertion_design_record>. That shared set is the complete list; the items below render the ones whose form is TypeScript-specific and never replace or bound it.

• Invert the expect matcher. Only the linked test changes; no harness or collaborator code changes.
• Read the test callback alone. Every pass/fail predicate is visible there.
• Trace each case to the spec scenario, complete source-owned mapping, fast-check arbitrary, external conformance oracle, governing compliance rule, or inert whole-payload fixture.
• Trace each expected result to an oracle outside the production table, algorithm, parser, branch logic, or collaborator verdict method under test.
• Mutate the assertion-relevant production behavior. The test fails.
• Read each harness and factory. It returns observations, state, or handles — never a verdict, and never a *Succeeds, isValid, or wasCalledWith method.
• Read a failure message. It reports actual against expected at the expect site, not expect(helper()).toBe(true).
• Ask whether the same harness could serve a test claiming the opposite about the same observation. It can when the predicate is test-owned.

</predicate_and_oracle_litmus>

<property_based_testing> Property assertions about parsers, serializers, mathematical operations, or invariant-preserving algorithms require fast-check and a meaningful property.

Code type	Required property	Pattern
Parsers	parse(format(x)) == x	assertProperty(parserRoundtripProperty())
Serialization	decode(encode(x)) == x	assertProperty(serializationRoundtripProperty())
Mathematical operations	algebraic laws	assertProperty(algebraicLawProperty())
Complex algorithms	invariant preservation	assertProperty(invariantPreservationProperty())

An assertion that only checks "does not throw" is insufficient. The property must fail when the requirement is broken. Route property assertions through a harness or wrapper that owns seed selection, run count, and replay diagnostics; failure output must include the seed and replay path. </property_based_testing>

<test_data_policy> <existing_code_stance> When writing tests for existing code, evaluate whether the source contract needs to change before writing around it. Maintainable evidence may require architecture improvements first: extract pure functions, inject side-effect dependencies, expose source-owned registries, split command boundaries from domain logic, or add typed constructors that make the behavior observable without copying internals.

If a test can only be written by copying source literals, pinning arbitrary example objects, mocking the behavior under test, or storing inert fixture strings, stop and improve the source contract first. </existing_code_stance> <data_ownership_decision> Use this decision table for every assertion in the spec file. Every test file can only cover assertions of the same assertion type: mapping goes in one file, compliance goes in another file. See <core_model> above.

Executed TypeScript test functions and callbacks own every predicate and assertion API call. A const, let, var, framework fixture parameter, or property-generated parameter may receive an actual result, source-owned contract, generated value, harness observation, callback input, resource handle, or fixture path when it introduces no data or policy. Bindings that choose cases, expected results, configuration, setup policy, generator domains, fixture contents, or verdict rules belong in their semantic owner.

1. Data that the source imports or should import ALWAYS verify that the code under test imports routes, selectors, ids, feature flags, registry names, and all other public constants from the module that owns them.

ALWAYS verify that the code under test imports standard values like HTTP status codes from the canonical source of the runtime (Node) or framework (e.g., React or Next.js).

Reject test-local constants that rename runtime-owned values. Accept production-owned domain constants that add source-owned meaning.

2. Data that the code under test owns or should own

Most code under test Claude encounters will hardcode the same numbers and string literals several times in the source code.

This means the code is not testable in a maintainable way because any change to the source file will invalidate the test and lead to churn and extra work.

ALWAYS refactor source-owned domain constants, protocol tokens, command names, status values, rule ids, message ids, option labels, and other closed vocabulary into a semantically named registry, tuple, constructor, schema, or other suitable data structure.

Use a source-owned registry or tuple for status tokens, command names, rule ids, message ids, option labels, and other closed vocabulary. Derive unions, schemas, and arrays from that one declaration.

ALWAYS make sure that these data structures reflect semantically what they represent.

Reject test-owned copies of source vocabulary.

3. Data that only the test needs

Tests may need representative input domains that production code does not own. Those domains are still not a license to create shared constant bags.

Variable domains move to generators; runner settings, seed policy, and replay diagnostics move to harnesses. Curated eval cases may live in eval case data when generating a JSONL case set would be wasteful and not tractable.

</data_ownership_decision>

<valid_test_data_sources>

<source_imports>

Import source-owned values and source-owned singleton constructors directly from the module that owns them. Do not wrap a source-owned singleton in fc.constant(...) or a generator merely to satisfy a "generator" rule.

There are no valid shared test-owned constant bags for protocol values, domain values, expected outputs, or edge-case sets. A value that belongs to production is imported from production. A value that varies is generated. A real payload whose whole shape matters is read as an inert fixture. Modules that collect "typical" or "edge" examples are literal laundering.

Reject test-owned protocol literals and shared constant bags:

// REJECTED: the test copies a source protocol shape
const ABSENT_CONFIG_READ = { kind: "absent" } as const;

expect(isAbsentConfigReadResult(ABSENT_CONFIG_READ)).toBe(true);

// REJECTED: shared example bags preserve hand-picked values
export const EXAMPLE_SOURCE_PATHS = ["src/index.ts", "src/config.ts"];
export const BOUNDARY_SOURCE_PATHS = ["", "../outside.ts"];

Use the source-owned constructor directly:

import { createAbsentConfigReadResult, isAbsentConfigReadResult } from "@/config/read-result";

expect(isAbsentConfigReadResult(createAbsentConfigReadResult())).toBe(true);

Valid direct imports include:

• Public registries, command tokens, status values, rule names, and message ids
• Typed constructors or factory functions that production code owns
• External standard values from the runtime, framework, or protocol package

</source_imports>

<generators>

Use generators for input domains that vary, compose, shrink, or explore more than one meaningful value. A generator is a pure function - it emits values, holds no state, and has no side effects. Use fast-check for randomized scalars and fc.Arbitrary for structured domain values.

// testing/generators/{domain}.ts
export function arbitraryNodePath(config: Config): fc.Arbitrary<string>;
export function arbitraryDecisionPath(config: Config): fc.Arbitrary<string>;
export function arbitraryProductTree(config: Config): fc.Arbitrary<ProductTreeFixture>;

Use arbitrary*() generator functions for tests that should search a domain with assertProperty. Use createGenerated*() functions only as single-sample wrappers around the same arbitrary when a full property loop would make local infrastructure evidence too expensive.

Reject generators that only rename constants:

// REJECTED: this hides a singleton behind fast-check
export function arbitraryAbsentConfig(): fc.Arbitrary<ConfigReadResult> {
  return fc.constant({ kind: CONFIG_FILE_READ_KIND.ABSENT });
}

Use the source-owned constructor directly instead:

import { createAbsentConfigReadResult } from "@/config/read-result";

A constant branch inside a larger arbitrary is valid only when it expands boundary coverage and source-owned values still come from source APIs:

import { createEndOfInputToken, createIdentifierToken, type Token } from "@/lexer/tokens";

export function arbitraryToken(): fc.Arbitrary<Token> {
  return fc.oneof(
    fc.string({ minLength: 1 }).map(createIdentifierToken),
    fc.constant(createEndOfInputToken()),
  );
}

</generators>

<harnesses>

Use harnesses for tests that interact with external systems — filesystems, browsers, APIs, Docker. A harness manages setup and teardown of the external resource; it is not self-contained.

// testing/harnesses/{domain}.ts

// Filesystem harness: manages a temp product directory for product-tree operations
export async function withTestEnv(
  config: Config,
  callback: (env: ProductTreeEnv) => Promise<void>,
): Promise<void>;

// Browser harness: wraps Playwright with page lifecycle management
export function withPlaywright(testFn: BasePlaywrightTest): PlaywrightHarnessTest {
  return testFn.extend<PlaywrightHarnessFixtures>({
    playwrightHarness: async ({ page }, provideFixture) => {
      await provideFixture(createPlaywrightHarness(page));
    },
  });
}

export const test = withPlaywright(baseTest);

</harnesses>

<fixtures>

Fixtures are inert files. Use them for real-world data the code under test would encounter: a captured JSONL from a chat session, a saved API response payload, a document the parser must handle, a sample TypeScript source file for a linter, or a Product Tree copied into a temp directory.

Executed tests may read fixtures from disk, copy them into temp projects, or pass their paths to the code or program under test. Executed tests must never import fixture modules, require fixture files, or consume fixture exports. A fixture file can have a .ts extension when the test is verifying linters, parsers, pre-commit hooks, or scanners; it remains input data, not a test dependency.

Strings and numbers are never valid fixtures by themselves. A string literal that represents a domain value belongs in the production module or a generator, not a static file or a test-file constant.

</fixtures>

• Keep descriptive test titles and assertion diagnostics inline; they are the only valid string literals in a test file.
• Use aliases such as @testing/* for shared test infrastructure
• Keep spx/<node>/tests/ limited to typed assertion files; local test-adjacent modules that carry harness, generator, or fixture behavior are rejected

</valid_test_data_sources>

</test_data_policy>

<test_infrastructure> Test infrastructure (harnesses, generators, inert fixtures) is production code governed by the product's test-infrastructure PDR. In TypeScript, it lives in a testing/ directory at the product root, path-mapped to @testing/ via tsconfig.json and vitest.config.ts. The PDR mandates the three category subdirectories: @testing/harnesses/*, @testing/generators/*, @testing/fixtures/*. This testing/ directory is sibling to product code (src/ or product root), never inside spx/ and never inside any tests/ directory. Each test-infrastructure artifact is governed by the naturally placed spec node whose assertions, child specs, or category contract cover the artifact's behavior, policy, lifecycle, or reusable semantics.

testing/
+-- harnesses/
|   +-- index.ts
|   +-- hugo.ts          # Hugo build harness
|   +-- postgres.ts      # PostgreSQL harness
|   +-- factories.ts     # Harness factories
+-- generators/
|   +-- paths.ts         # Variable input domains
+-- fixtures/
    +-- sample-rule.ts   # Inert source file read by linter tests

Configure the @testing alias in tsconfig.json and vitest.config.ts:

import { createAuditResult } from "@testing/harnesses/factories";
import { createHugoHarness } from "@testing/harnesses/hugo";

Reusable setup, context management, generated domains, and inert fixtures live under testing/. Product Tree tests/ directories contain typed assertion files only.

</test_infrastructure>

<script_testing> Committed scripts/ entrypoints get thin tests:

• Argument parsing through the repository's canonical parser
• Dispatch into the imported orchestrator
• Exit-code mapping and observable terminal output

The orchestrator carries the main behavioral evidence. Script files should stay small and route to tested modules. </script_testing>

<anti_patterns> Reject or rewrite these patterns:

• Legacy file suffixes: .unit.test.ts, .integration.test.ts, .e2e.test.ts, .spec.ts
• Runner-level collapse: assuming Playwright means l3
• Level-evidence collapse: assuming scenario means high-cost execution
• Framework mocks replacing the dependency under test
• Property claims implemented only with examples
• Source-owned values copied into local constants
• Production modules created only to aggregate values for tests
• Deep relative imports into stable shared test infrastructure
• Importing fixture files into executed tests instead of reading or copying inert files
• Manual argument parsing in script tests when the repo has a canonical parser
• it.skip, it.skipIf, and test.skip on credentialed evidence -- use a credential harness wrapper such as withLhciCredentials() that throws instead

The cross-file literal-reuse check (check IDs L3/L4: literal in a test also present in src/, or duplicated across test files) is not an ESLint rule — it runs as spx validation literal because cross-file analysis doesn't fit ESLint's per-file execution model.

<playwright_request_context>

Playwright's { request } fixture uses its own APIRequestContext that does NOT share cookies with the BrowserContext. Cookies set via context.addCookies(...) do not reach { request }.

// WRONG: request fixture -- no cookie inheritance
test("API returns flag-gated payload", async ({ request }) => {
  await expect(request.get("/api/data").then((response) => response.json())).resolves.toContain(FLAGGED_ITEM); // fails
});

// RIGHT: context.request -- shares cookies with browser context
import { flagGatedApiPayload } from "@testing/harnesses/browser/api";

test("API returns flag-gated payload", async () => {
  await expect(flagGatedApiPayload()).resolves.toContain(FLAGGED_ITEM);
});

page.request also shares cookies with the browser context and works when a test already uses a page.

</playwright_request_context>

</anti_patterns>

<reference_guides>

File	When to read
${CLAUDE_SKILL_DIR}/references/l1-patterns.md	Writing pure function, typed factory, or temp dir tests
${CLAUDE_SKILL_DIR}/references/exception-implementations.md	Implementing a Stage 5 exception from /test
${CLAUDE_SKILL_DIR}/references/l2-patterns.md	Writing tests that require real infrastructure (Docker, etc.)

</reference_guides>

<success_criteria> TypeScript test guidance follows this standard when:

• /test determines the assertion type, execution level, and exception path before implementation
• Test filenames use <subject>.<evidence>.<level>[.<runner>].test.ts
• Runner configuration uses explicit runner tokens instead of .spec.ts
• Executed test functions and callbacks own every predicate and assertion API call
• Test-file bindings introduce no case data, expectation, configuration, setup policy, generator domain, fixture content, or verdict rule
• Doubles are passed through dependency injection and mapped to a Stage 5 exception
• Property assertions use meaningful fast-check properties through a seed-reporting wrapper
• Source-owned values come from the owning production module
• Shared test infrastructure lives under testing/ as spec-governed harnesses, generators, or inert fixtures

</success_criteria>