AGENT SKILL

debug-test

Master debugging failing tests: when tests fail, how to determine if it's a real bug or test issue, and how to fix test problems without breaking production code. Use when tests fail, test debugging, fixing broken tests, or when tests don't work as expected. Triggers especially when user says "test failed", "debug test", "fix test", "test not working", "why is this test failing".

适用职业开发工程师
使用场景测试与质量
License 许可
MIT
最近更新
2026-04-18
Stars
★ 0
Forks
0

01 前言 Front Matter

name
debug-test
category
coding
priority
P2
description
Master debugging failing tests: when tests fail, how to determine if it's a real bug or test issue, and how to fix test problems without breaking production code. Use when tests fail, test debugging, fixing broken tests, or when tests don't work as expected. Triggers especially when user says "test failed", "debug test", "fix test", "test not working", "why is this test failing".
trigger_patterns
[ "test failed", "debug test", "fix test", "test not working", "why is this test failing", "test debugging" ]

03 SKILL.md 正文

Debug Test — Test Failure Analysis Framework

Overview

Debug-test is a behavioral specification skill that encapsulates the methodology for systematically debugging failing tests to determine if the failure is a real bug or a test problem.

This skill embodies the expertise of developers who've spent hours chasing phantom test bugs and know the discipline of separating "my code is broken" from "my test is wrong."

Core Philosophy

The Debugging First Principle

"When a test fails, your first question should be: 'Is my test testing what I think it's testing?' The answer is 'not always' more often than you'd think."

Test Failure Types

┌─────────────────────────────────────────────────────────────┐
│                    TEST FAILURE CATEGORIES                    │
│                                                               │
│  REAL BUG IN CODE:          TEST PROBLEM:                     │
│  ─────────────────          ─────────────                      │
│  • Code doesn't work       • Test checks wrong thing          │
│  • Logic is wrong           • Test has wrong assertion        │
│  • Edge case not handled    • Test data is wrong              │
│  • Type error               • Test is flaky                  │
│                                                               │
│  BOTH:                             NEITHER:                     │
│  • Code was right, test was vague      • Environment issue    │
│  • Both code and test wrong           • Tooling issue         │
└─────────────────────────────────────────────────────────────┘

Decision Framework

The Test Failure Diagnosis Tree

                    ┌─ Does test pass locally?
                    │   └─ NO → Continue
                    │       └─ YES → Check CI environment
                    │
                    └─ Is test flaky?
                        └─ YES → Flaky test pattern
                        └─ NO → Continue
                                │
                                └─ Does the failing assertion make sense?
                                    └─ NO → Test problem
                                    └─ YES → Continue
                                            │
                                            └─ Is the code logic correct?
                                                └─ YES → Test problem
                                                └─ NO → Real bug

Behavioral Specification

Phase 1: Initial Assessment

1.1 Gather Information

## Test Failure Report

| Field | Value |
|-------|-------|
| Test name | should_return_user_when_id_exists |
| File | users.test.ts |
| Line | 42 |
| Error type | expect(received).toBe(expected) |
| Expected | { id: "123", name: "Alice" } |
| Received | { id: "123", name: "Alice" } |

Note: Objects look same but test fails...

1.2 First Checks

// Check 1: Are objects actually equal?
// Deep equality vs reference equality
expect(user).toBe(expectedUser); // Reference
expect(user).toEqual(expectedUser); // Deep (usually what you want)

// Check 2: Is there a type coercion issue?
// "1" vs 1
// true vs "true"
// null vs undefined

// Check 3: Are you comparing what you think?
console.log("Expected:", JSON.stringify(expected, null, 2));
console.log("Received:", JSON.stringify(received, null, 2));

Phase 2: Isolate the Problem

2.1 Simplify the Test

// ORIGINAL (complex)
it("should process complex order correctly", () => {
  const order = createComplexOrder({ ... });
  const result = processOrder(order, options, context);
  expect(result.status).toBe("shipped");
  expect(result.trackingNumber).toMatch(/^TRACK-/);
  // More assertions...
});

// SIMPLIFIED (isolate)
it("should process order", () => {
  const order = { status: "pending" }; // Minimal
  const result = processOrder(order);
  expect(result.status).toBe("processing"); // Test one thing
});

2.2 Add Diagnostic Logging

it("should calculate total correctly", () => {
  const items = [
    { price: 10, quantity: 2 },
    { price: 5, quantity: 3 },
  ];

  console.log("Input items:", items);
  const total = calculateTotal(items);
  console.log("Calculated total:", total);
  console.log("Expected:", 35); // 10*2 + 5*3 = 35

  expect(total).toBe(35);
});

2.3 Test in Isolation

// BEFORE: Full system test
it("should work with database", async () => {
  const result = await fullIntegrationTest();
  // Too many variables
});

// AFTER: Unit test
it("should calculate total", () => {
  const calculator = new OrderCalculator(mockLogger);
  const result = calculator.calculateTotal(items);
  // Isolated, clear failure point
});

Phase 3: Common Test Problems

3.1 Wrong Assertion Type

// ❌ PROBLEM: Checking reference, not value
const user1 = { name: "Alice" };
const user2 = { name: "Alice" };
expect(user1).toBe(user2); // Fails! Different objects

// ✅ FIX: Check value
expect(user1).toEqual(user2); // Passes! Same values

// ❌ PROBLEM: toBe for objects
expect({ a: 1 }).toBe({ a: 1 }); // Fails!

// ✅ FIX: toEqual for objects
expect({ a: 1 }).toEqual({ a: 1 }); // Passes!

3.2 Async Issues

// ❌ PROBLEM: Not waiting for async
it("should update user", async () => {
  await updateUser({ id: "1", name: "Bob" });
  const user = getUser("1");
  expect(user.name).toBe("Bob"); // Might pass/fail randomly
});

// ✅ FIX: Properly await
it("should update user", async () => {
  await updateUser({ id: "1", name: "Bob" });
  const user = await getUser("1"); // Await here too!
  expect(user.name).toBe("Bob");
});

// ❌ PROBLEM: Forgotten async
it("should return promise", () => {
  const promise = fetchData(); // Returns promise
  expect(promise).resolves.toEqual(data); // Missing async/await
});

// ✅ FIX: Handle promise correctly
it("should return promise", async () => {
  const promise = fetchData();
  await expect(promise).resolves.toEqual(data);
});

3.3 Mock Configuration Errors

// ❌ PROBLEM: Mock not configured correctly
const mockRepo = {
  findById: vi.fn(), // No return value!
};
// Test fails because findById returns undefined

// ✅ FIX: Configure mock properly
const mockRepo = {
  findById: vi.fn().mockResolvedValue(testUser),
};

// ❌ PROBLEM: Mock not reset between tests
beforeEach(() => {
  // Forgot to clear mocks!
});
it("calls repo", () => {
  service.getUser("1");
  expect(mockRepo.findById).toHaveBeenCalled(); // Passes from previous test?
});

// ✅ FIX: Always reset mocks
beforeEach(() => {
  vi.clearAllMocks();
});

3.4 Test Data Issues

// ❌ PROBLEM: Test data not matching expectations
const user = {
  name: "Alice",
  email: null, // What test expects?
};
// Test expects email to be defined

// ✅ FIX: Ensure test data matches scenario
const user = {
  name: "Alice",
  email: "alice@example.com",
};

Phase 4: Real Bug vs Test Bug

4.1 Indicators of Real Bug

## Real Bug Signals

□ Code logic can be manually verified wrong
□ Edge case not handled in code
□ Type mismatch causing issues
□ Expected behavior not implemented
□ Integration with external system broken

Example:
Code: if (user.isActive && user.isAdmin) // Should be OR
Test: Expect admin panel to show for admin user
Failure: isActive=false, isAdmin=true → fails
Conclusion: Real bug (wrong operator)

4.2 Indicators of Test Bug

## Test Bug Signals

□ Test assertions don't match spec
□ Test data doesn't match scenario
□ Wrong assertion type (toBe vs toEqual)
□ Missing async/await
□ Mock not configured correctly
□ Test checks implementation, not behavior

Example:
Code: Returns user object correctly
Test: expect(user).toBe({ name: "Alice" }) // Reference check!
Failure: Same values but different object reference
Conclusion: Test bug (wrong assertion)

4.3 Decision Framework

┌─────────────────────────────────────────────────────────────┐
│                   REAL BUG vs TEST BUG                        │
│                                                               │
│  1. Can you manually verify the expected behavior?            │
│     └─ NO → Test problem or unclear spec                     │
│     └─ YES → Continue                                        │
│                                                               │
│  2. Does changing the test fix the problem?                  │
│     └─ YES, but test seems correct → Test problem            │
│     └─ NO → Real bug in code                                │
│                                                               │
│  3. Does the code match requirements?                         │
│     └─ NO → Real bug                                         │
│     └─ YES, but test fails → Test problem                    │
└─────────────────────────────────────────────────────────────┘

Common Scenarios

Scenario 1: "toBe vs toEqual"

// FAILING TEST
it("should create user", () => {
  const user = createUser({ name: "Alice" });
  expect(user).toBe({ name: "Alice" }); // Reference check!
});

// WHY IT FAILS:
// toBe checks reference (are they the SAME object?)
// user is a new object created by createUser()
// { name: "Alice" } is a different object in memory

// FIX
it("should create user", () => {
  const user = createUser({ name: "Alice" });
  expect(user).toEqual({ name: "Alice" }); // Value check!
});

Scenario 2: "Mock not being called"

// FAILING TEST
it("should call logger", () => {
  const mockLogger = { info: vi.fn() };
  const service = new Service(mockLogger);

  service.doSomething();

  expect(mockLogger.info).toHaveBeenCalled(); // Fails!
});

// DEBUG
it("should call logger", () => {
  const mockLogger = { info: vi.fn() };
  const service = new Service(mockLogger);

  service.doSomething();

  console.log("Calls:", mockLogger.info.mock.calls); // []
  // Info was never called...

  // FIX: Check if the code actually calls logger.info
  // Maybe it calls logger.debug instead?
  // Or maybe doSomething doesn't actually log?
});

Scenario 3: "Flaky test that sometimes passes"

// FLAKY TEST
it("should return users", async () => {
  const users = await fetchUsers();
  expect(users.length).toBeGreaterThan(0);
});

// WHY FLAKY:
// fetchUsers() returns users from database
// Database might be empty in some test runs

// FIX: Ensure consistent state
beforeEach(async () => {
  await seedTestDatabase(); // Ensure known state
});

it("should return users", async () => {
  const users = await fetchUsers();
  expect(users.length).toBeGreaterThan(0); // Now reliable
});

Scenario 4: "Test passes but code is wrong"

// PASSING TEST (but code is buggy!)
it("should double numbers", () => {
  expect(double(2)).toBe(4);
  expect(double(3)).toBe(6);
});

function double(n) {
  return n + 1; // Bug! Should be n * 2
}

// WHY TEST PASSES:
// double(2) = 3 ≈ 4? No wait... actually 2+1=3, not 4
// Oh wait, it would fail...

// The point: Sometimes bugs cancel out
// Test has implicit bug that makes it pass despite code bug

// FIX: Better test coverage
it("should double numbers", () => {
  expect(double(2)).toBe(4);  // 2*2=4
  expect(double(0)).toBe(0);   // 0*2=0
  expect(double(-1)).toBe(-2); // -1*2=-2
});

The Test Debugging Checklist

TEST DEBUGGING CHECKLIST
========================

□ 1. Isolate the failing test
□ 2. Run only this test (not entire suite)
□ 3. Add diagnostic logging
□ 4. Verify expected vs actual values
□ 5. Check assertion type (toBe vs toEqual)
□ 6. Verify async/await is correct
□ 7. Check mock configuration
□ 8. Verify test data is correct
□ 9. Can you manually verify expected behavior?
□ 10. Is this a real bug or a test bug?

IF REAL BUG:
□ Create minimal reproduction
□ Fix the code
□ Add regression test
□ Verify test passes after fix

IF TEST BUG:
□ Fix the test
□ Verify test fails with buggy code
□ Verify test passes with fixed code
□ Consider: Is there a real bug the test missed?

Anti-Patterns

1. Ignoring Flaky Tests

// ❌ BAD: "It passes sometimes, good enough"
it.failing("should work", () => { ... });

// ✅ GOOD: Fix or mark explicitly
it("should work - KNOWN ISSUE: flaky on CI", () => {
  // File issue, fix later
  // Or fix the race condition
});

2. Changing Tests to Pass

// ❌ BAD: "Just change the test to match whatever code does"
expect(code).toBe(whateverCodeDoes); // Now always passes!

// ✅ GOOD: Verify code is correct, then update test if needed
// Or: If requirements changed, update both

3. Not Testing the Fix

// ❌ BAD: Fix code, don't verify test
// Code was fixed
git commit -m "Fix bug"
// Test still fails because it was wrong!

// ✅ GOOD: Verify test fails on buggy code, passes on fixed code

Key Principles

1. Trust Tests, But Verify

If a test fails, assume the test is correct. But verify before changing code.

2. Isolation First

Isolate the failing test. Don't run the whole suite while debugging.

3. Divide and Conquer

Comment out assertions one by one to find which specific check fails.

4. Manual Verification

Before fixing, manually verify what the code SHOULD do. This prevents fixing tests incorrectly.

5. Both Fail Together

When you find a bug, always ask: "Is there a test that should have caught this?"

Relationship to Other Skills

  • test-think — Determines what tests are needed
  • generate-mock — Mock issues often cause test failures
  • setup-test-data — Wrong test data is a common cause
  • debug-session — General debugging methodology
  • understand-error — Error messages hint at the problem

04 同仓库更多 Skills

ImaginerLabs
ImaginerLabs/skill-manager

meeting-minutes

Generate concise, actionable meeting minutes for internal meetings. Includes metadata, attendees, agenda, decisions, action items (owner + due date), and follow-up steps.

行政与办公人员Skill与智能体开发
★ 0
ImaginerLabs
ImaginerLabs/skill-manager

documentation-writer

Diátaxis Documentation Expert. An expert technical writer specializing in creating high-quality software documentation, guided by the principles and structure of the Diátaxis technical documentation authoring framework.

★ 0
ImaginerLabs
ImaginerLabs/skill-manager

doc-coauthoring

Guide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks.

内容创作者写作与文案Skill与智能体开发
★ 0
ImaginerLabs
ImaginerLabs/skill-manager

code-review-workflow

Executes comprehensive 4-phase code review: basic review, quality check, bundle analysis, and test coverage. Should be used when the user mentions comprehensive code review, checking code quality, or reviewing staged changes with multi-phase analysis. This is the comprehensive review workflow, distinct from focused skills like frontend-code-review, staged-code-review, or code-change-test-designer.

开发工程师代码审查测试与质量
★ 0
ImaginerLabs
ImaginerLabs/skill-manager

code-change-test-designer

Analyzes code changes and designs comprehensive test cases. Reads diffs, summarizes capability changes, and designs test cases covering main flows and edge cases. Should be used when the user mentions test coverage, designing tests for code changes, or analyzing diff/commit for testing purposes. Distinguished from code-review-workflow which provides comprehensive review, this skill focuses on test case design for code changes.

开发工程师设计师测试与质量视觉设计
★ 0
ImaginerLabs
ImaginerLabs/skill-manager

skill-creator

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

Skill与智能体开发
★ 0