AGENT SKILL

ln-514-test-log-analyzer

Analyzes application logs: classifies errors, checks log quality/format, maps stack traces to source, recommends fixes.

使用场景测试与质量
License 许可
MIT
最近更新
2026-05-02
Stars
★ 3
Forks
1

01 前言 Front Matter

name
ln-514-test-log-analyzer
license
MIT
description
Analyzes application logs: classifies errors, checks log quality/format, maps stack traces to source, recommends fixes.

02 资源文件 11

03 SKILL.md 正文

Paths: File paths (shared/, references/, ../ln-*) are relative to skills repo root. If not found at CWD, locate this SKILL.md directory and go up one level for repo root.

Test Log Analyzer

Two-layer analysis of application logs. Python script handles collection and quantitative analysis; AI handles classification, quality assessment, and fix recommendations.

Inputs

No required inputs. Runs in current project directory, auto-detects log sources.

Optional args — caller instructions (natural language): time window, expected errors, test context. Example: "review logs for last 30min, auth 401 errors expected from negative tests".

Purpose & Scope

  • Analyze application logs (after test runs, during development, or on demand)
  • Classify errors into 4 categories: Real Bug, Test Artifact, Expected Behavior, Operational Warning
  • Assess log quality: noisiness, completeness, level correctness, format, structured logging
  • Map stack traces to source files; provide fix recommendations
  • Report findings for quality verdict (only Real Bugs block)
  • No status changes or task creation — report only

When to Use

  • Analyze application logs in any project (default: last 1h)
  • After test runs to classify errors and assess log quality
  • Can be invoked with context instructions: Skill(skill: "ln-514-test-log-analyzer", args: "review last 30min, 401 errors expected")

Workflow

Phase 0: Parse Instructions

If args provided — extract: time window (default: 1h), expected errors list, test context. If no args — use defaults (last 1h, no expected errors).

Phase 1: Log Source Detection and Script Execution

MANDATORY READ: Load docs/project/infrastructure.md, docs/project/runbook.md

  1. Check if scripts/analyze_test_logs.py exists in target project. If missing, copy from references/analyze_test_logs.py.
  2. Detect log source mode (auto-detection priority: docker → file → loki):
Mode Detection Source
docker docker compose ps returns running containers docker compose logs --since {window}
file .log files exist, or tests/manual/results/ has output File paths from infrastructure.md or *.log glob
loki LOKI_URL env var or tools_config.md observability section Loki HTTP query_range API
  1. Run script: python scripts/analyze_test_logs.py --mode {detected} [options]
  2. If no log sources found → return NO_LOG_SOURCES status, skip to Phase 5.

Phase 2: 4-Category Error Classification

Classify each error group from script JSON output:

Category Action Criteria
Real Bug Fix Unexpected crash, data loss, broken pipeline
Test Artifact Skip From test scripts, deliberate error-path validation
Expected Behavior Skip Rate limiting, input validation, auth failures from invalid tokens
Operational Warning Monitor Clock drift, resource pressure, temporary unavailability

Test artifact detection heuristics:

  • Test name contains: invalid, error, fail, reject, unauthorized, forbidden, not_found, bad_request, timeout
  • Test asserts non-2xx status codes (4xx, 5xx)
  • Test uses pytest.raises, expect(...).rejects, assertThrows, should.throw
  • Errors correlate with test execution timestamps from regression test output
  • Patterns matching tests/manual/ scripts

Error taxonomy per references/error_taxonomy.md (9 categories: CRASH, TIMEOUT, AUTH, DB, NETWORK, VALIDATION, CONFIG, RESOURCE, DEPRECATION).

Phase 3: Log Quality Assessment

MANDATORY READ: Load references/error_taxonomy.md (per-level criteria table + level correctness reference)

Step 1: Detect configured log level. Check in order:

  1. LOG_LEVEL / LOGLEVEL env var (.env, docker-compose.yml, infrastructure.md)
  2. Framework config: Python logging.conf / Django LOGGING / Node LOG_LEVEL
  3. Default: assume INFO if not detected

Configured level determines WHICH levels appear in logs, but each level has its own noise threshold regardless.

Step 2: Assess 6 quality dimensions:

Dimension What to Check Signal
Noisiness Per-level noise thresholds from error_taxonomy.md section 4: TRACE (zero in prod), DEBUG (>50% monopoly), INFO (>30%), WARNING (>1% of total), ERROR (>0.1% of total) NOISY: {level} template "{msg}" at {ratio}%
Completeness & Traceability Critical operations missing log entries + traceability gaps (see table below) MISSING: No log for {operation} / TRACEABILITY_GAP: {type} in {file}:{line}
Level correctness Per-level criteria from error_taxonomy.md section 4: content, anti-patterns, library rule WRONG_LEVEL: should be {level}
Structured logging Missing trace_id/request_id/user context; unstructured plaintext UNSTRUCTURED: lacks {field}
Sensitivity PII/secrets/tokens/passwords in log messages SENSITIVE: {type} exposure
Context richness Errors without actionable context (order_id, user_id, operation) LOW_CONTEXT: lacks context

Traceability gap detection — scan source code for operations without INFO-level logging:

Operation Type Expected Log Where to Add
Incoming request handling Request received + response status Entry/exit of route handler
External API call Request sent + response status + duration Before/after HTTP client call
DB write (INSERT/UPDATE/DELETE) Operation + affected entity + count Before/after ORM/query call
Auth decision Result (allow/deny) + reason After auth check
State transition Old state → new state + trigger At transition point
Background job Start + complete/fail + duration Entry/exit of job handler
File/resource operation Open/close + path + size At I/O operation

Log Format Quality (10-criterion checklist per references/log_analysis_output_format.md):

# Criterion Check
1 Dual format JSON in prod, readable in dev
2 Timestamp Consistent, timezone-aware
3 Level field Present, uppercase
4 Trace/Correlation ID Present in every entry, async-safe
5 Service name Identifies source service
6 Source location module:line + function
7 Extra context Structured fields, not string interpolation
8 PII redaction Passwords, API keys, emails handled
9 Noise suppression Duplicate filters, third-party suppressed
10 Parseability Dev: pipe-delimited; prod: valid JSON per line

Score: passed criteria / 10.

Phase 4: Stack Trace Mapping + Fix Recommendations

For each Real Bug:

  1. Extract stack trace frames; identify origin frame (first frame in project code, not in node_modules/site-packages)
  2. Map to source file:line
  3. Generate fix recommendation: what to change, where, effort estimate (S/M/L)

Prioritize using Sentry-inspired dimensions:

  • High-volume (occurrence count), Post-test regression (new errors), High-impact path (auth/payment/DB), Correlated traces (trace_id across services)

Phase 5: Generate Report

MANDATORY READ: Load references/log_analysis_output_format.md

Output report to chat with header ## Test Log Analysis. Include:

  • Signals table (Real Bugs count, Test Artifacts filtered, Log Noise status, Log Format score, Log Quality score)
  • Real Bugs table (priority, category, error, source, fix recommendation)
  • Filtered table (category, count, examples)
  • Log Quality Issues table (dimension, service, issue, recommendation)
  • Noise Report table (count, ratio, service, level, template, action)
  • Machine-readable block <!-- LOG-ANALYSIS-DATA ... --> for programmatic consumption

Phase 6: Meta-Analysis

MANDATORY READ: Load shared/references/meta_analysis_protocol.md

Skill type: execution-worker. Run after all phases complete.

Verdict Contribution

Quality coordinator normalization matrix component:

Status Maps To Penalty
CLEAN -- 0
WARNINGS_ONLY -- 0
REAL_BUGS_FOUND FAIL -20
SKIPPED / NO_LOG_SOURCES ignored 0

Log quality/format issues are INFORMATIONAL — do not affect quality verdict. Only Real Bugs block.

Critical Rules

  • No status changes or task creation; report only.
  • Test Artifacts and Expected Behavior are ALWAYS filtered — never count as bugs.
  • Log quality issues are advisory — inform, don't block.
  • Script must handle gracefully: no Docker, no log files, no Loki → NO_LOG_SOURCES.
  • Language preservation in comments (EN/RU).

Definition of Done

  • Script deployed to target project scripts/ (or already exists).
  • Log source detected and script executed (or NO_LOG_SOURCES returned).
  • Errors classified into 4 categories; Real Bugs identified.
  • Log quality assessed (6 dimensions + 10-criterion format checklist).
  • Stack traces mapped to source files for Real Bugs.
  • Report output to chat with signals table + machine-readable block.

Reference Files

  • Error taxonomy: references/error_taxonomy.md
  • Output format: references/log_analysis_output_format.md
  • Analysis script: references/analyze_test_logs.py

Version: 1.0.0 Last Updated: 2026-03-13

04 同仓库更多 Skills

levalencia
levalencia/agent-god-mode

zarr-python

Chunked N-D arrays for cloud storage. Compressed arrays, parallel I/O, S3/GCS integration, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.

★ 3
levalencia
levalencia/agent-god-mode

zabbix

Zabbix monitoring system automation via API and Python. Use when: (1) Managing hosts, templates, items, triggers, or host groups, (2) Automating monitoring configuration, (3) Sending data via Zabbix trapper/sender, (4) Querying historical data or events, (5) Bulk operations on Zabbix objects, (6) Maintenance window management, (7) User/permission management

开发工程师后端与接口Skill与智能体开发
★ 3
levalencia
levalencia/agent-god-mode

x-twitter-growth

X/Twitter growth engine for building audience, crafting viral content, and analyzing engagement. Use when the user wants to grow on X/Twitter, write tweets or threads, analyze their X profile, research competitors on X, plan a posting strategy, or optimize engagement. Complements social-content (generic multi-platform) with X-specific depth: algorithm mechanics, thread engineering, reply strategy, profile optimization, and competitive intelligence via web search.

★ 3
levalencia
levalencia/agent-god-mode

what-if-oracle

Run structured What-If scenario analysis with multi-branch possibility exploration. Use this skill when the user asks speculative questions like "what if...", "what would happen if...", "what are the possibilities", "explore scenarios", "scenario analysis", "possibility space", "what could go wrong", "best case / worst case", "risk analysis", "contingency planning", "strategic options", or any question about uncertain futures. Also trigger when the user faces a fork-in-the-road decision, wants to stress-test an idea, or needs to think through consequences before committing.

★ 3
levalencia
levalencia/agent-god-mode

vercel-react-best-practices

React and Next.js performance optimization guidelines from Vercel Engineering. This skill should be used when writing, reviewing, or refactoring React/Next.js code to ensure optimal performance patterns. Triggers on tasks involving React components, Next.js pages, data fetching, bundle optimization, or performance improvements.

前端开发
★ 3
levalencia
levalencia/agent-god-mode

venue-templates

Access comprehensive LaTeX templates, formatting requirements, and submission guidelines for major scientific publication venues (Nature, Science, PLOS, IEEE, ACM), academic conferences (NeurIPS, ICML, CVPR, CHI), research posters, and grant proposals (NSF, NIH, DOE, DARPA). This skill should be used when preparing manuscripts for journal submission, conference papers, research posters, or grant proposals and need venue-specific formatting requirements and templates.

内容创作者写作与文案资料检索与研究
★ 3