AGENT SKILL

owasp-llm-top10-review

OWASP Top 10 for LLM Applications 2025 assessment of anything that calls a model: direct and indirect prompt injection, sensitive disclosure, output handling, excessive agency, system-prompt leakage, vector and embedding weaknesses, unbounded consumption. Use for chat, RAG, agentic or LLM-orchestration code, alongside the web OWASP review, such as 'can someone smuggle instructions in through retrieved documents' or 'is our chatbot safe to launch'. Not for systems that make no model calls (owasp-top10-review) or one delivered ticket (security-review).

License 许可
MIT
最近更新
2026-10-22
Stars
★ 3
Forks
0

01 前言 Front Matter

name
owasp-llm-top10-review
effort
xhigh
context
fork
description
OWASP Top 10 for LLM Applications 2025 assessment of anything that calls a model: direct and indirect prompt injection, sensitive disclosure, output handling, excessive agency, system-prompt leakage, vector and embedding weaknesses, unbounded consumption. Use for chat, RAG, agentic or LLM-orchestration code, alongside the web OWASP review, such as 'can someone smuggle instructions in through retrieved documents' or 'is our chatbot safe to launch'. Not for systems that make no model calls (owasp-top10-review) or one delivered ticket (security-review).
argument-hint
[system slug]

02 资源文件 1

03 SKILL.md 正文

OWASP LLM Top 10 Review Skill

Purpose

Perform a systematic, category-by-category assessment of an AI/LLM application against the OWASP Top 10 for LLM Applications 2025. Generic web security reviews (and even the OWASP web Top 10) do not capture risks that are specific to systems that build prompts from untrusted input, retrieve external content into model context, persist model output as product, grant models agency, or spend money per inference. This skill covers that model-specific surface.

Use it for anything LLM-shaped: chat/assistant apps, RAG pipelines, agentic systems, multi-stage model orchestration, LLM-as-judge, content generation, and tool-using agents.

When to Use

Use this skill when the system:

  • Calls an LLM (hosted API or self-hosted) anywhere in a request or job path
  • Substitutes user/tenant input or retrieved content into prompts
  • Implements RAG, embeddings, vector search, or any retrieval-augmented context
  • Grants a model agency (tool calls, external writes, sends, deploys) based on its output
  • Persists or surfaces model output to users as product
  • Has a per-inference cost (token/$ spend) that a caller can influence

Run it alongside the OWASP Top 10 (web) Review — the web list covers the HTTP/infra surface; this covers the model surface. Both are needed for an LLM app.

Input

Source: The LLM orchestration code plus its surrounding app Read from:

  • Prompt assembly code and prompt templates (how variables/retrieved content enter prompts)
  • The LLM client(s) — request body construction, output handling, retries, structured-output usage
  • Retrieval/ingestion stages (search, scraping, fetch, RAG) and any vector store
  • Telemetry/observation surfaces — what model I/O is exposed via API/UI
  • Cost/usage tracking and any budget/quota enforcement
  • Any agency surface — tool definitions, external writes, sends, exports
  • Every existing security review report (.gener8v/changes/*/reviews/*-security-review.md, and legacy .gener8v/reviews/*-security-review.md) and the web OWASP assessment, referenced as <change-slug>/<report-slug>/SEC-XXX (e.g. support-search/search-and-retrieval-ticket-002-security-review/SEC-002) or <system-slug>-owasp-top10-assessment/SEC-XXX, to map onto rather than duplicate

Expects: LLM-touching code to exist. If there is none, say so and stop — this skill does not apply.

Output

Produces: An OWASP LLM Top 10:2025 assessment Write to: .gener8v/reviews/[slug]-owasp-llm-top10-assessment.md Creates directory: .gener8v/reviews/ if absent Naming: -owasp-llm-top10-assessment suffix

New findings are numbered SEC-001… within this assessment and referenced from elsewhere as <system-slug>-owasp-llm-top10-assessment/SEC-XXX. Per-ticket security reviews are never edited by this skill.

The Framework — OWASP Top 10 for LLM Applications 2025

Assess every category:

  • LLM01:2025 — Prompt Injection (direct: user input into prompts; indirect: attacker-controlled retrieved content into prompts — often the higher-severity vector)
  • LLM02:2025 — Sensitive Information Disclosure (model leaks PII/secrets/other-tenant data; raw prompt/response exposed via API/UI)
  • LLM03:2025 — Supply Chain (model provenance/attestation, model gateway trust, client-lib CVEs, plugin/extension trust)
  • LLM04:2025 — Data and Model Poisoning (poisoned training/fine-tune data; poisoned retrieved content; SEO-gamed sources feeding context)
  • LLM05:2025 — Improper Output Handling (model output used unsafely downstream: eval, SQL/command building, unsanitized HTML/markdown render → XSS)
  • LLM06:2025 — Excessive Agency (model output triggers external actions — tool calls, writes, sends — without validation or human-in-the-loop)
  • LLM07:2025 — System Prompt Leakage (system prompts exposed via API/UI/error; secrets embedded in system prompts)
  • LLM08:2025 — Vector and Embedding Weaknesses (RAG/vector-store risks: embedding inversion, cross-tenant retrieval, index poisoning)
  • LLM09:2025 — Misinformation (hallucination/overreliance, especially for customer-facing analytical output; provenance/confidence)
  • LLM10:2025 — Unbounded Consumption (denial-of-wallet / DoS: no spend or token ceiling, no per-tenant quota, unbounded automation)

Output Format

# [System]: OWASP Top 10 for LLM Applications (2025) Assessment

## 1. Coverage Summary
[Table: # | Category | Status | Findings]
[Net: which 1-2 LLM risks are material and why.]

## 2. Category-by-Category Assessment
### LLM01 — Prompt Injection · [status]
**How prompts are built:** [evidence — file:line of substitution]
**Findings:** [direct + indirect; SEC-XXX]
**Mitigations present:** ...
[... repeat LLM02–LLM10 ...]

## 3. New Findings
[Each new SEC-XXX: severity, category (OWASP LLMxx), location, attack scenario, impact, recommendation.]

## 4. Positive Controls (LLM-specific)
## 5. Conclusion

Principles

Indirect Injection Is the Sharp Edge

Direct prompt injection (a user editing their own prompt) is usually low-impact — the user only harms their own session, and output is typically parsed, not executed. Indirect injection — adversarial instructions inside retrieved web pages, documents, or search results that the pipeline feeds to the model as trusted context — is the higher-severity vector because the attacker need not be a user. Always trace where retrieved/external content enters a prompt, and treat it as untrusted, clearly-delimited data.

Follow the Money (LLM10)

LLM apps have a per-inference cost. "Cost is tracked" is not "cost is bounded." Check whether anything enforces a per-request, per-tenant, or aggregate ceiling and aborts on breach. Unbounded, caller-influenced inference is denial-of-wallet — a real availability/financial risk, sharper when the endpoint is public or unauthenticated.

Agency Is the Multiplier

Rate every category higher when the model can act (tools, writes, sends). With no agency (read-only stages writing to own DB), LLM05/06 are low; the moment output drives external artifacts or actions, re-rate and demand output validation + human-in-the-loop. Always note planned agency ("Sheets export ships next slice → re-assess LLM06").

Check What's Exposed, Not Just What's Stored

For LLM02/LLM07, inspect the API/UI response schemas: is raw prompt or response text surfaced? Are system prompts reachable via any endpoint or error? Deliberate suppression of raw model I/O is a positive control worth crediting.

Structured Output Is a Backstop, Not a Fix

Strict JSON-schema/response_format constrains output shape and limits injection blast radius — credit it, but it does not stop injection from skewing the content within the schema. Don't let it substitute for input-trust handling.

Attack Scenarios for Medium+

Same rule as the other review skills: who, what access, what they achieve.

Process

  1. Confirm applicability — locate LLM-touching code; if none, stop.
  2. Trace prompt assembly — how do user input and retrieved content enter prompts? Is there delimiting/escaping? (LLM01)
  3. Trace retrieval/ingestion — what external content is fetched and fed to the model? Trust-scored? Sanitized? (LLM01 indirect, LLM04, LLM08)
  4. Inspect output handling — is model output parsed, or eval'd / used to build SQL/commands / rendered as raw HTML? (LLM05)
  5. Map agency — does output trigger external actions? With what validation/human gate? (LLM06)
  6. Inspect exposure surfaces — API/UI schemas for raw prompt/response/system-prompt leakage; cross-tenant context. (LLM02, LLM07)
  7. Follow the money — per-call caps vs aggregate/tenant ceilings; abort-on-breach; quotas. (LLM10)
  8. Model supply chain — provenance/pinning, gateway trust, client-lib CVEs, plugins. (LLM03)
  9. Misinformation controls — quality scoring, provenance/confidence surfacing, human-in-the-loop for customer-facing output. (LLM09)
  10. Raise & number findings, write positive controls, and write the report to .gener8v/reviews/.

Example (abbreviated)

LLM10 — Unbounded Consumption · Gaps. Per-call max_tokens exists, but nothing caps total tokens/$ per audit, engagement, or tenant, and no run aborts on cost. Cost is tracked post-hoc, never enforced. With public exposure + no rate limit → denial-of-wallet. → SEC-019 (Medium).

LLM01 — Prompt Injection · Gaps. Naive {{var}} substitution of tenant input (SEC-008, info) and of web-search-retrieved page content (SEC-020, Medium — an attacker who ranks a page for the target entity injects instructions into the model's context). Structured-output constrains the blast radius but does not remove it.

Troubleshooting

  • Step 1 found no LLM-touching code, but the system does call a model. Model calls often hide behind a gateway, an in-house client wrapper, a plain HTTP call to an inference endpoint, or another repository in the workspace. Before stopping, grep for provider SDK imports, chat or completion endpoint paths and gateway hostnames, and check context.md's ## Repositories table. When there is genuinely none, stop without writing an assessment and recommend the OWASP Top 10 (web) Review.
  • The assessment covered the wrong system, or missed the surface the user cared about. This skill runs in a forked subagent that sees the skill and its arguments, not the conversation. Put the system slug and the surface — the RAG ingestion path, the agent's tool set — in the argument. The slug names the report .gener8v/reviews/<slug>-owasp-llm-top10-assessment.md; Orchestrate finds assessments in .gener8v/reviews/ by their -assessment.md suffix, so keep the name exact.
  • Prompt templates live outside the code — a prompt-management service, a bucket, another repository. LLM01 and LLM07 then rest on templates this run could not read. Name them, assess the assembly code that is visible, and mark both categories as partial rather than clean.
  • There is no cost tracking or budget enforcement at all. That absence is the LLM10 finding, not "not applicable". Rate it by what a caller can make the system spend, as in the example.
  • Two different findings are both SEC-001. Numbering restarts per report. Cite qualified IDs throughout — <change-slug>/<report-slug>/SEC-XXX for ticket reviews, <slug>-owasp-top10-assessment/SEC-XXX for the web assessment, <slug>-owasp-llm-top10-assessment/SEC-XXX for this one.

Integration with Other Skills

Parallel: OWASP Top 10 (web) Review — run both for an LLM app; this covers the model surface, that the HTTP/infra surface. Upstream: Security Review (per-ticket SEC-XXX findings under changes/<change-slug>/reviews/); Technical Design (model/agency architecture). Downstream: Audit; Ticket Breakdown (findings that need code changes become tickets); Orchestrate (lists the assessment under cross_cutting.assessments).

Revisions

  • Re-assess on architectural change — adding RAG/vectors re-opens LLM08 and LLM02; adding tools/agents re-rates LLM05/06; going multi-tenant re-opens LLM02 cross-tenant — and when the OWASP GenAI list is revised.
  • Re-running replaces [slug]-owasp-llm-top10-assessment.md; carry forward open findings with their status.

Notes

  • Framework version: OWASP Top 10 for LLM Applications 2025. The list evolves quickly (the 2025 edition added Vector & Embedding Weaknesses and System Prompt Leakage and reordered others); confirm the current edition before assessing.
  • The OWASP GenAI Security Project also publishes companion guidance (agentic threats, red-teaming); pull from it for agentic systems.
  • Pairs naturally with a threat model for the orchestration/agency layer.

04 同仓库更多 Skills

gener8v
gener8v/gener8v.claude-skills

ticket-breakdown

Decompose one capability area's requirements for one change into implementable tickets with Priority, Value, acceptance criteria, Prior Art, Output contracts, Known Hazards, dependency ordering and relative sizing: one TICKET-NNN.md per ticket plus backlog.md, in the change's tickets directory. Use when a specification is approved and the team needs work items, such as 'break this spec into tickets' or 'create the backlog for search'. Not for implementing a ticket (delivery) or ordering whole capability areas (dependencies).

★ 3
gener8v
gener8v/gener8v.claude-skills

technical-design

Translate specifications, constraints and dependencies into architecture decisions (AD-XXX), component boundaries, data model and interface contracts under .gener8v/technical-design/. Use when technology or architecture choices must be settled before tickets are written, such as 'which datastore should we use' or 'design the components for search'. Skip it when the approach is obvious and uncontested. Not for judging an architecture that has already shipped (architecture-review).

设计师视觉设计
★ 3
gener8v
gener8v/gener8v.claude-skills

specification

Expand one capability area from .gener8v/prd.md into a functional specification with atomic, testable [PREFIX]-REQ-XXX requirements, behaviors, edge cases and open questions under .gener8v/specifications/. Use when a PRD capability area needs elaboration before constraints, technical design or tickets, or a change brief still marks an area as pending specification, such as 'spec out the search area' or 'write the requirements for billing'. Not for the whole product (planning) or work items (ticket-breakdown).

产品经理
★ 3
gener8v
gener8v/gener8v.claude-skills

setup

Bootstrap the gener8v pipeline in a project: create .gener8v/, copy the conventions, generate pipeline-state.yaml, and add the pipeline directives to CLAUDE.md. Use when the user says 'set up gener8v' or 'initialize the pipeline', when a project has no .gener8v/prd.md, or after a plugin upgrade to refresh the conventions. Not for deciding what to do next on a project already on the pipeline (orchestrate).

★ 3
gener8v
gener8v/gener8v.claude-skills

security-review

OWASP-informed, code-level security review of delivered code: injection, authentication and authorization, data exposure, configuration, dependencies, cryptography and logging, with attack scenarios for Medium+ findings and compliance constraints (CC-XXX) treated as Critical. Use after a delivery, especially one touching input handling, auth, sensitive data or external integrations, such as 'security review TICKET-007' or 'check what this delivery shipped for injection or auth bypass before we merge'. Not for a whole-codebase OWASP posture assessment (owasp-top10-review) or prompt-injection risk in LLM features (owasp-llm-top10-review).

律师与法务
★ 3
gener8v
gener8v/gener8v.claude-skills

rot-watch

Detect decay in code that is being actively written: measure six signals (packaging drift, dependency direction, comment language, growth slopes, tests, the detectable subset of SOLID) against a recorded baseline and report only what moved, at .gener8v/rot/. Use once code has been delivered and again every few deliveries, before a release, or when asked 'is the codebase getting worse' or 'has this module drifted'. The first run records the baseline and reports nothing. Not for one delivery's quality (quality-review), defects (defect-sweep) or whether the architecture was right (architecture-review).

★ 3