AGENT SKILL

mistral-incident-runbook

Execute Mistral AI incident response procedures with triage, mitigation, and postmortem. Use when responding to Mistral AI-related outages, investigating errors, or running post-incident reviews for Mistral AI integration failures. Trigger with phrases like "mistral incident", "mistral outage", "mistral down", "mistral on-call", "mistral emergency", "mistral broken".

License 许可
MIT
最近更新
2026-11-15
Stars
★ 2.0k
Forks
233

01 前言 Front Matter

name
mistral-incident-runbook
author
Jeremy Longshore <jeremy@intentsolutions.io>
license
MIT
version
1.0.0
description
Execute Mistral AI incident response procedures with triage, mitigation, and postmortem. Use when responding to Mistral AI-related outages, investigating errors, or running post-incident reviews for Mistral AI integration failures. Trigger with phrases like "mistral incident", "mistral outage", "mistral down", "mistral on-call", "mistral emergency", "mistral broken".
allowed-tools
Read, Grep, Bash(kubectl:*), Bash(curl:*)

03 SKILL.md 正文

Mistral AI Incident Runbook

Overview

Rapid incident response procedures for Mistral AI-related outages.

Prerequisites

  • Access to Mistral AI console
  • kubectl access to production cluster (if applicable)
  • Prometheus/Grafana access
  • Communication channels (Slack, PagerDuty)

Severity Levels

Level Definition Response Time Examples
P1 Complete outage < 15 min Mistral API unreachable, all requests failing
P2 Degraded service < 1 hour High latency, partial failures, rate limiting
P3 Minor impact < 4 hours Occasional errors, non-critical feature down
P4 No user impact Next business day Monitoring gaps, documentation issues

Quick Triage

#!/bin/bash
# mistral-triage.sh

echo "=== Mistral AI Quick Triage ==="
echo "Timestamp: $(date)"
echo ""

# 1. Check Mistral API health
echo "1. Checking Mistral API..."
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
  -H "Authorization: Bearer ${MISTRAL_API_KEY}" \
  https://api.mistral.ai/v1/models)
echo "   API Status: $HTTP_STATUS"

# 2. Check our health endpoint
echo ""
echo "2. Checking our service health..."
curl -s https://api.yourapp.com/health | jq '.services.mistral' 2>/dev/null || echo "   Health check failed"

# 3. Check recent error rate
echo ""
echo "3. Recent errors (if Prometheus available)..."
curl -s "localhost:9090/api/v1/query?query=rate(mistral_errors_total[5m])" | jq '.data.result' 2>/dev/null || echo "   Prometheus not available"

# 4. Check recent logs
echo ""
echo "4. Recent error logs..."
kubectl logs -l app=mistral-service --since=5m 2>/dev/null | grep -i error | tail -10 || echo "   kubectl not available"

Decision Tree

Mistral API returning errors?
├─ YES: Check api.mistral.ai/v1/models with curl
│   ├─ 401 → API key issue (see Auth section)
│   ├─ 429 → Rate limited (see Rate Limit section)
│   ├─ 5xx → Mistral service issue (wait & monitor)
│   └─ Timeout → Network issue (check connectivity)
└─ NO: Our service returning errors?
    ├─ YES → Check our logs & config
    └─ NO → Likely resolved, continue monitoring

Immediate Actions by Error Type

401 Unauthorized - Authentication Failed

# 1. Verify API key is set
echo "API Key length: ${#MISTRAL_API_KEY}"
echo "API Key prefix: ${MISTRAL_API_KEY:0:10}..."

# 2. Test API key directly
curl -v -H "Authorization: Bearer ${MISTRAL_API_KEY}" \
  https://api.mistral.ai/v1/models

# 3. Check if key was rotated
# → Verify in Mistral console: console.mistral.ai

# 4. Update key if needed
kubectl create secret generic mistral-secrets \
  --from-literal=api-key="$NEW_API_KEY" \
  --dry-run=client -o yaml | kubectl apply -f -

# 5. Restart pods
kubectl rollout restart deployment/mistral-service

429 Rate Limited

# 1. Check current rate limit status
curl -v -H "Authorization: Bearer ${MISTRAL_API_KEY}" \
  https://api.mistral.ai/v1/models 2>&1 | grep -i "rate\|retry"

# 2. Enable request queuing (if supported)
kubectl set env deployment/mistral-service RATE_LIMIT_MODE=queue

# 3. Reduce request concurrency
kubectl set env deployment/mistral-service MAX_CONCURRENT_REQUESTS=5

# 4. Long-term: Contact Mistral for limit increase
# → console.mistral.ai or support@mistral.ai

500/503 Service Error

# 1. Check Mistral status (if available)
echo "Checking Mistral status..."

# 2. Enable graceful degradation
kubectl set env deployment/mistral-service MISTRAL_FALLBACK=true

# 3. Notify users
# → Update status page

# 4. Monitor for recovery
watch -n 30 'curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer ${MISTRAL_API_KEY}" https://api.mistral.ai/v1/models'

Timeout/Network Error

# 1. Test connectivity
curl -v --connect-timeout 5 https://api.mistral.ai/v1/models

# 2. Check DNS resolution
nslookup api.mistral.ai

# 3. Increase timeout
kubectl set env deployment/mistral-service MISTRAL_TIMEOUT=60000

# 4. Check egress rules
kubectl get networkpolicy

Communication Templates

Internal (Slack)

:red_circle: P1 INCIDENT: Mistral AI Integration
**Status**: INVESTIGATING
**Impact**: [Users cannot use AI features / Degraded AI responses]
**Current action**: [What you're doing]
**Next update**: [Time - typically every 15-30 min for P1]
**Incident commander**: @[name]

External (Status Page)

AI Feature Degradation

We're experiencing issues with our AI-powered features.
Some users may experience slower responses or temporary unavailability.

Our team is actively investigating and working with our AI provider to resolve this.

Affected services:
- [List affected features]

Workaround: [If available]

Last updated: [timestamp]

Post-Incident

Evidence Collection

#!/bin/bash
# collect-evidence.sh

INCIDENT_DIR="incident-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$INCIDENT_DIR"

# Collect logs
kubectl logs -l app=mistral-service --since=1h > "$INCIDENT_DIR/logs.txt"

# Export metrics
curl "localhost:9090/api/v1/query_range?query=mistral_errors_total&start=$(date -d '2 hours ago' +%s)&end=$(date +%s)&step=60" \
  > "$INCIDENT_DIR/metrics.json"

# Collect config (redacted)
kubectl get deployment mistral-service -o yaml | grep -v "api-key" > "$INCIDENT_DIR/deployment.yaml"

# Create bundle
tar -czf "$INCIDENT_DIR.tar.gz" "$INCIDENT_DIR"
echo "Evidence bundle: $INCIDENT_DIR.tar.gz"

Postmortem Template

## Incident: Mistral AI [Error Type]
**Date:** YYYY-MM-DD
**Duration:** X hours Y minutes
**Severity:** P[1-4]
**Incident Commander:** [Name]

### Summary
[1-2 sentence description of what happened]

### Timeline (UTC)
| Time | Event |
|------|-------|
| HH:MM | First alert triggered |
| HH:MM | Incident declared |
| HH:MM | Root cause identified |
| HH:MM | Mitigation applied |
| HH:MM | Service restored |

### Root Cause
[Technical explanation of what went wrong]

### Impact
- Users affected: [Number or percentage]
- Duration of impact: [Time]
- Requests failed: [Number]
- Revenue impact: [If applicable]

### Detection
How was the incident detected?
- [ ] Automated alerting
- [ ] Customer report
- [ ] Internal testing
- [ ] Other: ___

### Resolution
[What was done to fix the issue]

### Action Items
| Priority | Action | Owner | Due Date | Status |
|----------|--------|-------|----------|--------|
| P1 | [Immediate fix] | @name | YYYY-MM-DD | [ ] |
| P2 | [Preventive measure] | @name | YYYY-MM-DD | [ ] |
| P3 | [Improvement] | @name | YYYY-MM-DD | [ ] |

### Lessons Learned
- What went well:
- What could be improved:
- What we got lucky with:

Output

  • Issue identified and categorized
  • Mitigation applied
  • Stakeholders notified
  • Evidence collected for postmortem

Error Handling

Issue Cause Solution
kubectl fails Auth expired Re-authenticate with cloud provider
Metrics unavailable Prometheus down Check backup metrics or logs
Secret rotation fails Permission denied Escalate to admin
Fallback not working Not implemented Use cached responses or error page

Examples

One-Line Health Check

curl -sf -H "Authorization: Bearer ${MISTRAL_API_KEY}" https://api.mistral.ai/v1/models | jq '.data[0].id' || echo "UNHEALTHY"

Quick Rollback

kubectl rollout undo deployment/mistral-service && \
kubectl rollout status deployment/mistral-service

Resources

Next Steps

For data handling, see mistral-data-handling.

04 同仓库更多 Skills

Dicklesworthstone
Dicklesworthstone/pi_agent_rust

theme-factory

Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.

视觉设计
★ 2.0k
Dicklesworthstone
Dicklesworthstone/pi_agent_rust

web-artifacts-builder

Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn/ui). Use for complex artifacts requiring state management, routing, or shadcn/ui components - not for simple single-file HTML/JSX artifacts.

前端开发
★ 2.0k
Dicklesworthstone
Dicklesworthstone/pi_agent_rust

webapp-testing

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.

测试与质量前端开发
★ 2.0k
Dicklesworthstone
Dicklesworthstone/pi_agent_rust

neuropixels-analysis

Neuropixels neural recording analysis. Load SpikeGLX/OpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen/IBL curation, AI-assisted visual analysis, for Neuropixels 1.0/2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.

★ 2.0k
Dicklesworthstone
Dicklesworthstone/pi_agent_rust

skill-creator

Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.

Skill与智能体开发
★ 2.0k
Dicklesworthstone
Dicklesworthstone/pi_agent_rust

notebooklm

Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. Browser automation, library management, persistent auth. Drastically reduced hallucinations through document-only responses.

Skill与智能体开发
★ 2.0k