[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6NTM321N577FV7XVJ92TNPY":3,"skill-same:01M6NTM321N577FV7XVJ92TNPY":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":30,"commitSha":52,"skillDirectory":53,"forksCount":13,"repositoryHtmlUrl":54,"repositoryHomepageUrl":6,"ownerHtmlUrl":55,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":56,"install":57,"resources":70},"01M6NTM321N577FV7XVJ92TNPY","ai-inference-service-mesh",null,"Use service mesh patterns for AI inference traffic management, mTLS, canary releases, policy enforcement, and cross-cluster resilience.","duc-duy-1612\u002FOnline-Delivery-Systems-Web","duc-duy-1612","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F192231384?v=4","01M6NSDKQQDKVTHZ8DZMZ1VYY6","01M6NSDKW8DDVTQHBT1HBDDJPT",0,"MIT","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fai-inference-service-mesh\u002FSKILL.md",[],[18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"inference","service","ai inference service mesh","service mesh","mesh","resilience","devops-security","mtls","access","only","releases","community","main","patterns","security","os","critical","scripts","relevant","traffic","2026-11-10 10:12:30",false,"---\nname: ai-inference-service-mesh\ndescription: Use service mesh patterns for AI inference traffic management, mTLS,\n  canary releases, policy enforcement, and cross-cluster resilience.\ncategory: devops\nrisk: critical\nsource: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\nsource_repo: BagelHole\u002FDevOps-Security-Agent-Skills\nsource_type: community\ndate_added: '2026-09-20'\nlicense: MIT\nlicense_source: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE\ncompatibility: Requires the relevant OS\u002Fplatform tooling and privileged access where\n  noted. Docs-only; helper scripts and templates not bundled.\nmetadata:\n  author: devops-skills\n  version: '1.0'\n---\n\n# AI Inference Service Mesh\n\nApply Istio\u002FLinkerd mesh controls to secure and optimize east-west AI traffic across inference microservices.\n\n## Why Mesh for AI\n\n- Enforce mTLS between gateway, retriever, reranker, and model services\n- Apply fine-grained traffic policies without app code changes\n- Run progressive delivery for model-serving backends\n- Observe latency hops for retrieval + generation chains\n- Route inference requests by model version, tenant, or priority tier\n- Protect expensive GPU-backed services from cascading failures\n\n## Prerequisites\n\n```bash\n# Install Istio with production profile\nistioctl install --set profile=default \\\n  --set meshConfig.accessLogFile=\u002Fdev\u002Fstdout \\\n  --set meshConfig.defaultConfig.holdApplicationUntilProxyStarts=true\n\n# Label inference namespace for sidecar injection\nkubectl create namespace ai-inference\nkubectl label namespace ai-inference istio-injection=enabled\n\n# Verify installation\nistioctl verify-install\nistioctl analyze -n ai-inference\n```\n\n## Core Patterns\n\n### mTLS Strict Mode Cluster-Wide\n\n```yaml\napiVersion: security.istio.io\u002Fv1beta1\nkind: PeerAuthentication\nmetadata:\n  name: default\n  namespace: istio-system\nspec:\n  mtls:\n    mode: STRICT\n---\n# Namespace-level override if needed for gradual rollout\napiVersion: security.istio.io\u002Fv1beta1\nkind: PeerAuthentication\nmetadata:\n  name: ai-inference-mtls\n  namespace: ai-inference\nspec:\n  mtls:\n    mode: STRICT\n  portLevelMtls:\n    # gRPC inference port\n    8081:\n      mode: STRICT\n    # Prometheus metrics port - allow plaintext scraping\n    9090:\n      mode: PERMISSIVE\n```\n\n### AuthorizationPolicy Per Service Account\n\n```yaml\napiVersion: security.istio.io\u002Fv1beta1\nkind: AuthorizationPolicy\nmetadata:\n  name: model-server-access\n  namespace: ai-inference\nspec:\n  selector:\n    matchLabels:\n      app: model-server\n  action: ALLOW\n  rules:\n  - from:\n    - source:\n        principals:\n        - \"cluster.local\u002Fns\u002Fai-inference\u002Fsa\u002Fapi-gateway\"\n        - \"cluster.local\u002Fns\u002Fai-inference\u002Fsa\u002Forchestrator\"\n    to:\n    - operation:\n        methods: [\"POST\"]\n        paths: [\"\u002Fv1\u002Fpredict\", \"\u002Fv1\u002Fembeddings\", \"\u002Fv2\u002Fmodels\u002F*\u002Finfer\"]\n---\napiVersion: security.istio.io\u002Fv1beta1\nkind: AuthorizationPolicy\nmetadata:\n  name: deny-external-to-retriever\n  namespace: ai-inference\nspec:\n  selector:\n    matchLabels:\n      app: vector-retriever\n  action: DENY\n  rules:\n  - from:\n    - source:\n        notNamespaces: [\"ai-inference\"]\n```\n\n### Egress Policy for Approved Model Endpoints\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: ServiceEntry\nmetadata:\n  name: openai-api\n  namespace: ai-inference\nspec:\n  hosts:\n  - api.openai.com\n  ports:\n  - number: 443\n    name: https\n    protocol: TLS\n  resolution: DNS\n  location: MESH_EXTERNAL\n---\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: openai-api-tls\n  namespace: ai-inference\nspec:\n  host: api.openai.com\n  trafficPolicy:\n    tls:\n      mode: SIMPLE\n    connectionPool:\n      http:\n        h2UpgradePolicy: UPGRADE\n      tcp:\n        maxConnections: 50\n---\napiVersion: security.istio.io\u002Fv1beta1\nkind: AuthorizationPolicy\nmetadata:\n  name: restrict-egress\n  namespace: ai-inference\nspec:\n  action: ALLOW\n  rules:\n  - to:\n    - operation:\n        hosts:\n        - \"api.openai.com\"\n        - \"models.anthropic.com\"\n        - \"*.blob.core.windows.net\"\n```\n\n## Traffic Management\n\n### VirtualService for A\u002FB Model Testing\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: VirtualService\nmetadata:\n  name: model-server\n  namespace: ai-inference\nspec:\n  hosts:\n  - model-server\n  http:\n  # Route by header for explicit model version selection\n  - match:\n    - headers:\n        x-model-version:\n          exact: \"v2-experimental\"\n    route:\n    - destination:\n        host: model-server\n        subset: v2-experimental\n    timeout: 120s\n  # Route by header for A\u002FB test cohort\n  - match:\n    - headers:\n        x-ab-cohort:\n          exact: \"treatment\"\n    route:\n    - destination:\n        host: model-server\n        subset: v2-experimental\n      weight: 100\n    timeout: 120s\n  # Default traffic split: 90\u002F10 canary\n  - route:\n    - destination:\n        host: model-server\n        subset: v1-stable\n      weight: 90\n    - destination:\n        host: model-server\n        subset: v2-experimental\n      weight: 10\n    timeout: 60s\n    retries:\n      attempts: 2\n      perTryTimeout: 30s\n      retryOn: unavailable,resource-exhausted\n```\n\n### DestinationRule with Subsets\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: model-server\n  namespace: ai-inference\nspec:\n  host: model-server\n  trafficPolicy:\n    connectionPool:\n      http:\n        h2UpgradePolicy: UPGRADE\n        maxRequestsPerConnection: 100\n      tcp:\n        maxConnections: 200\n        connectTimeout: 5s\n    loadBalancer:\n      simple: LEAST_REQUEST\n  subsets:\n  - name: v1-stable\n    labels:\n      version: v1\n    trafficPolicy:\n      connectionPool:\n        http:\n          maxRequestsPerConnection: 50\n  - name: v2-experimental\n    labels:\n      version: v2\n    trafficPolicy:\n      connectionPool:\n        http:\n          maxRequestsPerConnection: 20\n```\n\n### Circuit Breaking for Inference Backends\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: model-server-circuit-breaker\n  namespace: ai-inference\nspec:\n  host: model-server\n  trafficPolicy:\n    connectionPool:\n      tcp:\n        maxConnections: 100\n        connectTimeout: 10s\n      http:\n        http1MaxPendingRequests: 50\n        http2MaxRequests: 200\n        maxRequestsPerConnection: 10\n        maxRetries: 3\n    outlierDetection:\n      consecutive5xxErrors: 3\n      interval: 15s\n      baseEjectionTime: 30s\n      maxEjectionPercent: 50\n      minHealthPercent: 30\n      splitExternalLocalOriginErrors: true\n---\n# Separate circuit breaker for the vector retriever\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: vector-retriever-circuit-breaker\n  namespace: ai-inference\nspec:\n  host: vector-retriever\n  trafficPolicy:\n    connectionPool:\n      tcp:\n        maxConnections: 300\n      http:\n        http1MaxPendingRequests: 200\n        http2MaxRequests: 500\n    outlierDetection:\n      consecutive5xxErrors: 5\n      interval: 10s\n      baseEjectionTime: 15s\n      maxEjectionPercent: 30\n```\n\n### Retry Budget for Streaming Requests\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: VirtualService\nmetadata:\n  name: streaming-inference\n  namespace: ai-inference\nspec:\n  hosts:\n  - model-server\n  http:\n  # Streaming endpoint: no retries, long timeout\n  - match:\n    - uri:\n        prefix: \u002Fv1\u002Fstream\n    route:\n    - destination:\n        host: model-server\n        subset: v1-stable\n    timeout: 300s\n    retries:\n      attempts: 0\n  # Embeddings endpoint: safe to retry, short timeout\n  - match:\n    - uri:\n        prefix: \u002Fv1\u002Fembeddings\n    route:\n    - destination:\n        host: model-server\n        subset: v1-stable\n    timeout: 15s\n    retries:\n      attempts: 3\n      perTryTimeout: 5s\n      retryOn: 5xx,reset,connect-failure,retriable-status-codes\n```\n\n## Resilience\n\n### Locality-Aware Routing\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: model-server-locality\n  namespace: ai-inference\nspec:\n  host: model-server\n  trafficPolicy:\n    loadBalancer:\n      localityLbSetting:\n        enabled: true\n        distribute:\n        - from: \"us-east-1\u002Fus-east-1a\u002F*\"\n          to:\n            \"us-east-1\u002Fus-east-1a\u002F*\": 80\n            \"us-east-1\u002Fus-east-1b\u002F*\": 20\n        failover:\n        - from: us-east-1\n          to: us-west-2\n    outlierDetection:\n      consecutive5xxErrors: 3\n      interval: 10s\n      baseEjectionTime: 30s\n```\n\n## Observability\n\n```yaml\n# Telemetry resource for custom metrics on inference services\napiVersion: telemetry.istio.io\u002Fv1alpha1\nkind: Telemetry\nmetadata:\n  name: inference-telemetry\n  namespace: ai-inference\nspec:\n  metrics:\n  - providers:\n    - name: prometheus\n    overrides:\n    - match:\n        metric: REQUEST_DURATION\n        mode: CLIENT_AND_SERVER\n      tagOverrides:\n        model_name:\n          operation: UPSERT\n          value: \"request.headers['x-model-name']\"\n        tenant_id:\n          operation: UPSERT\n          value: \"request.headers['x-tenant-id']\"\n  tracing:\n  - providers:\n    - name: zipkin\n    randomSamplingPercentage: 10.0\n```\n\n### Kiali Dashboard Check\n\n```bash\n# Port-forward Kiali\nkubectl port-forward svc\u002Fkiali -n istio-system 20001:20001 &\n\n# Verify mesh health via API\ncurl -s http:\u002F\u002Flocalhost:20001\u002Fkiali\u002Fapi\u002Fnamespaces\u002Fai-inference\u002Fhealth | jq .\n\n# Check proxy sync status\nistioctl proxy-status -n ai-inference\n\n# Debug a specific pod sidecar config\nistioctl proxy-config routes deploy\u002Fmodel-server -n ai-inference -o json\nistioctl proxy-config cluster deploy\u002Fmodel-server -n ai-inference\n```\n\n## Pitfalls to Avoid\n\n- Aggressive timeouts that break streaming responses -- set 300s+ for generation endpoints\n- Blanket retries that amplify expensive generation calls -- disable retries on non-idempotent routes\n- Missing identity boundaries between tenant-facing and internal services\n- Forgetting to exempt health check and metrics ports from strict mTLS\n- Setting outlier ejection too aggressively on small pools (maxEjectionPercent too high)\n- Not using `holdApplicationUntilProxyStarts` causing race conditions on startup\n\n## Related Skills\n\n- service-mesh (`service-mesh`) - Foundational mesh concepts\n- llm-gateway (`llm-gateway`) - North-south API gateway controls\n- opentelemetry (`opentelemetry`) - End-to-end tracing and metrics\n\n## When to Use\n\n- You are provisioning, configuring, or troubleshooting the infrastructure component covered by this skill (servers, storage, databases, networking, cloud, local AI).\n\n## Limitations\n\n- Infrastructure commands can disrupt services: confirm target host\u002Fscope and have backups\u002Fsnapshots before mutating state.\n- Docs-only import: upstream scripts and templates not bundled.\n\n","# AI Inference Service Mesh\n\nApply Istio\u002FLinkerd mesh controls to secure and optimize east-west AI traffic across inference microservices.\n\n## Why Mesh for AI\n\n- Enforce mTLS between gateway, retriever, reranker, and model services\n- Apply fine-grained traffic policies without app code changes\n- Run progressive delivery for model-serving backends\n- Observe latency hops for retrieval + generation chains\n- Route inference requests by model version, tenant, or priority tier\n- Protect expensive GPU-backed services from cascading failures\n\n## Prerequisites\n\n```bash\n# Install Istio with production profile\nistioctl install --set profile=default \\\n  --set meshConfig.accessLogFile=\u002Fdev\u002Fstdout \\\n  --set meshConfig.defaultConfig.holdApplicationUntilProxyStarts=true\n\n# Label inference namespace for sidecar injection\nkubectl create namespace ai-inference\nkubectl label namespace ai-inference istio-injection=enabled\n\n# Verify installation\nistioctl verify-install\nistioctl analyze -n ai-inference\n```\n\n## Core Patterns\n\n### mTLS Strict Mode Cluster-Wide\n\n```yaml\napiVersion: security.istio.io\u002Fv1beta1\nkind: PeerAuthentication\nmetadata:\n  name: default\n  namespace: istio-system\nspec:\n  mtls:\n    mode: STRICT\n---\n# Namespace-level override if needed for gradual rollout\napiVersion: security.istio.io\u002Fv1beta1\nkind: PeerAuthentication\nmetadata:\n  name: ai-inference-mtls\n  namespace: ai-inference\nspec:\n  mtls:\n    mode: STRICT\n  portLevelMtls:\n    # gRPC inference port\n    8081:\n      mode: STRICT\n    # Prometheus metrics port - allow plaintext scraping\n    9090:\n      mode: PERMISSIVE\n```\n\n### AuthorizationPolicy Per Service Account\n\n```yaml\napiVersion: security.istio.io\u002Fv1beta1\nkind: AuthorizationPolicy\nmetadata:\n  name: model-server-access\n  namespace: ai-inference\nspec:\n  selector:\n    matchLabels:\n      app: model-server\n  action: ALLOW\n  rules:\n  - from:\n    - source:\n        principals:\n        - \"cluster.local\u002Fns\u002Fai-inference\u002Fsa\u002Fapi-gateway\"\n        - \"cluster.local\u002Fns\u002Fai-inference\u002Fsa\u002Forchestrator\"\n    to:\n    - operation:\n        methods: [\"POST\"]\n        paths: [\"\u002Fv1\u002Fpredict\", \"\u002Fv1\u002Fembeddings\", \"\u002Fv2\u002Fmodels\u002F*\u002Finfer\"]\n---\napiVersion: security.istio.io\u002Fv1beta1\nkind: AuthorizationPolicy\nmetadata:\n  name: deny-external-to-retriever\n  namespace: ai-inference\nspec:\n  selector:\n    matchLabels:\n      app: vector-retriever\n  action: DENY\n  rules:\n  - from:\n    - source:\n        notNamespaces: [\"ai-inference\"]\n```\n\n### Egress Policy for Approved Model Endpoints\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: ServiceEntry\nmetadata:\n  name: openai-api\n  namespace: ai-inference\nspec:\n  hosts:\n  - api.openai.com\n  ports:\n  - number: 443\n    name: https\n    protocol: TLS\n  resolution: DNS\n  location: MESH_EXTERNAL\n---\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: openai-api-tls\n  namespace: ai-inference\nspec:\n  host: api.openai.com\n  trafficPolicy:\n    tls:\n      mode: SIMPLE\n    connectionPool:\n      http:\n        h2UpgradePolicy: UPGRADE\n      tcp:\n        maxConnections: 50\n---\napiVersion: security.istio.io\u002Fv1beta1\nkind: AuthorizationPolicy\nmetadata:\n  name: restrict-egress\n  namespace: ai-inference\nspec:\n  action: ALLOW\n  rules:\n  - to:\n    - operation:\n        hosts:\n        - \"api.openai.com\"\n        - \"models.anthropic.com\"\n        - \"*.blob.core.windows.net\"\n```\n\n## Traffic Management\n\n### VirtualService for A\u002FB Model Testing\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: VirtualService\nmetadata:\n  name: model-server\n  namespace: ai-inference\nspec:\n  hosts:\n  - model-server\n  http:\n  # Route by header for explicit model version selection\n  - match:\n    - headers:\n        x-model-version:\n          exact: \"v2-experimental\"\n    route:\n    - destination:\n        host: model-server\n        subset: v2-experimental\n    timeout: 120s\n  # Route by header for A\u002FB test cohort\n  - match:\n    - headers:\n        x-ab-cohort:\n          exact: \"treatment\"\n    route:\n    - destination:\n        host: model-server\n        subset: v2-experimental\n      weight: 100\n    timeout: 120s\n  # Default traffic split: 90\u002F10 canary\n  - route:\n    - destination:\n        host: model-server\n        subset: v1-stable\n      weight: 90\n    - destination:\n        host: model-server\n        subset: v2-experimental\n      weight: 10\n    timeout: 60s\n    retries:\n      attempts: 2\n      perTryTimeout: 30s\n      retryOn: unavailable,resource-exhausted\n```\n\n### DestinationRule with Subsets\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: model-server\n  namespace: ai-inference\nspec:\n  host: model-server\n  trafficPolicy:\n    connectionPool:\n      http:\n        h2UpgradePolicy: UPGRADE\n        maxRequestsPerConnection: 100\n      tcp:\n        maxConnections: 200\n        connectTimeout: 5s\n    loadBalancer:\n      simple: LEAST_REQUEST\n  subsets:\n  - name: v1-stable\n    labels:\n      version: v1\n    trafficPolicy:\n      connectionPool:\n        http:\n          maxRequestsPerConnection: 50\n  - name: v2-experimental\n    labels:\n      version: v2\n    trafficPolicy:\n      connectionPool:\n        http:\n          maxRequestsPerConnection: 20\n```\n\n### Circuit Breaking for Inference Backends\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: model-server-circuit-breaker\n  namespace: ai-inference\nspec:\n  host: model-server\n  trafficPolicy:\n    connectionPool:\n      tcp:\n        maxConnections: 100\n        connectTimeout: 10s\n      http:\n        http1MaxPendingRequests: 50\n        http2MaxRequests: 200\n        maxRequestsPerConnection: 10\n        maxRetries: 3\n    outlierDetection:\n      consecutive5xxErrors: 3\n      interval: 15s\n      baseEjectionTime: 30s\n      maxEjectionPercent: 50\n      minHealthPercent: 30\n      splitExternalLocalOriginErrors: true\n---\n# Separate circuit breaker for the vector retriever\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: vector-retriever-circuit-breaker\n  namespace: ai-inference\nspec:\n  host: vector-retriever\n  trafficPolicy:\n    connectionPool:\n      tcp:\n        maxConnections: 300\n      http:\n        http1MaxPendingRequests: 200\n        http2MaxRequests: 500\n    outlierDetection:\n      consecutive5xxErrors: 5\n      interval: 10s\n      baseEjectionTime: 15s\n      maxEjectionPercent: 30\n```\n\n### Retry Budget for Streaming Requests\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: VirtualService\nmetadata:\n  name: streaming-inference\n  namespace: ai-inference\nspec:\n  hosts:\n  - model-server\n  http:\n  # Streaming endpoint: no retries, long timeout\n  - match:\n    - uri:\n        prefix: \u002Fv1\u002Fstream\n    route:\n    - destination:\n        host: model-server\n        subset: v1-stable\n    timeout: 300s\n    retries:\n      attempts: 0\n  # Embeddings endpoint: safe to retry, short timeout\n  - match:\n    - uri:\n        prefix: \u002Fv1\u002Fembeddings\n    route:\n    - destination:\n        host: model-server\n        subset: v1-stable\n    timeout: 15s\n    retries:\n      attempts: 3\n      perTryTimeout: 5s\n      retryOn: 5xx,reset,connect-failure,retriable-status-codes\n```\n\n## Resilience\n\n### Locality-Aware Routing\n\n```yaml\napiVersion: networking.istio.io\u002Fv1alpha3\nkind: DestinationRule\nmetadata:\n  name: model-server-locality\n  namespace: ai-inference\nspec:\n  host: model-server\n  trafficPolicy:\n    loadBalancer:\n      localityLbSetting:\n        enabled: true\n        distribute:\n        - from: \"us-east-1\u002Fus-east-1a\u002F*\"\n          to:\n            \"us-east-1\u002Fus-east-1a\u002F*\": 80\n            \"us-east-1\u002Fus-east-1b\u002F*\": 20\n        failover:\n        - from: us-east-1\n          to: us-west-2\n    outlierDetection:\n      consecutive5xxErrors: 3\n      interval: 10s\n      baseEjectionTime: 30s\n```\n\n## Observability\n\n```yaml\n# Telemetry resource for custom metrics on inference services\napiVersion: telemetry.istio.io\u002Fv1alpha1\nkind: Telemetry\nmetadata:\n  name: inference-telemetry\n  namespace: ai-inference\nspec:\n  metrics:\n  - providers:\n    - name: prometheus\n    overrides:\n    - match:\n        metric: REQUEST_DURATION\n        mode: CLIENT_AND_SERVER\n      tagOverrides:\n        model_name:\n          operation: UPSERT\n          value: \"request.headers['x-model-name']\"\n        tenant_id:\n          operation: UPSERT\n          value: \"request.headers['x-tenant-id']\"\n  tracing:\n  - providers:\n    - name: zipkin\n    randomSamplingPercentage: 10.0\n```\n\n### Kiali Dashboard Check\n\n```bash\n# Port-forward Kiali\nkubectl port-forward svc\u002Fkiali -n istio-system 20001:20001 &\n\n# Verify mesh health via API\ncurl -s http:\u002F\u002Flocalhost:20001\u002Fkiali\u002Fapi\u002Fnamespaces\u002Fai-inference\u002Fhealth | jq .\n\n# Check proxy sync status\nistioctl proxy-status -n ai-inference\n\n# Debug a specific pod sidecar config\nistioctl proxy-config routes deploy\u002Fmodel-server -n ai-inference -o json\nistioctl proxy-config cluster deploy\u002Fmodel-server -n ai-inference\n```\n\n## Pitfalls to Avoid\n\n- Aggressive timeouts that break streaming responses -- set 300s+ for generation endpoints\n- Blanket retries that amplify expensive generation calls -- disable retries on non-idempotent routes\n- Missing identity boundaries between tenant-facing and internal services\n- Forgetting to exempt health check and metrics ports from strict mTLS\n- Setting outlier ejection too aggressively on small pools (maxEjectionPercent too high)\n- Not using `holdApplicationUntilProxyStarts` causing race conditions on startup\n\n## Related Skills\n\n- service-mesh (`service-mesh`) - Foundational mesh concepts\n- llm-gateway (`llm-gateway`) - North-south API gateway controls\n- opentelemetry (`opentelemetry`) - End-to-end tracing and metrics\n\n## When to Use\n\n- You are provisioning, configuring, or troubleshooting the infrastructure component covered by this skill (servers, storage, databases, networking, cloud, local AI).\n\n## Limitations\n\n- Infrastructure commands can disrupt services: confirm target host\u002Fscope and have backups\u002Fsnapshots before mutating state.\n- Docs-only import: upstream scripts and templates not bundled.\n\n",{"name":5,"risk":34,"source":43,"license":14,"category":44,"metadata":45,"date_added":48,"description":7,"source_repo":49,"source_type":29,"compatibility":50,"license_source":51},"https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills","devops",{"author":46,"version":47},"devops-skills","1.0","2026-09-20","BagelHole\u002FDevOps-Security-Agent-Skills","Requires the relevant OS\u002Fplatform tooling and privileged access where noted. Docs-only; helper scripts and templates not bundled.","https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE","f3312ba749561b6159f477dbfc9313c875d40a56",".agents\u002Fskills_archive\u002Fai-inference-service-mesh","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612","2026-10-25 23:50:58",{"source":8,"skillName":5,"commands":58,"prompt":68,"zipAvailable":69},[59,62,65],{"tool":60,"command":61},"npx","npx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill ai-inference-service-mesh",{"tool":63,"command":64},"bunx","bunx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill ai-inference-service-mesh",{"tool":66,"command":67},"pnpm","pnpm dlx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill ai-inference-service-mesh","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fai-inference-service-mesh\u002FSKILL.md",true,[],[72,95,119,144,172,196],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":83,"publishedAt":38},"01M6Q0FA5FZ410Z33Z3TFQ4DBA","zero-trust","Implement zero-trust network architecture. Configure identity-based access, micro-segmentation, and continuous verification. Use when implementing modern security architectures.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fzero-trust\u002FSKILL.md",[78],{"categoryNo":79,"categoryName":80,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000011","运维与部署","SCENARIO","使用场景",[84,85,86,87,88,89,74,90,24,26,91,27,29,30,92,34,93,35,36,94],"ZeRO","trust","zero trust","architecture","assessment","verification","segmentation","configure","active","scope","implementing",{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":38},"01M6Q0F1YZJW3KX5FBKEVZ8CV4","zcode-delegate","Delegate coding tasks to the Z.AI ZCode CLI only when the user explicitly requests it, while the orchestrator retains review and landing responsibility.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fzcode-delegate\u002FSKILL.md",[],[102,103,104,105,106,107,108,109,110,111,112,113,114,27,115,29,116,117,34,118],"delegate","zcode","zcode delegate","review","cli","orchestration","tasks","run","Bash","model","node","read","orchestrator","files","desktop app","app","agent-orchestration",{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":123,"categories":124,"keywords":125,"publishedAt":38},"01M6Q0ETK41VGNMYEXW0T00AJN","youtube-transcript","Fetch YouTube transcripts through DeepAPI or local fallback tooling and save clean text output.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-transcript\u002FSKILL.md",[],[126,127,128,129,130,131,132,133,134,135,136,137,138,29,30,139,140,141,142,143],"transcript","youtube transcript","youtube","safe","github","codex","research","license","https","text","fetch","tooling","clean","local","output","deepapi","youtube transcripts","blob",{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":161,"publishedAt":38},"01M6Q0EPPH2PVMDW40017F8WX0","youtube-transcript-skills","Fetch YouTube video transcripts, search videos\u002Fchannels, browse channels, and extract playlists via the getyoutubetranscript.com API - free tier, no card required.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-transcript-skills\u002FSKILL.md",[150,153,156],{"categoryNo":151,"categoryName":152,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":154,"categoryName":155,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000016","图像与多媒体",{"categoryNo":157,"categoryName":158,"taxonomyType":159,"taxonomyTypeDesc":160},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[126,127,128,162,129,163,130,131,133,164,165,166,167,136,168,29,30,169,170,171],"youtube transcript skills","gemini","search","api","Video","browse","api-integration","free","card","channels",{"skillNo":173,"name":174,"nameZh":6,"description":175,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":176,"categories":177,"keywords":182,"publishedAt":38},"01M6Q0EAB96PX83W5PTE8A0R8D","youtube-notetaker","Turn YouTube talks into local study notes with slides, transcripts, editable annotations, and a markdown-backed viewer.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-notetaker\u002FSKILL.md",[178,181],{"categoryNo":179,"categoryName":180,"taxonomyType":81,"taxonomyTypeDesc":82},"01JBSEEDCAT000000000000003","演示文稿",{"categoryNo":154,"categoryName":155,"taxonomyType":81,"taxonomyTypeDesc":82},[128,183,184,106,129,185,186,187,188,133,166,189,190,191,192,30,193,139,194,195],"youtube notetaker","notetaker","workflow","official","notes","code","markdown","slides","annotations","README","study","dair-academy","viewer",{"skillNo":197,"name":198,"nameZh":6,"description":199,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":200,"categories":201,"keywords":203,"publishedAt":38},"01M6Q0E55F0GW5MX48V8HXS8B5","youtube-full","Fetch YouTube transcripts, search videos, browse channels, and extract playlists via TranscriptAPI — no yt-dlp, no Google API key, works from any cloud server.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-full\u002FSKILL.md",[202],{"categoryNo":151,"categoryName":152,"taxonomyType":81,"taxonomyTypeDesc":82},[204,128,205,206,129,163,164,207,208,165,166,167,136,209,168,29,30,169,210,142],"youtube full","full","google","server","dlp","credits","automatic"]