[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6P2682JHJKWWMJFGJSTBS9Y":3,"skill-same:01M6P2682JHJKWWMJFGJSTBS9Y":82},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":33,"commitSha":57,"skillDirectory":58,"forksCount":13,"repositoryHtmlUrl":59,"repositoryHomepageUrl":6,"ownerHtmlUrl":60,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":61,"install":62,"resources":75},"01M6P2682JHJKWWMJFGJSTBS9Y","ebpf-observability",null,"Use eBPF for deep kernel-level observability — trace syscalls, network flows, and application behavior without code changes using Cilium, Tetragon, and bpftrace.","duc-duy-1612\u002FOnline-Delivery-Systems-Web","duc-duy-1612","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F192231384?v=4","01M6NSDKQQDKVTHZ8DZMZ1VYY6","01M6NSDKW8DDVTQHBT1HBDDJPT",0,"MIT","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Febpf-observability\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000011","运维与部署","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"ebpf observability","ebpf","observability","only","flows","relevant","scripts","critical","bpftrace","deep","main","community","security","access","devops-security","kubectl","network","ci","git","code","2026-11-10 10:12:30",false,"---\nname: ebpf-observability\ndescription: Use eBPF for deep kernel-level observability — trace syscalls, network\n  flows, and application behavior without code changes using Cilium, Tetragon, and\n  bpftrace.\ncategory: devops\nrisk: critical\nsource: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\nsource_repo: BagelHole\u002FDevOps-Security-Agent-Skills\nsource_type: community\ndate_added: '2026-09-20'\nlicense: MIT\nlicense_source: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE\ncompatibility: Requires the relevant platform CLIs (kubectl, helm, terraform, git,\n  CI runners) and authorized access to the target environment. Docs-only; helper scripts\n  and templates not bundled.\nmetadata:\n  author: devops-skills\n  version: '1.0'\n---\n\n# eBPF Observability\n\neBPF (extended Berkeley Packet Filter) allows you to run sandboxed programs in the Linux kernel without modifying kernel source code or loading kernel modules. This skill covers using eBPF for deep observability, network monitoring, and security enforcement across cloud-native infrastructure.\n\n---\n\n## 2. Prerequisites\n\n### Kernel Version Requirements\n\n| Feature                  | Minimum Kernel | Recommended Kernel |\n|--------------------------|----------------|--------------------|\n| Basic BPF maps & probes  | 4.9            | 5.10+              |\n| BPF CO-RE (BTF support)  | 5.2            | 5.10+              |\n| BPF ring buffer          | 5.8            | 5.10+              |\n| BPF LSM hooks            | 5.7            | 5.15+              |\n| Cilium full features     | 4.19           | 5.10+              |\n| Tetragon                 | 4.19           | 5.13+              |\n\n### Verify Kernel Support\n\n```bash\n# Check kernel version\nuname -r\n\n# Verify BTF (BPF Type Format) is enabled -- required for CO-RE\nls \u002Fsys\u002Fkernel\u002Fbtf\u002Fvmlinux\n\n# Check BPF filesystem is mounted\nmount | grep bpf\n\n# If not mounted, mount it\nsudo mount -t bpf bpf \u002Fsys\u002Ffs\u002Fbpf\n\n# Verify BPF JIT is enabled\ncat \u002Fproc\u002Fsys\u002Fnet\u002Fcore\u002Fbpf_jit_enable\n# Should return 1; if not:\nsudo sysctl net.core.bpf_jit_enable=1\n```\n\n### Install Toolchain\n\n```bash\n# Ubuntu\u002FDebian -- install bpftrace, bcc tools, and libbpf\nsudo apt-get update\nsudo apt-get install -y bpftrace bpfcc-tools libbpf-dev linux-headers-$(uname -r)\n\n# Fedora\u002FRHEL\nsudo dnf install -y bpftrace bcc-tools libbpf-devel kernel-devel\n\n# Verify bpftrace works\nsudo bpftrace -e 'BEGIN { printf(\"eBPF is working\\n\"); exit(); }'\n```\n\n---\n\n## 3. Cilium Setup\n\nCilium replaces kube-proxy with eBPF-based networking, providing identity-aware security and deep network observability via Hubble.\n\n### Install Cilium on Kubernetes\n\n```bash\n# Add the Cilium Helm repo\nhelm repo add cilium https:\u002F\u002Fhelm.cilium.io\u002F\nhelm repo update\n\n# Install Cilium with Hubble enabled\nhelm install cilium cilium\u002Fcilium --version 1.16.4 \\\n  --namespace kube-system \\\n  --set kubeProxyReplacement=true \\\n  --set k8sServiceHost=\"${API_SERVER_IP}\" \\\n  --set k8sServicePort=\"${API_SERVER_PORT}\" \\\n  --set hubble.enabled=true \\\n  --set hubble.relay.enabled=true \\\n  --set hubble.ui.enabled=true \\\n  --set hubble.metrics.enableOpenMetrics=true \\\n  --set hubble.metrics.enabled=\"{dns,drop,tcp,flow,port-distribution,icmp,httpV2:exemplars=true;labelsContext=source_ip\\,source_namespace\\,source_workload\\,destination_ip\\,destination_namespace\\,destination_workload}\"\n\n# Wait for Cilium to be ready\ncilium status --wait\n```\n\n### Install the Cilium CLI and Hubble CLI\n\n```bash\n# Cilium CLI\nCILIUM_CLI_VERSION=$(curl -s https:\u002F\u002Fraw.githubusercontent.com\u002Fcilium\u002Fcilium-cli\u002Fmain\u002Fstable.txt)\ncurl -L --remote-name \"https:\u002F\u002Fgithub.com\u002Fcilium\u002Fcilium-cli\u002Freleases\u002Fdownload\u002F${CILIUM_CLI_VERSION}\u002Fcilium-linux-amd64.tar.gz\"\nsudo tar xzvf cilium-linux-amd64.tar.gz -C \u002Fusr\u002Flocal\u002Fbin\nrm cilium-linux-amd64.tar.gz\n\n# Hubble CLI\nHUBBLE_VERSION=$(curl -s https:\u002F\u002Fraw.githubusercontent.com\u002Fcilium\u002Fhubble\u002Fmaster\u002Fstable.txt)\ncurl -L --remote-name \"https:\u002F\u002Fgithub.com\u002Fcilium\u002Fhubble\u002Freleases\u002Fdownload\u002F${HUBBLE_VERSION}\u002Fhubble-linux-amd64.tar.gz\"\nsudo tar xzvf hubble-linux-amd64.tar.gz -C \u002Fusr\u002Flocal\u002Fbin\nrm hubble-linux-amd64.tar.gz\n```\n\n### Hubble Network Observability\n\n```bash\n# Port-forward the Hubble Relay\ncilium hubble port-forward &\n\n# Observe all flows in real time\nhubble observe --follow\n\n# Filter flows by namespace\nhubble observe --namespace production --follow\n\n# Filter by verdict (dropped traffic)\nhubble observe --verdict DROPPED --follow\n\n# Filter by DNS queries\nhubble observe --protocol DNS --follow\n\n# Filter HTTP traffic to a specific service\nhubble observe --to-label \"app=api-server\" --protocol HTTP --follow\n\n# Export flows as JSON for ingestion into SIEM\nhubble observe --output json --last 1000 > flows.json\n```\n\n### Hubble UI Access\n\n```bash\n# Port-forward the Hubble UI\nkubectl port-forward -n kube-system svc\u002Fhubble-ui 12000:80\n\n# Access at http:\u002F\u002Flocalhost:12000 -- provides a real-time service dependency map\n```\n\n---\n\n## 4. Tetragon for Security\n\nTetragon is Cilium's runtime security enforcement engine. It uses eBPF to observe and enforce security policies at the kernel level with zero application changes.\n\n### Install Tetragon\n\n```bash\nhelm repo add cilium https:\u002F\u002Fhelm.cilium.io\u002F\nhelm repo update\n\nhelm install tetragon cilium\u002Ftetragon \\\n  --namespace kube-system \\\n  --set tetragon.grpc.enabled=true \\\n  --set tetragon.exportFilename=\u002Fvar\u002Frun\u002Fcilium\u002Ftetragon\u002Ftetragon.log\n\n# Install the tetra CLI\ncurl -LO \"https:\u002F\u002Fgithub.com\u002Fcilium\u002Ftetragon\u002Freleases\u002Flatest\u002Fdownload\u002Ftetra-linux-amd64.tar.gz\"\nsudo tar xzvf tetra-linux-amd64.tar.gz -C \u002Fusr\u002Flocal\u002Fbin\nrm tetra-linux-amd64.tar.gz\n```\n\n### Process Execution Monitoring\n\n```yaml\n# process-monitor.yaml -- TracingPolicy to monitor all process executions\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: process-execution-monitor\nspec:\n  kprobes: []\n  tracepoints: []\n  uprobes: []\n  enforcers: []\n  # process_exec and process_exit events are always emitted by default\n  # Use tetra CLI to observe them:\n```\n\n```bash\n# Watch all process executions cluster-wide\nkubectl exec -n kube-system ds\u002Ftetragon -c tetragon -- tetra getevents -o compact --process-exec\n\n# Filter to a specific namespace\nkubectl exec -n kube-system ds\u002Ftetragon -c tetragon -- tetra getevents -o compact \\\n  --namespace production\n```\n\n### File Access Tracking\n\n```yaml\n# file-access-policy.yaml -- detect reads\u002Fwrites to sensitive files\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: sensitive-file-access\nspec:\n  kprobes:\n    - call: \"security_file_open\"\n      syscall: false\n      args:\n        - index: 0\n          type: \"file\"\n      selectors:\n        - matchArgs:\n            - index: 0\n              operator: \"Prefix\"\n              values:\n                - \"\u002Fetc\u002Fshadow\"\n                - \"\u002Fetc\u002Fpasswd\"\n                - \"\u002Fetc\u002Fkubernetes\u002Fpki\"\n                - \"\u002Fvar\u002Frun\u002Fsecrets\u002Fkubernetes.io\"\n                - \"\u002Froot\u002F.ssh\"\n```\n\n```bash\nkubectl apply -f file-access-policy.yaml\n\n# Observe file access events\nkubectl exec -n kube-system ds\u002Ftetragon -c tetragon -- tetra getevents -o compact \\\n  | grep \"sensitive-file-access\"\n```\n\n### Network Connection Enforcement\n\n```yaml\n# restrict-egress.yaml -- block unexpected outbound connections\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: restrict-egress-connections\nspec:\n  kprobes:\n    - call: \"tcp_connect\"\n      syscall: false\n      args:\n        - index: 0\n          type: \"sock\"\n      selectors:\n        - matchArgs:\n            - index: 0\n              operator: \"DAddr\"\n              values:\n                - \"169.254.169.254\"  # Block IMDS access\n          matchActions:\n            - action: Sigkill\n        - matchNamespaces:\n            - namespace: Mnt\n              operator: NotIn\n              values:\n                - \"host_mnt\"\n```\n\n```bash\nkubectl apply -f restrict-egress.yaml\n```\n\n### Privileged Escalation Detection\n\n```yaml\n# detect-privilege-escalation.yaml\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: detect-privilege-escalation\nspec:\n  kprobes:\n    - call: \"__x64_sys_setuid\"\n      syscall: true\n      args:\n        - index: 0\n          type: \"int\"\n      selectors:\n        - matchArgs:\n            - index: 0\n              operator: \"Equal\"\n              values:\n                - \"0\"\n          matchActions:\n            - action: Post\n              rateLimit: \"1m\"\n    - call: \"__x64_sys_setns\"\n      syscall: true\n      args:\n        - index: 1\n          type: \"int\"\n      selectors:\n        - matchActions:\n            - action: Post\n```\n\n```bash\nkubectl apply -f detect-privilege-escalation.yaml\n```\n\n---\n\n## 5. bpftrace One-Liners\n\nThese are practical bpftrace commands you can run directly in production for targeted debugging.\n\n### Syscall Latency\n\n```bash\n# Trace read() syscall latency distribution (microseconds)\nsudo bpftrace -e 'tracepoint:syscalls:sys_enter_read { @start[tid] = nsecs; }\n  tracepoint:syscalls:sys_exit_read \u002F@start[tid]\u002F {\n    @usecs = hist((nsecs - @start[tid]) \u002F 1000);\n    delete(@start[tid]);\n  }'\n\n# Top 10 slowest syscalls by total time\nsudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @start[tid] = nsecs; }\n  tracepoint:raw_syscalls:sys_exit \u002F@start[tid]\u002F {\n    @ns[probe] = sum(nsecs - @start[tid]);\n    delete(@start[tid]);\n  } END { print(@ns, 10); }'\n```\n\n### DNS Tracing\n\n```bash\n# Trace DNS queries via UDP port 53 sends\nsudo bpftrace -e 'kprobe:udp_sendmsg {\n    $sk = (struct sock *)arg0;\n    $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) \u003C\u003C 8);\n    if ($dport == 53) {\n      printf(\"%-8d %-16s DNS query to %s\\n\", pid, comm,\n        ntop($sk->__sk_common.skc_daddr));\n    }\n  }'\n\n# Count DNS queries by source process\nsudo bpftrace -e 'kprobe:udp_sendmsg {\n    $sk = (struct sock *)arg0;\n    $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) \u003C\u003C 8);\n    if ($dport == 53) { @dns[comm] = count(); }\n  }'\n```\n\n### TCP Retransmits\n\n```bash\n# Trace TCP retransmits with source\u002Fdestination\nsudo bpftrace -e 'kprobe:tcp_retransmit_skb {\n    $sk = (struct sock *)arg0;\n    $daddr = ntop($sk->__sk_common.skc_daddr);\n    $saddr = ntop($sk->__sk_common.skc_rcv_saddr);\n    $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) \u003C\u003C 8);\n    $sport = $sk->__sk_common.skc_num;\n    printf(\"%-20s %-6d -> %-20s %-6d (%s)\\n\", $saddr, $sport, $daddr, $dport, comm);\n  }'\n```\n\n### Disk I\u002FO Latency\n\n```bash\n# Block I\u002FO latency histogram by device\nsudo bpftrace -e 'tracepoint:block:block_rq_issue { @start[args->dev, args->sector] = nsecs; }\n  tracepoint:block:block_rq_complete \u002F@start[args->dev, args->sector]\u002F {\n    @usecs[args->dev] = hist((nsecs - @start[args->dev, args->sector]) \u002F 1000);\n    delete(@start[args->dev, args->sector]);\n  }'\n\n# Top processes by disk I\u002FO bytes\nsudo bpftrace -e 'tracepoint:block:block_rq_issue {\n    @bytes[comm] = sum(args->bytes);\n  } interval:s:5 { print(@bytes, 10); clear(@bytes); }'\n```\n\n### Container-Aware Tracing\n\n```bash\n# Trace process exec inside containers (cgroup-filtered)\nsudo bpftrace -e 'tracepoint:syscalls:sys_enter_execve {\n    printf(\"%-8d %-8d %-16s %s\\n\", pid, cgroup, comm, str(args->filename));\n  }'\n\n# Memory allocation hotspots per container\nsudo bpftrace -e 'kprobe:__alloc_pages { @pages[cgroup] = count(); }\n  interval:s:10 { print(@pages, 10); clear(@pages); }'\n```\n\n---\n\n\n## Contents\n\n- [6. Prometheus Integration](references\u002Fdetails.md)\n- [7. Network Observability](references\u002Fdetails.md)\n- [8. Security Monitoring](references\u002Fdetails.md)\n- [9. Performance Profiling](references\u002Fdetails.md)\n- [10. Troubleshooting](references\u002Fdetails.md)\n\n## When to Use\n\nUse eBPF-based observability when you need:\n\n- **Deep performance debugging** -- trace kernel-level latency, syscall overhead, and scheduling delays that application-level metrics cannot reveal.\n- **Network observability without sidecars** -- capture L3\u002FL4\u002FL7 flows, DNS queries, and TCP state transitions directly from the kernel, eliminating the CPU and memory overhead of sidecar proxies.\n- **Security monitoring at the kernel boundary** -- detect container escapes, unexpected process execution, sensitive file access, and anomalous syscall patterns in real time.\n- **Continuous profiling in production** -- generate CPU flame graphs and memory allocation profiles with negligible overhead (typically under 1% CPU).\n- **Service mesh replacement or augmentation** -- Cilium can replace kube-proxy and provide identity-aware network policies enforced at the kernel level.\n\nAvoid eBPF when your kernel version is below 4.19, when you are running on managed platforms that restrict BPF capabilities, or when your debugging needs are fully met by application-level tracing.\n\n---\n\n## Limitations\n\n- Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.\n- Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.\n\n### Example\n\n```bash\ngit status && git diff --stat\nkubectl diff -f manifest.yaml\n```\n\n> Adapted from [BagelHole\u002FDevOps-Security-Agent-Skills](https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills) (MIT); frontmatter, When to Use\u002FLimitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.\n","# eBPF Observability\n\neBPF (extended Berkeley Packet Filter) allows you to run sandboxed programs in the Linux kernel without modifying kernel source code or loading kernel modules. This skill covers using eBPF for deep observability, network monitoring, and security enforcement across cloud-native infrastructure.\n\n---\n\n## 2. Prerequisites\n\n### Kernel Version Requirements\n\n| Feature                  | Minimum Kernel | Recommended Kernel |\n|--------------------------|----------------|--------------------|\n| Basic BPF maps & probes  | 4.9            | 5.10+              |\n| BPF CO-RE (BTF support)  | 5.2            | 5.10+              |\n| BPF ring buffer          | 5.8            | 5.10+              |\n| BPF LSM hooks            | 5.7            | 5.15+              |\n| Cilium full features     | 4.19           | 5.10+              |\n| Tetragon                 | 4.19           | 5.13+              |\n\n### Verify Kernel Support\n\n```bash\n# Check kernel version\nuname -r\n\n# Verify BTF (BPF Type Format) is enabled -- required for CO-RE\nls \u002Fsys\u002Fkernel\u002Fbtf\u002Fvmlinux\n\n# Check BPF filesystem is mounted\nmount | grep bpf\n\n# If not mounted, mount it\nsudo mount -t bpf bpf \u002Fsys\u002Ffs\u002Fbpf\n\n# Verify BPF JIT is enabled\ncat \u002Fproc\u002Fsys\u002Fnet\u002Fcore\u002Fbpf_jit_enable\n# Should return 1; if not:\nsudo sysctl net.core.bpf_jit_enable=1\n```\n\n### Install Toolchain\n\n```bash\n# Ubuntu\u002FDebian -- install bpftrace, bcc tools, and libbpf\nsudo apt-get update\nsudo apt-get install -y bpftrace bpfcc-tools libbpf-dev linux-headers-$(uname -r)\n\n# Fedora\u002FRHEL\nsudo dnf install -y bpftrace bcc-tools libbpf-devel kernel-devel\n\n# Verify bpftrace works\nsudo bpftrace -e 'BEGIN { printf(\"eBPF is working\\n\"); exit(); }'\n```\n\n---\n\n## 3. Cilium Setup\n\nCilium replaces kube-proxy with eBPF-based networking, providing identity-aware security and deep network observability via Hubble.\n\n### Install Cilium on Kubernetes\n\n```bash\n# Add the Cilium Helm repo\nhelm repo add cilium https:\u002F\u002Fhelm.cilium.io\u002F\nhelm repo update\n\n# Install Cilium with Hubble enabled\nhelm install cilium cilium\u002Fcilium --version 1.16.4 \\\n  --namespace kube-system \\\n  --set kubeProxyReplacement=true \\\n  --set k8sServiceHost=\"${API_SERVER_IP}\" \\\n  --set k8sServicePort=\"${API_SERVER_PORT}\" \\\n  --set hubble.enabled=true \\\n  --set hubble.relay.enabled=true \\\n  --set hubble.ui.enabled=true \\\n  --set hubble.metrics.enableOpenMetrics=true \\\n  --set hubble.metrics.enabled=\"{dns,drop,tcp,flow,port-distribution,icmp,httpV2:exemplars=true;labelsContext=source_ip\\,source_namespace\\,source_workload\\,destination_ip\\,destination_namespace\\,destination_workload}\"\n\n# Wait for Cilium to be ready\ncilium status --wait\n```\n\n### Install the Cilium CLI and Hubble CLI\n\n```bash\n# Cilium CLI\nCILIUM_CLI_VERSION=$(curl -s https:\u002F\u002Fraw.githubusercontent.com\u002Fcilium\u002Fcilium-cli\u002Fmain\u002Fstable.txt)\ncurl -L --remote-name \"https:\u002F\u002Fgithub.com\u002Fcilium\u002Fcilium-cli\u002Freleases\u002Fdownload\u002F${CILIUM_CLI_VERSION}\u002Fcilium-linux-amd64.tar.gz\"\nsudo tar xzvf cilium-linux-amd64.tar.gz -C \u002Fusr\u002Flocal\u002Fbin\nrm cilium-linux-amd64.tar.gz\n\n# Hubble CLI\nHUBBLE_VERSION=$(curl -s https:\u002F\u002Fraw.githubusercontent.com\u002Fcilium\u002Fhubble\u002Fmaster\u002Fstable.txt)\ncurl -L --remote-name \"https:\u002F\u002Fgithub.com\u002Fcilium\u002Fhubble\u002Freleases\u002Fdownload\u002F${HUBBLE_VERSION}\u002Fhubble-linux-amd64.tar.gz\"\nsudo tar xzvf hubble-linux-amd64.tar.gz -C \u002Fusr\u002Flocal\u002Fbin\nrm hubble-linux-amd64.tar.gz\n```\n\n### Hubble Network Observability\n\n```bash\n# Port-forward the Hubble Relay\ncilium hubble port-forward &\n\n# Observe all flows in real time\nhubble observe --follow\n\n# Filter flows by namespace\nhubble observe --namespace production --follow\n\n# Filter by verdict (dropped traffic)\nhubble observe --verdict DROPPED --follow\n\n# Filter by DNS queries\nhubble observe --protocol DNS --follow\n\n# Filter HTTP traffic to a specific service\nhubble observe --to-label \"app=api-server\" --protocol HTTP --follow\n\n# Export flows as JSON for ingestion into SIEM\nhubble observe --output json --last 1000 > flows.json\n```\n\n### Hubble UI Access\n\n```bash\n# Port-forward the Hubble UI\nkubectl port-forward -n kube-system svc\u002Fhubble-ui 12000:80\n\n# Access at http:\u002F\u002Flocalhost:12000 -- provides a real-time service dependency map\n```\n\n---\n\n## 4. Tetragon for Security\n\nTetragon is Cilium's runtime security enforcement engine. It uses eBPF to observe and enforce security policies at the kernel level with zero application changes.\n\n### Install Tetragon\n\n```bash\nhelm repo add cilium https:\u002F\u002Fhelm.cilium.io\u002F\nhelm repo update\n\nhelm install tetragon cilium\u002Ftetragon \\\n  --namespace kube-system \\\n  --set tetragon.grpc.enabled=true \\\n  --set tetragon.exportFilename=\u002Fvar\u002Frun\u002Fcilium\u002Ftetragon\u002Ftetragon.log\n\n# Install the tetra CLI\ncurl -LO \"https:\u002F\u002Fgithub.com\u002Fcilium\u002Ftetragon\u002Freleases\u002Flatest\u002Fdownload\u002Ftetra-linux-amd64.tar.gz\"\nsudo tar xzvf tetra-linux-amd64.tar.gz -C \u002Fusr\u002Flocal\u002Fbin\nrm tetra-linux-amd64.tar.gz\n```\n\n### Process Execution Monitoring\n\n```yaml\n# process-monitor.yaml -- TracingPolicy to monitor all process executions\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: process-execution-monitor\nspec:\n  kprobes: []\n  tracepoints: []\n  uprobes: []\n  enforcers: []\n  # process_exec and process_exit events are always emitted by default\n  # Use tetra CLI to observe them:\n```\n\n```bash\n# Watch all process executions cluster-wide\nkubectl exec -n kube-system ds\u002Ftetragon -c tetragon -- tetra getevents -o compact --process-exec\n\n# Filter to a specific namespace\nkubectl exec -n kube-system ds\u002Ftetragon -c tetragon -- tetra getevents -o compact \\\n  --namespace production\n```\n\n### File Access Tracking\n\n```yaml\n# file-access-policy.yaml -- detect reads\u002Fwrites to sensitive files\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: sensitive-file-access\nspec:\n  kprobes:\n    - call: \"security_file_open\"\n      syscall: false\n      args:\n        - index: 0\n          type: \"file\"\n      selectors:\n        - matchArgs:\n            - index: 0\n              operator: \"Prefix\"\n              values:\n                - \"\u002Fetc\u002Fshadow\"\n                - \"\u002Fetc\u002Fpasswd\"\n                - \"\u002Fetc\u002Fkubernetes\u002Fpki\"\n                - \"\u002Fvar\u002Frun\u002Fsecrets\u002Fkubernetes.io\"\n                - \"\u002Froot\u002F.ssh\"\n```\n\n```bash\nkubectl apply -f file-access-policy.yaml\n\n# Observe file access events\nkubectl exec -n kube-system ds\u002Ftetragon -c tetragon -- tetra getevents -o compact \\\n  | grep \"sensitive-file-access\"\n```\n\n### Network Connection Enforcement\n\n```yaml\n# restrict-egress.yaml -- block unexpected outbound connections\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: restrict-egress-connections\nspec:\n  kprobes:\n    - call: \"tcp_connect\"\n      syscall: false\n      args:\n        - index: 0\n          type: \"sock\"\n      selectors:\n        - matchArgs:\n            - index: 0\n              operator: \"DAddr\"\n              values:\n                - \"169.254.169.254\"  # Block IMDS access\n          matchActions:\n            - action: Sigkill\n        - matchNamespaces:\n            - namespace: Mnt\n              operator: NotIn\n              values:\n                - \"host_mnt\"\n```\n\n```bash\nkubectl apply -f restrict-egress.yaml\n```\n\n### Privileged Escalation Detection\n\n```yaml\n# detect-privilege-escalation.yaml\napiVersion: cilium.io\u002Fv1alpha1\nkind: TracingPolicy\nmetadata:\n  name: detect-privilege-escalation\nspec:\n  kprobes:\n    - call: \"__x64_sys_setuid\"\n      syscall: true\n      args:\n        - index: 0\n          type: \"int\"\n      selectors:\n        - matchArgs:\n            - index: 0\n              operator: \"Equal\"\n              values:\n                - \"0\"\n          matchActions:\n            - action: Post\n              rateLimit: \"1m\"\n    - call: \"__x64_sys_setns\"\n      syscall: true\n      args:\n        - index: 1\n          type: \"int\"\n      selectors:\n        - matchActions:\n            - action: Post\n```\n\n```bash\nkubectl apply -f detect-privilege-escalation.yaml\n```\n\n---\n\n## 5. bpftrace One-Liners\n\nThese are practical bpftrace commands you can run directly in production for targeted debugging.\n\n### Syscall Latency\n\n```bash\n# Trace read() syscall latency distribution (microseconds)\nsudo bpftrace -e 'tracepoint:syscalls:sys_enter_read { @start[tid] = nsecs; }\n  tracepoint:syscalls:sys_exit_read \u002F@start[tid]\u002F {\n    @usecs = hist((nsecs - @start[tid]) \u002F 1000);\n    delete(@start[tid]);\n  }'\n\n# Top 10 slowest syscalls by total time\nsudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @start[tid] = nsecs; }\n  tracepoint:raw_syscalls:sys_exit \u002F@start[tid]\u002F {\n    @ns[probe] = sum(nsecs - @start[tid]);\n    delete(@start[tid]);\n  } END { print(@ns, 10); }'\n```\n\n### DNS Tracing\n\n```bash\n# Trace DNS queries via UDP port 53 sends\nsudo bpftrace -e 'kprobe:udp_sendmsg {\n    $sk = (struct sock *)arg0;\n    $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) \u003C\u003C 8);\n    if ($dport == 53) {\n      printf(\"%-8d %-16s DNS query to %s\\n\", pid, comm,\n        ntop($sk->__sk_common.skc_daddr));\n    }\n  }'\n\n# Count DNS queries by source process\nsudo bpftrace -e 'kprobe:udp_sendmsg {\n    $sk = (struct sock *)arg0;\n    $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) \u003C\u003C 8);\n    if ($dport == 53) { @dns[comm] = count(); }\n  }'\n```\n\n### TCP Retransmits\n\n```bash\n# Trace TCP retransmits with source\u002Fdestination\nsudo bpftrace -e 'kprobe:tcp_retransmit_skb {\n    $sk = (struct sock *)arg0;\n    $daddr = ntop($sk->__sk_common.skc_daddr);\n    $saddr = ntop($sk->__sk_common.skc_rcv_saddr);\n    $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) \u003C\u003C 8);\n    $sport = $sk->__sk_common.skc_num;\n    printf(\"%-20s %-6d -> %-20s %-6d (%s)\\n\", $saddr, $sport, $daddr, $dport, comm);\n  }'\n```\n\n### Disk I\u002FO Latency\n\n```bash\n# Block I\u002FO latency histogram by device\nsudo bpftrace -e 'tracepoint:block:block_rq_issue { @start[args->dev, args->sector] = nsecs; }\n  tracepoint:block:block_rq_complete \u002F@start[args->dev, args->sector]\u002F {\n    @usecs[args->dev] = hist((nsecs - @start[args->dev, args->sector]) \u002F 1000);\n    delete(@start[args->dev, args->sector]);\n  }'\n\n# Top processes by disk I\u002FO bytes\nsudo bpftrace -e 'tracepoint:block:block_rq_issue {\n    @bytes[comm] = sum(args->bytes);\n  } interval:s:5 { print(@bytes, 10); clear(@bytes); }'\n```\n\n### Container-Aware Tracing\n\n```bash\n# Trace process exec inside containers (cgroup-filtered)\nsudo bpftrace -e 'tracepoint:syscalls:sys_enter_execve {\n    printf(\"%-8d %-8d %-16s %s\\n\", pid, cgroup, comm, str(args->filename));\n  }'\n\n# Memory allocation hotspots per container\nsudo bpftrace -e 'kprobe:__alloc_pages { @pages[cgroup] = count(); }\n  interval:s:10 { print(@pages, 10); clear(@pages); }'\n```\n\n---\n\n\n## Contents\n\n- [6. Prometheus Integration](references\u002Fdetails.md)\n- [7. Network Observability](references\u002Fdetails.md)\n- [8. Security Monitoring](references\u002Fdetails.md)\n- [9. Performance Profiling](references\u002Fdetails.md)\n- [10. Troubleshooting](references\u002Fdetails.md)\n\n## When to Use\n\nUse eBPF-based observability when you need:\n\n- **Deep performance debugging** -- trace kernel-level latency, syscall overhead, and scheduling delays that application-level metrics cannot reveal.\n- **Network observability without sidecars** -- capture L3\u002FL4\u002FL7 flows, DNS queries, and TCP state transitions directly from the kernel, eliminating the CPU and memory overhead of sidecar proxies.\n- **Security monitoring at the kernel boundary** -- detect container escapes, unexpected process execution, sensitive file access, and anomalous syscall patterns in real time.\n- **Continuous profiling in production** -- generate CPU flame graphs and memory allocation profiles with negligible overhead (typically under 1% CPU).\n- **Service mesh replacement or augmentation** -- Cilium can replace kube-proxy and provide identity-aware network policies enforced at the kernel level.\n\nAvoid eBPF when your kernel version is below 4.19, when you are running on managed platforms that restrict BPF capabilities, or when your debugging needs are fully met by application-level tracing.\n\n---\n\n## Limitations\n\n- Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.\n- Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.\n\n### Example\n\n```bash\ngit status && git diff --stat\nkubectl diff -f manifest.yaml\n```\n\n> Adapted from [BagelHole\u002FDevOps-Security-Agent-Skills](https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills) (MIT); frontmatter, When to Use\u002FLimitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.\n",{"name":5,"risk":30,"source":48,"license":14,"category":49,"metadata":50,"date_added":53,"description":7,"source_repo":54,"source_type":34,"compatibility":55,"license_source":56},"https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills","devops",{"author":51,"version":52},"devops-skills","1.0","2026-09-20","BagelHole\u002FDevOps-Security-Agent-Skills","Requires the relevant platform CLIs (kubectl, helm, terraform, git, CI runners) and authorized access to the target environment. Docs-only; helper scripts and templates not bundled.","https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE","f3312ba749561b6159f477dbfc9313c875d40a56",".agents\u002Fskills_archive\u002Febpf-observability","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612","2026-10-25 23:50:58",{"source":8,"skillName":5,"commands":63,"prompt":73,"zipAvailable":74},[64,67,70],{"tool":65,"command":66},"npx","npx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill ebpf-observability",{"tool":68,"command":69},"bunx","bunx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill ebpf-observability",{"tool":71,"command":72},"pnpm","pnpm dlx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill ebpf-observability","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Febpf-observability\u002FSKILL.md",true,[76],{"path":77,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":80,"sourceHtmlUrl":81},"references\u002Fdetails.md","REFERENCE","参考文档",13516,"https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Febpf-observability\u002Freferences\u002Fdetails.md",[83,102,126,151,179,202],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":90,"publishedAt":43},"01M6Q0FA5FZ410Z33Z3TFQ4DBA","zero-trust","Implement zero-trust network architecture. Configure identity-based access, micro-segmentation, and continuous verification. Use when implementing modern security architectures.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fzero-trust\u002FSKILL.md",[89],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},[91,92,93,94,95,96,85,97,37,36,98,26,34,33,99,30,100,29,28,101],"ZeRO","trust","zero trust","architecture","assessment","verification","segmentation","configure","active","scope","implementing",{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":106,"categories":107,"keywords":108,"publishedAt":43},"01M6Q0F1YZJW3KX5FBKEVZ8CV4","zcode-delegate","Delegate coding tasks to the Z.AI ZCode CLI only when the user explicitly requests it, while the orchestrator retains review and landing responsibility.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fzcode-delegate\u002FSKILL.md",[],[109,110,111,112,113,114,115,116,117,118,119,120,121,26,122,34,123,124,30,125],"delegate","zcode","zcode delegate","review","cli","orchestration","tasks","run","Bash","model","node","read","orchestrator","files","desktop app","app","agent-orchestration",{"skillNo":127,"name":128,"nameZh":6,"description":129,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":130,"categories":131,"keywords":132,"publishedAt":43},"01M6Q0ETK41VGNMYEXW0T00AJN","youtube-transcript","Fetch YouTube transcripts through DeepAPI or local fallback tooling and save clean text output.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-transcript\u002FSKILL.md",[],[133,134,135,136,137,138,139,140,141,142,143,144,145,34,33,146,147,148,149,150],"transcript","youtube transcript","youtube","safe","github","codex","research","license","https","text","fetch","tooling","clean","local","output","deepapi","youtube transcripts","blob",{"skillNo":152,"name":153,"nameZh":6,"description":154,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":155,"categories":156,"keywords":168,"publishedAt":43},"01M6Q0EPPH2PVMDW40017F8WX0","youtube-transcript-skills","Fetch YouTube video transcripts, search videos\u002Fchannels, browse channels, and extract playlists via the getyoutubetranscript.com API - free tier, no card required.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-transcript-skills\u002FSKILL.md",[157,160,163],{"categoryNo":158,"categoryName":159,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":161,"categoryName":162,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000016","图像与多媒体",{"categoryNo":164,"categoryName":165,"taxonomyType":166,"taxonomyTypeDesc":167},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[133,134,135,169,136,170,137,138,140,171,172,173,174,143,175,34,33,176,177,178],"youtube transcript skills","gemini","search","api","Video","browse","api-integration","free","card","channels",{"skillNo":180,"name":181,"nameZh":6,"description":182,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":183,"categories":184,"keywords":189,"publishedAt":43},"01M6Q0EAB96PX83W5PTE8A0R8D","youtube-notetaker","Turn YouTube talks into local study notes with slides, transcripts, editable annotations, and a markdown-backed viewer.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-notetaker\u002FSKILL.md",[185,188],{"categoryNo":186,"categoryName":187,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000003","演示文稿",{"categoryNo":161,"categoryName":162,"taxonomyType":20,"taxonomyTypeDesc":21},[135,190,191,113,136,192,193,194,42,140,173,195,196,197,198,33,199,146,200,201],"youtube notetaker","notetaker","workflow","official","notes","markdown","slides","annotations","README","study","dair-academy","viewer",{"skillNo":203,"name":204,"nameZh":6,"description":205,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":206,"categories":207,"keywords":209,"publishedAt":43},"01M6Q0E55F0GW5MX48V8HXS8B5","youtube-full","Fetch YouTube transcripts, search videos, browse channels, and extract playlists via TranscriptAPI — no yt-dlp, no Google API key, works from any cloud server.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-full\u002FSKILL.md",[208],{"categoryNo":158,"categoryName":159,"taxonomyType":20,"taxonomyTypeDesc":21},[210,135,211,212,136,170,171,213,214,172,173,174,143,215,175,34,33,176,216,149],"youtube full","full","google","server","dlp","credits","automatic"]