[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6ACBJQMZV0S42P4WMCFFPGP":3,"skill-same:01M6ACBJQMZV0S42P4WMCFFPGP":51},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":23,"linkOnly":24,"markdownContent":25,"markdownBody":26,"frontMatter":27,"refName":28,"commitSha":29,"skillDirectory":30,"forksCount":31,"repositoryHtmlUrl":32,"repositoryHomepageUrl":33,"ownerHtmlUrl":34,"ownerBlogUrl":35,"ownerDescription":6,"updatedAt":36,"install":37,"resources":50},"01M6ACBJQMZV0S42P4WMCFFPGP","new-monitoring-feature",null,"Add a new monitoring target \u002F layer to SkyWalking OAP. Orients you to the OAL \u002F MAL \u002F LAL \u002F TAL \u002F SpanListener extension points, the UI template + submodule touchpoints, the docs set that must move together, and the cross-cutting traps that don't live in any one skill.","apache\u002Fskywalking","apache","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F47359?v=4","01M6AC83E6K4FDKK7HZ8WNMQKE","The Apache Software Foundation","01M6ACB19NRCBY51M74TPEM71Y",25002,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Fnew-monitoring-feature\u002FSKILL.md",[],[19,20,21,22],"new monitoring feature","skywalking","docs","extension points","2026-11-10 10:12:30",false,"---\nname: new-monitoring-feature\ndescription: Add a new monitoring target \u002F layer to SkyWalking OAP. Orients you to the OAL \u002F MAL \u002F LAL \u002F TAL \u002F SpanListener extension points, the UI template + submodule touchpoints, the docs set that must move together, and the cross-cutting traps that don't live in any one skill.\n---\n\n# Adding a New Monitoring Feature \u002F Layer\n\nSkyWalking monitoring for a new target (a new `Layer` — mobile OS, gateway, DB, language runtime, …) is composed from the same set of reusable extension points. This skill is the index: which extension point fits which signal, where its contract lives, which docs must be updated together, and which harness-wide traps the extension-specific docs don't cover.\n\nThere is one skill per narrow concern. This one is the wiring map.\n\n---\n\n## Working posture\n\n- **Default scope for a new monitoring feature is narrow.** For most new monitoring targets, traces are **re-used as-is** — you do not write or modify a trace analyzer. Metrics and log extensions via OTLP (MAL + LAL) are the common shape. OTLP \u002F Zipkin span-based metrics analysis (`SpanListener`) already exists and only needs a new implementation if your feature introduces genuinely new semantic-convention attributes to extract (e.g. `gen_ai.*`, `mcp.*` for the GenAI layer). Do **not** touch the native trace rules (`tal\u002Fnative-*.yaml`) unless the OAP core protocol gains a new `SpanLayer` enum value — details in §2.\n- **If you find yourself editing more than this skill suggests, stop and confirm.** Scope creep on trace analysis, span-layer mapping, receiver handlers, or protobuf\u002Fproto contracts is a strong signal the framing is wrong. Ask the developer before committing that direction.\n- **Verify every new feature locally before pushing to CI.** Compile + checkstyle + license-eye are minimums, but they only tell you the code builds — they do **not** tell you the new MAL rule compiles its closures at startup, the LAL layer:auto dispatch works, the OTLP payload shape matches the MAL labels, the UI template renders against real data, or the layer shows up in `swctl service ly \u003CLAYER>`. Run the new e2e case end-to-end locally against a live OAP (see [run-e2e skill](..\u002Frun-e2e\u002FSKILL.md)) and fire every verify step with `swctl` by hand at least once. Pushing to CI without a local green run wastes shared CI cycles and slow-loops review.\n\n---\n\n## 0. Register the `Layer` — the feature's entry point\n\nA `Layer` is how OAP slices services \u002F instances \u002F endpoints by data source. The UI, storage partitioning, menu navigation, and OAL aggregation all key off it.\n\n`Layer` is a registry-backed value type (no longer a closed enum). Built-in layers are declared as `public static final Layer` constants in `oap-server\u002Fserver-core\u002Fsrc\u002Fmain\u002Fjava\u002Forg\u002Fapache\u002Fskywalking\u002Foap\u002Fserver\u002Fcore\u002Fanalysis\u002FLayer.java`; external layers are registered through the `Layer.register(name, ordinal, normal)` API at boot. **Pick the registration path that matches the scope of your feature:**\n\n| Your feature ships as | Registration path |\n|---|---|\n| Part of the OAP distribution (in-tree, the common case for new SkyWalking-supported targets) | Add a `public static final Layer` constant to `Layer.java` with the next sequential ordinal in `0–49`. Examples: `IOS = register(\"IOS\", 47, true)`, `APISIX = register(\"APISIX\", 21, true)`, `VIRTUAL_DATABASE = register(\"VIRTUAL_DATABASE\", 14, false)` for inferred\u002Fnon-real services. |\n| An out-of-tree MAL or LAL rule file | Add a top-level `layerDefinitions:` block to the rule file. The DSL loader funnels each entry through `Layer.register` before compiling the rule. One file ships the layer + the rules that produce its telemetry. |\n| An out-of-tree plugin module (jar) | Implement `org.apache.skywalking.oap.server.core.analysis.LayerExtension` and register via `META-INF\u002Fservices\u002F`. Discovered by `LayerExtensionLoader` during `CoreModuleProvider.prepare()`. |\n| Operator-deployed config (no code, no DSL) | Add an entry to `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Flayer-extensions.yml` (or override on the OAP node's classpath). |\n\n**Ordinal conventions:** `0–49` is in active use by built-ins. `50–999` is reserved by convention for future built-in layers. External layers are recommended (not required) to start at `>= 1000` to avoid colliding with future built-ins on OAP upgrade. Collisions in either direction are detected at boot via the ordinal-uniqueness check, which fails OAP startup loudly.\n\n**Storage encoding is the ordinal int**, persisted in BanyanDB \u002F Elasticsearch \u002F JDBC. Every OAP node that reads or writes a given layer must agree on its `(name, ordinal)` mapping — deploy `layer-extensions.yml` and any `layerDefinitions:` rule files identically across all nodes. The registry is sealed at the start of `Core.notifyAfterCompleted()`; later registration attempts throw.\n\n**UI template folders are auto-discovered by file scan, not by `Layer.values()`.** `UITemplateInitializer` walks `ui-initialized-templates\u002F**\u002F*.json` recursively (depth 2) and trusts each template's own `configuration.layer` field. Drop a folder of dashboard JSONs on disk and the initializer picks them up on the next boot — folder name is purely organizational.\n\n**Component ID lookup in Java code**: IDs declared in `component-libraries.yml` are loaded at runtime into `ComponentLibraryCatalogService`'s `componentName2Id` map — they are **not** exposed as Java enum constants. To look up by name in listener code, inject the catalog service and resolve once at construction:\n```java\nIComponentLibraryCatalogService catalog = moduleManager.find(CoreModule.NAME)\n    .provider().getService(IComponentLibraryCatalogService.class);\nint myComponentId = catalog.getComponentId(\"My-Component-Name\");\n```\nCache as an `int` field; runtime comparisons are then plain `componentId == myComponentId`. **Trap:** there is a `ComponentsDefine` class under `skywalking-trace-receiver-plugin\u002Fsrc\u002Ftest\u002Fjava\u002F...\u002Fmock\u002FComponentsDefine.java` — it is a test-only mock holding five hand-picked constants (Tomcat, Dubbo, RocketMQ, MongoDB). Do not import or extend it from production code.\n\nEmit the layer from every source object your feature produces. Built-in layers have a static-field accessor; external layers are looked up by name through the registry:\n\n```java\n\u002F\u002F Built-in layer (constant)\nservice.setLayer(Layer.IOS);\n\n\u002F\u002F External layer (registered via yaml \u002F SPI \u002F layerDefinitions:)\nservice.setLayer(Layer.nameOf(\"IOT_FLEET\"));\n```\n\nDownstream (the core OAL, `service ly \u003CLAYER>` swctl query, topology filters, UI root dashboard's layer selector) all work off this single enum value.\n\n---\n\n## 1. Pick the right extension point\n\n| Signal you want to process | Extension point | Register via |\n|---|---|---|\n| **SkyWalking native trace segments** (from language agents) | TAL rules with `inputType: sw-native` — `tal\u002Fnative-*.yaml`, run by `trace-analyzer` | Append the file to `nativeTalFiles` (`SW_TRACE_NATIVE_TAL_FILES`); order matters |\n| **OTLP trace spans of a layer** (OTel SDKs that set `service.layer`) | TAL rules with `inputType: otlp` — `tal\u002F*.yaml` | Append the file to `talFiles` (`SW_TRACE_TAL_FILES`) |\n| **OTLP \u002F Zipkin trace spans** needing Java (iOS\u002FAndroid, GenAI, …) | `SpanListener` in `core.trace` | `META-INF\u002Fservices\u002Forg.apache.skywalking.oap.server.core.trace.SpanListener` |\n| **OTel metrics** (Prometheus scrape, Envoy stats, Istio, …) | MAL rules — `otel-rules\u002F\u003Cnamespace>\u002F*.yaml` | Append the rule path to `enabledOtelMetricsRules` in `application.yml` |\n| **Logs** (OTLP, SkyWalking native, Fluent Bit, Envoy ALS) | LAL rules — `lal\u002F\u003Crule>.yaml` | Append to `SW_LOG_LAL_FILES` (env) or `lalFiles` (YAML) under `log-analyzer` |\n| **Pre-computed meter \u002F custom meter** (custom agent protocol, meter receiver) | MAL `meter-analyzer-config\u002F*.yaml` | Append to `meter-analyzer-config` entries |\n| **Aggregation from Source entities** (service, instance, endpoint, relation) | OAL rules — `oal\u002F*.oal` | Already loaded by `OAL grammar`; just add lines |\n\nThe DSLs (OAL, MAL, LAL, TAL, Hierarchy) compile via ANTLR4 + Javassist at startup. Source-of-truth details are in:\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Foal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Foal.md)\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Ftal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal.md), [`tal-sw-native.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal-sw-native.md), [`tal-otlp.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal-otlp.md); the [`tal` skill](..\u002Ftal\u002FSKILL.md) covers compiling, inspecting and verifying TAL rules\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Fmal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Fmal.md)\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Flal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Flal.md)\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Fscope-definitions.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Fscope-definitions.md)\n- [`oap-server\u002Fanalyzer\u002Fmeter-analyzer\u002FCLAUDE.md`](..\u002F..\u002F..\u002Foap-server\u002Fanalyzer\u002Fmeter-analyzer\u002FCLAUDE.md) — MAL compiler internals\n- [`oap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md`](..\u002F..\u002F..\u002Foap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md) — LAL compiler internals\n\n---\n\n## 2. Trace ingestion — two parallel pipelines\n\nSkyWalking OAP ingests traces through two distinct entry points. Pick based on source.\n\n> **Before writing any trace extension — ask \"do I actually need one?\"**\n>\n> For **most new layers you do not touch trace analysis at all.** Native segments\n> flow through the existing native rules: `native-rpc-exit` (`tal\u002Fnative-rpc.yaml`) produces\n> the outbound `ServiceRelation` edges, the backend-side agent produces the inbound\n> metrics via `native-rpc-entry`; OTLP spans go through the OTLP TAL rules of `talFiles` instead, and the new-layer Service \u002F Instance \u002F Endpoint\n> entities come from **MAL + LAL**, not from trace analysis. This is how mobile mini-programs\n> (`Layer.WECHAT_MINI_PROGRAM` \u002F `Layer.ALIPAY_MINI_PROGRAM`) and most scraped-metric layers\n> work. Browser (`Layer.BROWSER`) has its own receiver and OAL sources (`browser.oal`), and iOS\n> (`Layer.IOS`) has its own OTLP span listeners: `IOSHTTPSpanListener` sends core OAL sources,\n> `IOSMetricKitSpanListener` sends MAL samples, and LAL reads its diagnostics.\n>\n> Extend trace analysis **only** when the feature genuinely needs per-span\n> interpretation that the existing pipeline cannot express:\n>\n> - **A new `SpanLayer`** — OAP core protobuf has a new `SpanLayer` enum value that\n>   the native rules' layer branches must dispatch on.\n>   That means touching `tal\u002Fnative-rpc.yaml` and its siblings.\n> - **A genuinely new signal class not covered by existing rules or listeners** — e.g. the GenAI\n>   work added `GenAISpanListener` for LLM \u002F MCP instrumentation because OTLP spans\n>   carry semantic-convention attributes (`gen_ai.*`, `mcp.*`) that no prior listener\n>   extracted. That is rare.\n>\n> **Anti-pattern we've been bitten by:** using a componentId mapping in\n> the native trace analysis to assign a client-side layer to exit-only segments. This\n> does not produce inbound metrics anyway (no entry spans → no\n> `service_cpm` \u002F `endpoint_cpm` from OAL), and it is orthogonal to the Service entity\n> layer (which MAL \u002F LAL sets). Mirror the mini-program pattern instead: ship `Layer` +\n> `component-libraries.yml` + MAL + LAL + dashboards, and leave the native rules\n> alone.\n\n### 2.1 SkyWalking native segments — TAL `sw-native` rules\n\n**Source protocol**: `apm-protocol\u002Fapm-network\u002Fsrc\u002Fmain\u002Fproto\u002Flanguage-agent\u002FTracing.proto` (`SegmentObject`).\n\n**Receiver**: `oap-server\u002Fserver-receiver-plugin\u002Fskywalking-trace-receiver-plugin` (and the Kafka fetcher). After wire decoding, segments go to `ISegmentParserService` of `trace-analyzer`, whose `SwNativeRuleRunner` runs every rule of every file in `nativeTalFiles`, in list order, on every span of the segment.\n\n**Extension**: a TAL rule file under `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Ftal\u002F` with `inputType: sw-native`, appended to `nativeTalFiles` in `application.yml`. Outputs are made inside `build {}`: `source \u003CScope> {}` for the existing sources (`core.oal` metrics, records such as `DatabaseSlowStatement`, or a module's own source, as `ai-evaluation` takes `GenAIEvaluationSample` through its `SourceDispatcher`), `metrics {}` for new meter metrics. A file that needs a module (its functions or sources) lists it under `requiredModules`. Reference files: `tal\u002Fnative-rpc.yaml` (services, endpoints, relations), `tal\u002Fnative-virtual-service.yaml` (virtual DB\u002Fcache\u002FMQ), `tal\u002Fnative-virtual-genai.yaml`. Language and runner: [`docs\u002Fen\u002Fconcepts-and-designs\u002Ftal-sw-native.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal-sw-native.md); file order and loggers: [`docs\u002Fen\u002Fsetup\u002Fbackend\u002Fnative-trace-analysis.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fsetup\u002Fbackend\u002Fnative-trace-analysis.md).\n\nA module contributes functions to TAL rules through `TALFunctionProvider` (`server-core`, `supportsInputType(...)` naming the inputs it serves, registered in `META-INF\u002Fservices\u002Forg.apache.skywalking.oap.server.core.source.TALFunctionProvider`), as `ai-evaluation` does for `native-ai-evaluation` and `otlp-ai-evaluation`.\n\nUse when: the data arrives on the SW native wire (language agents — Java, Python, Go, Node.js, .NET, PHP, Lua, etc.).\n\n### 2.2 OTLP \u002F Zipkin spans — `SpanListener`\n\n**Source protocol**: OTLP proto (`opentelemetry\u002Fproto\u002Ftrace\u002Fv1\u002Ftrace.proto`) or Zipkin v2 JSON.\n\n**Receiver**: `oap-server\u002Fserver-receiver-plugin\u002Fotel-receiver-plugin\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002Fotlp\u002FOpenTelemetryTraceHandler.java` (OTLP) and `oap-server\u002Fserver-receiver-plugin\u002Fzipkin-receiver-plugin\u002F...\u002FITZipkinTraceCollector` (Zipkin). After decoding, spans are iterated and passed to every registered `SpanListener` through `SpanListenerManager`.\n\n**Extension**: implement `org.apache.skywalking.oap.server.core.trace.SpanListener`:\n\n```java\npublic interface SpanListener {\n    String[] requiredModules();\n    void init(ModuleManager moduleManager);\n    SpanListenerResult onOTLPSpan(OTLPSpanReader span, String serviceName, Map\u003CString,String> resourceAttributes,\n                                  String scopeName, String scopeVersion);\n}\n```\n\nName the service by `serviceName`, not the resource's `service.name`: it is the service the span is analyzed and\nstored under, which `receiver-otel.otlpServiceGroupKeys` may have grouped (`shop::cart` for `service.name=cart`).\n\nRegister in `META-INF\u002Fservices\u002Forg.apache.skywalking.oap.server.core.trace.SpanListener`. Reference implementations:\n- `oap-server\u002Fanalyzer\u002Fios-analyzer\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002FIOSHTTPSpanListener.java` — OTel Swift URLSession → OAL (Service \u002F ServiceInstance \u002F Endpoint)\n- `oap-server\u002Fanalyzer\u002Fios-analyzer\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002FIOSMetricKitSpanListener.java` — OTel Swift MetricKit → MAL samples via `OpenTelemetryMetricRequestProcessor.toMeter`\n- `oap-server\u002Fanalyzer\u002Fgen-ai-analyzer\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002FGenAISpanListener.java` — LLM \u002F MCP instrumentation\n\nReturn `SpanListenerResult.builder().shouldPersist(false).build()` to prevent a span being stored as a trace (useful for e.g. 24-hour aggregate spans that are not meaningful as traces).\n\n**When both native and OTLP should feed the same metric**, write your extension against the downstream scope (Service \u002F ServiceInstance \u002F Endpoint + OAL) and emit those entities from **both** a native TAL rule and an OTLP TAL rule with `source {}` (or a `SpanListener` on the OTLP side). The OAL layer is protocol-agnostic. A TAL `metrics {}` block makes a meter metric, not an OAL source, so it feeds the same metric only when both inputs use `metrics {}`.\n\n---\n\n## 3. Metrics — MAL \u002F OAL rules\n\n### 3.1 OAL (for entities already produced by core sources)\n\n`oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Foal\u002Fcore.oal` already defines the standard `service_cpm`, `service_resp_time`, `service_sla`, `endpoint_cpm`, etc. — keyed off the `Service`, `ServiceInstance`, `Endpoint` source scopes. If your new layer emits those entities with the right `Layer`, you get the core metrics for free.\n\n**Who emits those entities matters.** `service_cpm` \u002F `service_resp_time` \u002F `service_sla` \u002F `endpoint_cpm` \u002F `endpoint_resp_time` come from the `native-rpc-entry` rule's `source Service {}` \u002F `source Endpoint {}` (`tal\u002Fnative-rpc.yaml`). That only fires on **inbound entry spans**. Client-side \u002F edge layers only emit exit spans, so `native-rpc-entry` never sends these sources for them. Mini-programs get their entities from MAL \u002F LAL and use MAL metrics (`meter_\u003Clayer>_*`) for load \u002F latency; browser uses its own OAL sources and metrics (`browser_app_*` in `browser.oal`). iOS is the exception: `IOSHTTPSpanListener` sends the core `Service` \u002F `Endpoint` sources itself for each URLSession HTTP call (an `NSURLSession`-scope CLIENT span from an iOS \u002F iPadOS resource with a non-empty `service.name`, carrying an HTTP method and a peer domain), and `ServiceInstance` when the resource carries a non-empty `service.version`, so `service_cpm` \u002F `service_resp_time` \u002F `service_sla` \u002F `endpoint_cpm` do populate under `IOS`. Don't wire dashboards to OAL-derived metrics that the layer never produces.\n\nAdditional OAL rules go in the same file. OAL grammar: `oap-server\u002Foal-grammar\u002Fsrc\u002Fmain\u002Fantlr4\u002FOALParser.g4`. Syntax and examples: [`docs\u002Fen\u002Fconcepts-and-designs\u002Foal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Foal.md).\n\n### 3.2 MAL (for meter \u002F metric samples)\n\n**File location**: `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fotel-rules\u002F\u003Cnamespace>\u002F*.yaml` (OTel scrape) or `meter-analyzer-config\u002F*.yaml` (SW native meter).\n\n**Registration** (MANDATORY): append the rule path to the default in `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fapplication.yml`:\n```yaml\nreceiver-otel:\n  default:\n    enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:\"...,\u003Cyour-namespace>\u002F*\"}\n```\nIf you skip this, the rule isn't loaded in the default starter and downstream `converters.forEach(...)` is a silent no-op — samples just disappear.\n\n**Also update** `test\u002Fe2e-v2\u002Fcases\u002Fstorage\u002Fexpected\u002Fconfig-dump.yml`. This e2e diffs `\u002Fdebugging\u002Fconfig\u002Fdump` output against a static file; every default-value change in `application.yml` must mirror here or the storage e2e fails.\n\n**Naming**: keep attribute scope in the metric name. If a metric reads from `foo.bar.foreground.exit_count`, call it `foo_foreground_exit_count`, not `foo_exit_count`. When two attributes (foreground vs background, ingress vs egress) share a theme, keep them as separate distinguishable metrics — don't flatten.\n\n### 3.3 Histogram pitfalls (both OAL and MAL)\n\n- **Per-bucket counts, not cumulative.** `histogram_percentile` accumulates internally. Feeding cumulative counts doubles the accumulator and shifts percentiles up. Each observation contributes `1` to exactly one `le` bucket.\n- **`le` label scale.** `SampleFamily.histogram()` rescales `le` by `defaultHistogramBucketUnit.toMillis(1)` (default `SECONDS` ⇒ ×1000). If your raw labels are already in ms, use `SampleFamilyBuilder.newBuilder(samples).defaultHistogramBucketUnit(TimeUnit.MILLISECONDS).build()`.\n- **Overflow bucket.** MAL parses `le=\"Infinity\"` as `Double.POSITIVE_INFINITY` → stored as `Long.MAX_VALUE`. If a percentile lands there, the UI renders ≈ 9.2 × 10¹⁸. Prefer a finite sentinel (e.g., 30 s for hang\u002Flaunch) so outliers remain visible without crashing the chart.\n- **`NamingControl`** — every `service` \u002F `serviceInstance` \u002F `endpoint` label that ends up as an entity key must go through `namingControl.formatServiceName()` \u002F `formatInstanceName()` \u002F `formatEndpointName()`. Raw resource attributes (long git-SHA build metadata, path-style endpoints) will otherwise exceed storage limits.\n\n---\n\n## 4. Logs — LAL\n\n**File location**: `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Flal\u002F\u003Cfeature>.yaml`.\n\n**Registration**: append to `SW_LOG_LAL_FILES` env var, or `log-analyzer.default.lalFiles` in `application.yml`.\n\n**Grammar**: `oap-server\u002Fanalyzer\u002Flog-analyzer\u002Fsrc\u002Fmain\u002Fantlr4\u002FLALParser.g4`. The compiler docs are thorough: [`oap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md`](..\u002F..\u002F..\u002Foap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md) — covers:\n- `filter { … }` block with `parsed.*`, `tag()`, `sourceAttribute()`, `log.*`, `def` locals, `extractor {}`, `metrics {}`, `sink {}`\n- `layer: auto` mode (script decides layer from source attrs)\n- `inputType` \u002F `outputType` SPI (`LALSourceTypeProvider`) to bind typed proto to `parsed.*`\n- Test harness at `oap-server\u002Fanalyzer\u002Flog-analyzer\u002Fsrc\u002Ftest\u002Fresources\u002Fscripts\u002Flal\u002Ftest-lal\u002Foap-cases\u002F` — ship YAML + `.data.yaml` pair; `LALScriptExecutionTest` picks them up automatically.\n\n**`layer: auto` gotcha**: the extractor's `layer \"X\"` assignment must propagate to `LogMetadata.layer` (not just the output builder) — `FilterSpec.doSink` reads metadata to decide drop\u002Froute. The current codegen (`LALBlockCodegen.generateFieldToOutput`) handles this correctly; if you extend codegen, preserve that invariant.\n\n---\n\n## 5. UI — templates, submodule, and menu\n\n### 5.1 Dashboard templates\n\nJSON files under `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fui-initialized-templates\u002F\u003Cfeature>\u002F`. Conventional fileset:\n\n- `\u003Cfeature>-root.json` — `ServiceList` widget, layered service index.\n- `\u003Cfeature>-service.json` — per-service dashboard, usually a `Tab` with Overview \u002F Instance \u002F Endpoint \u002F Log sub-tabs.\n- `\u003Cfeature>-instance.json` — per-instance dashboard.\n- `\u003Cfeature>-endpoint.json` — per-endpoint dashboard.\n- Menu link in `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fui-initialized-templates\u002Fmenu.yaml`.\n\n> **Collaborate with the developer on dashboard design.** Dashboard layout and metric selection are product decisions, not derivable from the MAL rules alone. Before authoring the JSON, check in with the developer on: which metric goes on which panel, which percentiles \u002F aggregations to show, what's on the root list vs. per-service vs. per-instance vs. per-endpoint pages, what stays in the Overview tab vs. a sub-tab, widget sizes \u002F ordering, and whether Trace \u002F Log tabs belong at all (depends on whether the feature emits traces and logs). Then **manually set up the feature against a live OAP** — emit a few real OTLP payloads, open each dashboard in the UI, confirm every widget renders non-empty, and walk the drill-down from root → service → instance → endpoint. Only after that eyes-on pass is the dashboard ready to ship.\n\nDetails that always bite:\n\n- **Tab chrome eats ~4 rows.** Set outer `Tab.h = content_max_y + 4` or you get an inner scrollbar on top of the page scrollbar.\n- **Widget heights consistent.** Use `h=15` per row for `Line` \u002F `Bar` widgets to match mesh \u002F general peer templates. Don't mix h=12 \u002F h=15 \u002F h=24 across a feature's dashboards.\n- **List widgets (ServiceList \u002F InstanceList \u002F EndpointList)** render two pieces per cell:\n  - `expressions` → the aggregated summary number (right side). Required. Wrap raw metrics in `avg(...)` \u002F `sum(...)`. Omitting it leaves the column empty.\n  - `subExpressions` → the sparkline trend (left side). Optional, falls back to `expressions` if absent.\n  - `metricConfig` pairs 1-to-1 with `expressions` (not `subExpressions`). Use for `label` (column header) and `unit`.\n  - Compound columns via MQE addition: `avg(metric_a + metric_b)` in `expressions`, `metric_a + metric_b` in `subExpressions`.\n  - Normal widgets with multiple expressions on one chart — always give a human label per expression in `metricConfig`, otherwise legend renders raw metric names.\n\n### 5.2 Hot-reload (edits to shipped templates take effect on restart)\n\n`UITemplateInitializer` normally uses `addIfNotExist` (keyed on JSON `id`), so on a plain restart your edits to shipped templates are ignored — the seeded copy in storage wins. For dev \u002F extension work, set the environment variable and restart:\n\n```bash\nSW_UI_TEMPLATE_FORCE_RELOAD=true\n# then restart OAP — shipped templates overwrite the seeded copies via addOrReplace.\n# Hard-refresh the browser (Cmd+Shift+R) because the booster UI caches config in memory.\n```\n\nThis is a dev \u002F extension knob only — read directly from the OS environment, not wired through `application.yml`. Unset or `false` preserves the default production behavior where operators' UI edits persist across restarts.\n\n### 5.3 New top-level menu group (e.g. \"Mobile\", \"Gateway\")\n\nNew **child entries** only need template JSON + `menu.yaml`. A new **top-level group** needs submodule work in `skywalking-ui` (git submodule at `skywalking-ui\u002F`):\n\n1. `skywalking-ui\u002Fsrc\u002Fassets\u002Ficons\u002F\u003Cname>.svg` — menu icon.\n2. `skywalking-ui\u002Fsrc\u002Flocales\u002Flang\u002Fmenus\u002F{en,es,zh}.ts` — i18n labels (`\u003Cname>`, `\u003Cname>_desc`, `\u003Cname>_\u003Cchild>`, `\u003Cname>_\u003Cchild>_desc`). Missing i18n falls back to raw keys on screen.\n3. Commit those upstream in `apache\u002Fskywalking-booster-ui`, bump the submodule pointer here.\n4. Rebuild with `-Pui,dist` (not `-Pbackend,dist`) and `make docker.ui`, otherwise the new assets aren't in the image. See [package skill](..\u002Fpackage\u002FSKILL.md).\n\n---\n\n## 6. Docs set — everything that moves together\n\n| File | What to add |\n|---|---|\n| `docs\u002Fen\u002Fsetup\u002Fbackend\u002F\u003Cfeature>-monitoring.md` | User-facing setup guide. Required SDK \u002F collector, required env vars, sample OTLP payload, metrics \u002F log tags list. |\n| `docs\u002Fmenu.yml` | Add the guide to the docs navigation tree. |\n| `docs\u002Fen\u002Fsecurity\u002FREADME.md` | **Client-side monitoring**: mobile \u002F browser \u002F mini-program endpoints accept data from the public internet without auth by default. Add a rate-limit + abuse-protection note. Agent-side monitoring over private VPC usually doesn't need this. |\n| `docs\u002Fen\u002Fchanges\u002Fchanges.md` | Per-release changelog. Add under `#### OAP Server` for features\u002Fbug-fixes, `#### UI` if the submodule bumps, `#### Documentation` if new guides. |\n| `docs\u002Fen\u002Fswip\u002FSWIP-\u003CN>.md` | Design doc for non-trivial features; keep reconciled with final code after review. |\n| `docs\u002Fen\u002Fswip\u002Freadme.md` | Move SWIP from \"Proposed\" to \"Accepted\" once merged. |\n\n---\n\n## 7. E2E test\n\nFolder: `test\u002Fe2e-v2\u002Fcases\u002F\u003Cfeature>\u002F`. See [run-e2e skill](..\u002Frun-e2e\u002FSKILL.md) for harness basics (setup \u002F trigger \u002F verify phases, template functions, expected-file authoring via `contains`).\n\nFeature-specific extras:\n\n1. **Docker-compose**: extend `..\u002F..\u002Fscript\u002Fdocker-compose\u002Fbase-compose.yml` oap + storage. Set `SW_OTEL_RECEIVER_ENABLED_HANDLERS`, `SW_LOG_LAL_FILES`, and any feature-specific env. Don't list default rules here — rely on `application.yml` so the config-dump test stays consistent.\n2. **Register in CI**: add an entry under the e2e matrix in `.github\u002Fworkflows\u002Fskywalking.yaml`.\n3. **OTLP data hygiene**:\n   - `spanId` is exactly 16 hex chars; `traceId` is exactly 32. Protobuf-JSON decodes them as base64; other lengths return HTTP 400.\n   - Setup-step curl loops must fail loudly: `curl -sS -f --retry 30 --retry-delay 5 --retry-connrefused --retry-all-errors --max-time 10 ...` + `set -e`. The pattern `for …; do curl && break || sleep 5; done` silently succeeds when every attempt connection-refused (the final `sleep` returns 0) because OAP takes ~50 s to start in CI.\n4. **Verify each case by hand before pushing** and use a published simulator image where the upstream SDK has one. See [`run-e2e` skill §6–§9](..\u002Frun-e2e\u002FSKILL.md) for verify-query triage, expected-file authoring, and the list of expected-file traps (YAML quoting on colons, nested `contains` fragility, `timeUnixNano: \"0\"`, setup-step curl loops that swallow failure).\n5. **Config dump**: when you modify any `application.yml` default (rule list, handler list, ...), also update `test\u002Fe2e-v2\u002Fcases\u002Fstorage\u002Fexpected\u002Fconfig-dump.yml`.\n\n---\n\n## 8. Pre-submit checklist\n\nRun in order; each has a dedicated skill:\n\n1. [`compile`](..\u002Fcompile\u002FSKILL.md) — `.\u002Fmvnw clean flatten:flatten install javadoc:javadoc -B -q -Pall -Dmaven.test.skip -Dcheckstyle.skip -Dgpg.skip`. Javadoc `[ERROR] ... error:` must be fixed; `[ERROR] ... warning:` is noise.\n2. Tests: `.\u002Fmvnw -pl oap-server\u002Fanalyzer\u002Flog-analyzer -am test -Dtest=LALScriptExecutionTest` for LAL changes, equivalent for MAL, OAL unit tests where the rule lives.\n3. [`license`](..\u002Flicense\u002FSKILL.md) — `license-eye header check` (Apache 2.0 headers). If the UI submodule bumped npm deps, also `license-eye dependency resolve --summary .\u002Fdist-material\u002Frelease-docs\u002FLICENSE.tpl` and commit the `LICENSE` diff.\n4. [`package`](..\u002Fpackage\u002FSKILL.md) — full `make docker` (or dist-rebuild + `make docker.oap` \u002F `make docker.ui`). Don't trust `docker.oap` alone to pick up code changes; the dist tarball is a file-level prereq.\n5. [`run-e2e`](..\u002Frun-e2e\u002FSKILL.md) — run the new case end-to-end locally before pushing.\n6. [`gh-pull-request`](..\u002Fgh-pull-request\u002FSKILL.md) — commit and open PR with the template, attach UI screenshots.\n\n---\n\n## 9. Cross-cutting traps collected from real extensions\n\n| Trap | Symptom | Fix |\n|---|---|---|\n| Layer folder named with hyphens | `ui-initialized-templates\u002Fmy-layer\u002F*.json` on disk but dashboard empty | Folder must be `Layer.name().toLowerCase()` (underscores). `wechat_mini_program\u002F`, not `wechat-mini-program\u002F` |\n| Extending the native trace rules for a client-side layer | Added componentId → layer mapping \"to get topology for free\"; then `service_cpm` etc. don't populate anyway | Client-side (exit-only) layers don't need trace-analysis changes. Ship `Layer` + `component-libraries.yml` + MAL + LAL. See §2 header for when extending trace analysis is actually justified |\n| Dashboards wired to OAL-derived metrics on a client-side layer | Charts stay empty — `service_cpm` \u002F `service_resp_time` \u002F `endpoint_cpm` never populate | Those come from the `native-rpc-entry` rule's inbound entry spans, which mini-program \u002F browser don't emit. Use the layer's own metrics instead: MAL `meter_\u003Clayer>_*` for mini-programs, `browser_app_*` (`browser.oal`) for browser. iOS does get them, from `IOSHTTPSpanListener`'s own `Service` \u002F `Endpoint` sources |\n| LAL `#` comment in DSL body | OAP boot fails with `extraneous input 'metrics'` at the `#` line | LAL grammar accepts `\u002F\u002F` and `\u002F* *\u002F` only. No `#` comments |\n| LAL bare def-var value access — `tag 'x': myvar` | OAP boot fails with `Cannot resolve expression '[myvar]'` | `LALValueCodegen` only treats def-vars as valid when there's a method chain (`myvar?.foo`). Inline `sourceAttribute(...)` or `tag(...)` in the three value-access sites instead of storing to a def, or add a no-op chain like `myvar.toString()`. Conditions (`if (myvar == ...)`) go through a different codegen path and DO handle bare def refs |\n| MAL rule file with multi-doc YAML (`---` separator) | `MALFilterExecutionTest` fails with `expected a single document`; OAP would also fail at startup via `Rules.loadRules` (calls `Yaml.loadAs`, single-doc) | Split into one file per top-level `expSuffix` \u002F `filter` \u002F `metricPrefix` combo. Reference each file by name in `application.yml` (`lalFiles`, `malFiles`, `enabledOtelMetricsRules`) |\n| Histogram bucket bounds in ms with default `.histogram()` call | Percentile values come out 1000× inflated (`P50 = 100000` for a 100 ms request) | MAL's default `defaultHistogramBucketUnit = SECONDS` multiplies `le` by 1000 when converting to stored ms. For SDKs that emit ms bounds directly (mini-program, most in-house JS SDKs), use `.histogram(\"le\", TimeUnit.MILLISECONDS)`. `TimeUnit` is registered in `MALCodegenHelper.ENUM_FQCN` so rule YAML can reference it |\n| `.rate()` \u002F `.increase()` on a DELTA counter \u002F histogram `_count` | Compute delta-of-delta — nonsense values | OTLP DELTA means each flush is already the per-flush count. MAL's per-minute aggregator sums directly to requests-per-minute. For CUMULATIVE (Prometheus default), use `.rate(\"PT1M\")` for per-sec or `.increase(\"PT1M\")` for per-min; they are **not** equivalent (differ by 60×) |\n| MAL bucket-family name with `_histogram` suffix | Rule compiles, metric stays empty | `PrometheusMetricConverter` emits the bucket family as `\u003Cmetric_name>` (no suffix), with `\u003Cmetric_name>_count` and `\u003Cmetric_name>_sum` as the companion families. Reference the bucket family by its bare name in the rule |\n| LAL `tag(\"key\") == null` check on a missing tag | Check never triggers — LAL's `tag()` returns empty string `\"\"` for absent tags, not `null`, so conditional abort is a no-op | Guard on both: `tag(\"key\") == null || tag(\"key\") == \"\"`. Same for `sourceAttribute(\"key\")` |\n| MAL rule file with `expSuffix: service(...)` mixing service-scoped and endpoint-scoped rules | Endpoint rules' metrics appear under the service entity (no endpoint_traffic rows created); `swctl endpoint ls` returns empty. `MetricConvert.formatExp:132` appends `(exp).expSuffix` to every rule, so `.endpoint(...)` in the rule's expression is followed by `.service(...)` and the service scope wins. | Follow the APISIX pattern — leave `expSuffix:` empty and explicitly chain `.service(...)` \u002F `.endpoint(...)` on each rule. If every rule in a file is same-scope (e.g. all instance-scoped in `*-instance.yaml`), `expSuffix` is still fine |\n\nE2E-side authoring traps (log-content YAML quoting, nested `contains` fragility, `timeUnixNano: \"0\"`, setup-step curl swallowing failure) live in the [`run-e2e` skill §9](..\u002Frun-e2e\u002FSKILL.md).\n| Missing layer-root template | Menu item lands on empty \"no dashboard\" view | `Layer.vue:41-44` requires a template with `isRoot: true`; ship a `\u003Clayer>-root.json` (precedent: `ios\u002Fios-root.json`) |\n| New layer not selectable in UI \u002F swctl | `service ly \u003CLAYER>` returns empty | Add the enum in `Layer.java` with a fresh id; see §0 |\n| Stale dist in image | `make docker.oap` succeeds but behavior is old | See [`package` skill](..\u002Fpackage\u002FSKILL.md); rebuild dist first, then image |\n| Shipped template edits don't render after restart | Edited JSON on disk, but OAP shows old copy | Set `SW_UI_TEMPLATE_FORCE_RELOAD=true` (environment variable, not `application.yml`) and restart; OAP writes `addOrReplace`. Unset in production so operator UI edits aren't clobbered. |\n| MAL service-scoped rule fragmented by instance | \"Overall app\" metric has N series per version | Keep `service_instance_id` out of service-scoped `.sum`\u002F`.avg`. Put instance-scoped metrics in a separate `\u003Cfeature>-instance.yaml` with `expSuffix: instance(...)` — see `otel-rules\u002Fios\u002F` pattern. |\n| Config dump drift | Storage e2e fails after `application.yml` edit | Mirror the change in `test\u002Fe2e-v2\u002Fcases\u002Fstorage\u002Fexpected\u002Fconfig-dump.yml` |\n| LAL `layer:auto` WARN drop | Every log silently dropped | Codegen must set `metadata.layer` alongside output builder's layer |\n| Histogram 1000× off | p50 = 2 000 000 ms | `defaultHistogramBucketUnit(MILLISECONDS)` on the `SampleFamily` |\n| Histogram shifted up | p50 = 5 000 on 500–2500 ms data | Emit per-bucket, not cumulative |\n| Empty list column | Dashboard instance table has no numbers | Add `expressions` (not just `subExpressions`) |\n| Double scroll | Inner tab scroll + page scroll | `Tab.h = content_max + 4` chrome buffer |\n| Raw metric name in legend | Multiple expressions on one widget | Add parallel `metricConfig[].label` entries |\n| `spanKind` mismatch | Span skipped silently | `OTLPSpanReaderImpl.spanKind()` returns enum `.name()` like `SPAN_KIND_CLIENT`, not `CLIENT` |\n| Negative latency | `*_sla` inflated | Clamp `rawLatency \u003C 0 ? 0 : (int) min(Integer.MAX_VALUE, rawLatency)` |\n| SLA too optimistic | Client-side DNS \u002F timeout \u002F connection-refused counted as success | Require a real response code for success: `status = code > 0 && code \u003C 400`. Do not treat missing (0) as success. |\n| Silent drop with no rules | `converters == null` NPE elsewhere | Init to empty list at declaration |\n| `+Inf` bucket surfaces as `9.2e18` | UI shows garbage when a sample lands in overflow | Use finite sentinel bucket (e.g. 30 s) |\n| OTel submodule bump misses UI image | New icons \u002F i18n not visible | Rebuild with `-Pui,dist` + `make docker.ui` |\n| LICENSE drift after submodule bump | CI `Dependency licenses` fails | `license-eye dependency resolve --summary .\u002Fdist-material\u002Frelease-docs\u002FLICENSE.tpl` and commit |\n| Setup-step curl loop swallows failure | e2e passes metrics but fails log verify | `curl -f --retry-connrefused --retry-all-errors` + `set -e` |\n| Verify case flag mismatch | CI fails early on `swctl` flag unknown to pinned commit | Fire every verify command locally first; bump `SW_CTL_COMMIT` if needed |\n\n---\n\n## 10. Reference extensions (read their code)\n\nBrowse these when you need a template to copy from. Grouped by the kind of data source, because the extension points differ significantly:\n\n### SW native trace (language agents) — the \"General\" layer and friends\n\n`Layer.GENERAL` is the default layer for services reporting via the SkyWalking native trace protocol (Java \u002F Python \u002F Go \u002F Node.js \u002F .NET \u002F PHP \u002F Lua \u002F Rust agents). It's the richest reference because the native pipeline is where the **full topology** lives:\n\n- **Service → service dependency** (topology): derived by `tal\u002Fnative-rpc.yaml` and `tal\u002Fnative-network-alias.yaml`. Entry\u002Fexit spans produce `ServiceRelation`, `ServiceInstanceRelation` sources.\n- **Endpoint → endpoint dependency**: `native-rpc-entry` emits `EndpointRelation` sources from an entry span's references, and `native-endpoint-cross-thread` from exit and local spans with references. This is unique to the native protocol — OTLP spans don't preserve upstream entry-span metadata the same way, so OTLP-only layers generally lose endpoint-level topology.\n- OAL aggregation: `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Foal\u002Fcore.oal` — all `service_relation_*`, `service_instance_relation_*`, `endpoint_relation_*` metrics are already defined there; a new native-tracing layer just needs to produce the right sources with the right `Layer`.\n\nRead order: `nativeTalFiles` in `application.yml` → `SwNativeRuleRunner` → the `tal\u002Fnative-*.yaml` rules → OAL rules.\n\n### Service-mesh (Istio \u002F Envoy ALS) — topology from proxy-side signals\n\nNo agent in the service; Envoy emits HTTP\u002FTCP access logs (ALS) and metrics. Service-to-service topology is reconstructed from ALS request\u002Fresponse pairs.\n\n- `oap-server\u002Fserver-receiver-plugin\u002Fenvoy-metrics-receiver-plugin\u002F` — ALS receiver + `EnvoyAccessLogBuilder` persistence (LAL output type).\n- `otel-rules\u002Fistio-*`, `otel-rules\u002Fenvoy\u002F*` — MAL rules for proxy-emitted OTel metrics.\n- `lal\u002Fenvoy-als.yaml` — LAL rule that extracts topology-generating fields from ALS.\n- Layer: `Layer.MESH`, plus `Layer.MESH_CP` (control plane) and `Layer.MESH_DP` (data plane) for split-plane topology.\n\nTopology parity with native traces: ALS gives you service-level relations; endpoint-level relations are approximate (proxy sees inbound + outbound, not the full call-chain that a native agent observes).\n\n### OTLP-only layers (OTel SDKs, iOS\u002FAndroid, AI gateway, …)\n\nOTLP spans feed a `SpanListener`. Topology is **partial by construction** — an OTel Swift SDK on a mobile device only knows about its own outbound requests; there's no upstream caller. So these layers typically produce only `Service` \u002F `ServiceInstance` \u002F `Endpoint` sources (no `*Relation`) and the Topology view shows isolated islands, by design.\n\n- **iOS** (`Layer.IOS`, OTLP traces + MetricKit + LAL diagnostics): `oap-server\u002Fanalyzer\u002Fios-analyzer\u002F` + `docs\u002Fen\u002Fswip\u002FSWIP-11.md`.\n- **Envoy AI Gateway** (`Layer.ENVOY_AI_GATEWAY`, LLM \u002F MCP, OTLP metrics + Zipkin sampled traces + LAL access-log sampling): `oap-server\u002Fanalyzer\u002Fgen-ai-analyzer\u002F`, `otel-rules\u002Fenvoy-ai-gateway\u002F*`, `lal\u002Fenvoy-ai-gateway.yaml`.\n\n### OTel-metrics-only layers (scraped exporters)\n\nNo spans, no logs — just MAL rules converting OTel metric scrapes.\n\n- **MySQL \u002F PostgreSQL**: `otel-rules\u002Fmysql\u002F*`, `otel-rules\u002Fpostgresql\u002F*`.\n- **Kubernetes**: `otel-rules\u002Fk8s\u002F*` — paired with `virtual-k8s-service` (`Layer.K8S_SERVICE`) sources for synthetic services.\n- **AWS**: `otel-rules\u002Faws-eks\u002F*`, `otel-rules\u002Faws-s3\u002F*`, `otel-rules\u002Faws-dynamodb\u002F*`.\n\n### SW native meter (custom protocol)\n\nNo OTel; the agent \u002F plugin sends pre-built meter samples on the SW `MeterProtocol`.\n\n- **Nginx Lua** (`Layer.NGINX`): `meter-analyzer-config\u002Fnginx-service.yaml`, with the Lua agent in `apache\u002Fskywalking-nginx-lua`.\n- **APISIX** (`Layer.APISIX`): `meter-analyzer-config\u002Fapisix-service.yaml` + LAL trace extraction.\n\n### Which to copy from\n\n| You're adding | Closest reference |\n|---|---|\n| Language agent (new JVM \u002F runtime) | The General \u002F Java layer flow; usually no change — the native TAL rules already analyze it. |\n| Service mesh \u002F proxy | `mesh-*` + `envoy-*` rules; LAL on ALS. |\n| Mobile \u002F browser \u002F mini-program SDK (OTLP) | `ios-analyzer` (OTLP client-side). Expect no topology relations. |\n| Database \u002F middleware (metrics only) | `mysql`, `postgresql` — pure MAL rules. |\n| Cloud service (no agent, just metrics) | `aws-eks` \u002F `aws-s3`. |\n| Custom protocol \u002F home-grown telemetry | `apisix` \u002F `nginx` — SW native meter. |\n","# Adding a New Monitoring Feature \u002F Layer\n\nSkyWalking monitoring for a new target (a new `Layer` — mobile OS, gateway, DB, language runtime, …) is composed from the same set of reusable extension points. This skill is the index: which extension point fits which signal, where its contract lives, which docs must be updated together, and which harness-wide traps the extension-specific docs don't cover.\n\nThere is one skill per narrow concern. This one is the wiring map.\n\n---\n\n## Working posture\n\n- **Default scope for a new monitoring feature is narrow.** For most new monitoring targets, traces are **re-used as-is** — you do not write or modify a trace analyzer. Metrics and log extensions via OTLP (MAL + LAL) are the common shape. OTLP \u002F Zipkin span-based metrics analysis (`SpanListener`) already exists and only needs a new implementation if your feature introduces genuinely new semantic-convention attributes to extract (e.g. `gen_ai.*`, `mcp.*` for the GenAI layer). Do **not** touch the native trace rules (`tal\u002Fnative-*.yaml`) unless the OAP core protocol gains a new `SpanLayer` enum value — details in §2.\n- **If you find yourself editing more than this skill suggests, stop and confirm.** Scope creep on trace analysis, span-layer mapping, receiver handlers, or protobuf\u002Fproto contracts is a strong signal the framing is wrong. Ask the developer before committing that direction.\n- **Verify every new feature locally before pushing to CI.** Compile + checkstyle + license-eye are minimums, but they only tell you the code builds — they do **not** tell you the new MAL rule compiles its closures at startup, the LAL layer:auto dispatch works, the OTLP payload shape matches the MAL labels, the UI template renders against real data, or the layer shows up in `swctl service ly \u003CLAYER>`. Run the new e2e case end-to-end locally against a live OAP (see [run-e2e skill](..\u002Frun-e2e\u002FSKILL.md)) and fire every verify step with `swctl` by hand at least once. Pushing to CI without a local green run wastes shared CI cycles and slow-loops review.\n\n---\n\n## 0. Register the `Layer` — the feature's entry point\n\nA `Layer` is how OAP slices services \u002F instances \u002F endpoints by data source. The UI, storage partitioning, menu navigation, and OAL aggregation all key off it.\n\n`Layer` is a registry-backed value type (no longer a closed enum). Built-in layers are declared as `public static final Layer` constants in `oap-server\u002Fserver-core\u002Fsrc\u002Fmain\u002Fjava\u002Forg\u002Fapache\u002Fskywalking\u002Foap\u002Fserver\u002Fcore\u002Fanalysis\u002FLayer.java`; external layers are registered through the `Layer.register(name, ordinal, normal)` API at boot. **Pick the registration path that matches the scope of your feature:**\n\n| Your feature ships as | Registration path |\n|---|---|\n| Part of the OAP distribution (in-tree, the common case for new SkyWalking-supported targets) | Add a `public static final Layer` constant to `Layer.java` with the next sequential ordinal in `0–49`. Examples: `IOS = register(\"IOS\", 47, true)`, `APISIX = register(\"APISIX\", 21, true)`, `VIRTUAL_DATABASE = register(\"VIRTUAL_DATABASE\", 14, false)` for inferred\u002Fnon-real services. |\n| An out-of-tree MAL or LAL rule file | Add a top-level `layerDefinitions:` block to the rule file. The DSL loader funnels each entry through `Layer.register` before compiling the rule. One file ships the layer + the rules that produce its telemetry. |\n| An out-of-tree plugin module (jar) | Implement `org.apache.skywalking.oap.server.core.analysis.LayerExtension` and register via `META-INF\u002Fservices\u002F`. Discovered by `LayerExtensionLoader` during `CoreModuleProvider.prepare()`. |\n| Operator-deployed config (no code, no DSL) | Add an entry to `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Flayer-extensions.yml` (or override on the OAP node's classpath). |\n\n**Ordinal conventions:** `0–49` is in active use by built-ins. `50–999` is reserved by convention for future built-in layers. External layers are recommended (not required) to start at `>= 1000` to avoid colliding with future built-ins on OAP upgrade. Collisions in either direction are detected at boot via the ordinal-uniqueness check, which fails OAP startup loudly.\n\n**Storage encoding is the ordinal int**, persisted in BanyanDB \u002F Elasticsearch \u002F JDBC. Every OAP node that reads or writes a given layer must agree on its `(name, ordinal)` mapping — deploy `layer-extensions.yml` and any `layerDefinitions:` rule files identically across all nodes. The registry is sealed at the start of `Core.notifyAfterCompleted()`; later registration attempts throw.\n\n**UI template folders are auto-discovered by file scan, not by `Layer.values()`.** `UITemplateInitializer` walks `ui-initialized-templates\u002F**\u002F*.json` recursively (depth 2) and trusts each template's own `configuration.layer` field. Drop a folder of dashboard JSONs on disk and the initializer picks them up on the next boot — folder name is purely organizational.\n\n**Component ID lookup in Java code**: IDs declared in `component-libraries.yml` are loaded at runtime into `ComponentLibraryCatalogService`'s `componentName2Id` map — they are **not** exposed as Java enum constants. To look up by name in listener code, inject the catalog service and resolve once at construction:\n```java\nIComponentLibraryCatalogService catalog = moduleManager.find(CoreModule.NAME)\n    .provider().getService(IComponentLibraryCatalogService.class);\nint myComponentId = catalog.getComponentId(\"My-Component-Name\");\n```\nCache as an `int` field; runtime comparisons are then plain `componentId == myComponentId`. **Trap:** there is a `ComponentsDefine` class under `skywalking-trace-receiver-plugin\u002Fsrc\u002Ftest\u002Fjava\u002F...\u002Fmock\u002FComponentsDefine.java` — it is a test-only mock holding five hand-picked constants (Tomcat, Dubbo, RocketMQ, MongoDB). Do not import or extend it from production code.\n\nEmit the layer from every source object your feature produces. Built-in layers have a static-field accessor; external layers are looked up by name through the registry:\n\n```java\n\u002F\u002F Built-in layer (constant)\nservice.setLayer(Layer.IOS);\n\n\u002F\u002F External layer (registered via yaml \u002F SPI \u002F layerDefinitions:)\nservice.setLayer(Layer.nameOf(\"IOT_FLEET\"));\n```\n\nDownstream (the core OAL, `service ly \u003CLAYER>` swctl query, topology filters, UI root dashboard's layer selector) all work off this single enum value.\n\n---\n\n## 1. Pick the right extension point\n\n| Signal you want to process | Extension point | Register via |\n|---|---|---|\n| **SkyWalking native trace segments** (from language agents) | TAL rules with `inputType: sw-native` — `tal\u002Fnative-*.yaml`, run by `trace-analyzer` | Append the file to `nativeTalFiles` (`SW_TRACE_NATIVE_TAL_FILES`); order matters |\n| **OTLP trace spans of a layer** (OTel SDKs that set `service.layer`) | TAL rules with `inputType: otlp` — `tal\u002F*.yaml` | Append the file to `talFiles` (`SW_TRACE_TAL_FILES`) |\n| **OTLP \u002F Zipkin trace spans** needing Java (iOS\u002FAndroid, GenAI, …) | `SpanListener` in `core.trace` | `META-INF\u002Fservices\u002Forg.apache.skywalking.oap.server.core.trace.SpanListener` |\n| **OTel metrics** (Prometheus scrape, Envoy stats, Istio, …) | MAL rules — `otel-rules\u002F\u003Cnamespace>\u002F*.yaml` | Append the rule path to `enabledOtelMetricsRules` in `application.yml` |\n| **Logs** (OTLP, SkyWalking native, Fluent Bit, Envoy ALS) | LAL rules — `lal\u002F\u003Crule>.yaml` | Append to `SW_LOG_LAL_FILES` (env) or `lalFiles` (YAML) under `log-analyzer` |\n| **Pre-computed meter \u002F custom meter** (custom agent protocol, meter receiver) | MAL `meter-analyzer-config\u002F*.yaml` | Append to `meter-analyzer-config` entries |\n| **Aggregation from Source entities** (service, instance, endpoint, relation) | OAL rules — `oal\u002F*.oal` | Already loaded by `OAL grammar`; just add lines |\n\nThe DSLs (OAL, MAL, LAL, TAL, Hierarchy) compile via ANTLR4 + Javassist at startup. Source-of-truth details are in:\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Foal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Foal.md)\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Ftal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal.md), [`tal-sw-native.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal-sw-native.md), [`tal-otlp.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal-otlp.md); the [`tal` skill](..\u002Ftal\u002FSKILL.md) covers compiling, inspecting and verifying TAL rules\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Fmal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Fmal.md)\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Flal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Flal.md)\n- [`docs\u002Fen\u002Fconcepts-and-designs\u002Fscope-definitions.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Fscope-definitions.md)\n- [`oap-server\u002Fanalyzer\u002Fmeter-analyzer\u002FCLAUDE.md`](..\u002F..\u002F..\u002Foap-server\u002Fanalyzer\u002Fmeter-analyzer\u002FCLAUDE.md) — MAL compiler internals\n- [`oap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md`](..\u002F..\u002F..\u002Foap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md) — LAL compiler internals\n\n---\n\n## 2. Trace ingestion — two parallel pipelines\n\nSkyWalking OAP ingests traces through two distinct entry points. Pick based on source.\n\n> **Before writing any trace extension — ask \"do I actually need one?\"**\n>\n> For **most new layers you do not touch trace analysis at all.** Native segments\n> flow through the existing native rules: `native-rpc-exit` (`tal\u002Fnative-rpc.yaml`) produces\n> the outbound `ServiceRelation` edges, the backend-side agent produces the inbound\n> metrics via `native-rpc-entry`; OTLP spans go through the OTLP TAL rules of `talFiles` instead, and the new-layer Service \u002F Instance \u002F Endpoint\n> entities come from **MAL + LAL**, not from trace analysis. This is how mobile mini-programs\n> (`Layer.WECHAT_MINI_PROGRAM` \u002F `Layer.ALIPAY_MINI_PROGRAM`) and most scraped-metric layers\n> work. Browser (`Layer.BROWSER`) has its own receiver and OAL sources (`browser.oal`), and iOS\n> (`Layer.IOS`) has its own OTLP span listeners: `IOSHTTPSpanListener` sends core OAL sources,\n> `IOSMetricKitSpanListener` sends MAL samples, and LAL reads its diagnostics.\n>\n> Extend trace analysis **only** when the feature genuinely needs per-span\n> interpretation that the existing pipeline cannot express:\n>\n> - **A new `SpanLayer`** — OAP core protobuf has a new `SpanLayer` enum value that\n>   the native rules' layer branches must dispatch on.\n>   That means touching `tal\u002Fnative-rpc.yaml` and its siblings.\n> - **A genuinely new signal class not covered by existing rules or listeners** — e.g. the GenAI\n>   work added `GenAISpanListener` for LLM \u002F MCP instrumentation because OTLP spans\n>   carry semantic-convention attributes (`gen_ai.*`, `mcp.*`) that no prior listener\n>   extracted. That is rare.\n>\n> **Anti-pattern we've been bitten by:** using a componentId mapping in\n> the native trace analysis to assign a client-side layer to exit-only segments. This\n> does not produce inbound metrics anyway (no entry spans → no\n> `service_cpm` \u002F `endpoint_cpm` from OAL), and it is orthogonal to the Service entity\n> layer (which MAL \u002F LAL sets). Mirror the mini-program pattern instead: ship `Layer` +\n> `component-libraries.yml` + MAL + LAL + dashboards, and leave the native rules\n> alone.\n\n### 2.1 SkyWalking native segments — TAL `sw-native` rules\n\n**Source protocol**: `apm-protocol\u002Fapm-network\u002Fsrc\u002Fmain\u002Fproto\u002Flanguage-agent\u002FTracing.proto` (`SegmentObject`).\n\n**Receiver**: `oap-server\u002Fserver-receiver-plugin\u002Fskywalking-trace-receiver-plugin` (and the Kafka fetcher). After wire decoding, segments go to `ISegmentParserService` of `trace-analyzer`, whose `SwNativeRuleRunner` runs every rule of every file in `nativeTalFiles`, in list order, on every span of the segment.\n\n**Extension**: a TAL rule file under `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Ftal\u002F` with `inputType: sw-native`, appended to `nativeTalFiles` in `application.yml`. Outputs are made inside `build {}`: `source \u003CScope> {}` for the existing sources (`core.oal` metrics, records such as `DatabaseSlowStatement`, or a module's own source, as `ai-evaluation` takes `GenAIEvaluationSample` through its `SourceDispatcher`), `metrics {}` for new meter metrics. A file that needs a module (its functions or sources) lists it under `requiredModules`. Reference files: `tal\u002Fnative-rpc.yaml` (services, endpoints, relations), `tal\u002Fnative-virtual-service.yaml` (virtual DB\u002Fcache\u002FMQ), `tal\u002Fnative-virtual-genai.yaml`. Language and runner: [`docs\u002Fen\u002Fconcepts-and-designs\u002Ftal-sw-native.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Ftal-sw-native.md); file order and loggers: [`docs\u002Fen\u002Fsetup\u002Fbackend\u002Fnative-trace-analysis.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fsetup\u002Fbackend\u002Fnative-trace-analysis.md).\n\nA module contributes functions to TAL rules through `TALFunctionProvider` (`server-core`, `supportsInputType(...)` naming the inputs it serves, registered in `META-INF\u002Fservices\u002Forg.apache.skywalking.oap.server.core.source.TALFunctionProvider`), as `ai-evaluation` does for `native-ai-evaluation` and `otlp-ai-evaluation`.\n\nUse when: the data arrives on the SW native wire (language agents — Java, Python, Go, Node.js, .NET, PHP, Lua, etc.).\n\n### 2.2 OTLP \u002F Zipkin spans — `SpanListener`\n\n**Source protocol**: OTLP proto (`opentelemetry\u002Fproto\u002Ftrace\u002Fv1\u002Ftrace.proto`) or Zipkin v2 JSON.\n\n**Receiver**: `oap-server\u002Fserver-receiver-plugin\u002Fotel-receiver-plugin\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002Fotlp\u002FOpenTelemetryTraceHandler.java` (OTLP) and `oap-server\u002Fserver-receiver-plugin\u002Fzipkin-receiver-plugin\u002F...\u002FITZipkinTraceCollector` (Zipkin). After decoding, spans are iterated and passed to every registered `SpanListener` through `SpanListenerManager`.\n\n**Extension**: implement `org.apache.skywalking.oap.server.core.trace.SpanListener`:\n\n```java\npublic interface SpanListener {\n    String[] requiredModules();\n    void init(ModuleManager moduleManager);\n    SpanListenerResult onOTLPSpan(OTLPSpanReader span, String serviceName, Map\u003CString,String> resourceAttributes,\n                                  String scopeName, String scopeVersion);\n}\n```\n\nName the service by `serviceName`, not the resource's `service.name`: it is the service the span is analyzed and\nstored under, which `receiver-otel.otlpServiceGroupKeys` may have grouped (`shop::cart` for `service.name=cart`).\n\nRegister in `META-INF\u002Fservices\u002Forg.apache.skywalking.oap.server.core.trace.SpanListener`. Reference implementations:\n- `oap-server\u002Fanalyzer\u002Fios-analyzer\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002FIOSHTTPSpanListener.java` — OTel Swift URLSession → OAL (Service \u002F ServiceInstance \u002F Endpoint)\n- `oap-server\u002Fanalyzer\u002Fios-analyzer\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002FIOSMetricKitSpanListener.java` — OTel Swift MetricKit → MAL samples via `OpenTelemetryMetricRequestProcessor.toMeter`\n- `oap-server\u002Fanalyzer\u002Fgen-ai-analyzer\u002Fsrc\u002Fmain\u002Fjava\u002F...\u002FGenAISpanListener.java` — LLM \u002F MCP instrumentation\n\nReturn `SpanListenerResult.builder().shouldPersist(false).build()` to prevent a span being stored as a trace (useful for e.g. 24-hour aggregate spans that are not meaningful as traces).\n\n**When both native and OTLP should feed the same metric**, write your extension against the downstream scope (Service \u002F ServiceInstance \u002F Endpoint + OAL) and emit those entities from **both** a native TAL rule and an OTLP TAL rule with `source {}` (or a `SpanListener` on the OTLP side). The OAL layer is protocol-agnostic. A TAL `metrics {}` block makes a meter metric, not an OAL source, so it feeds the same metric only when both inputs use `metrics {}`.\n\n---\n\n## 3. Metrics — MAL \u002F OAL rules\n\n### 3.1 OAL (for entities already produced by core sources)\n\n`oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Foal\u002Fcore.oal` already defines the standard `service_cpm`, `service_resp_time`, `service_sla`, `endpoint_cpm`, etc. — keyed off the `Service`, `ServiceInstance`, `Endpoint` source scopes. If your new layer emits those entities with the right `Layer`, you get the core metrics for free.\n\n**Who emits those entities matters.** `service_cpm` \u002F `service_resp_time` \u002F `service_sla` \u002F `endpoint_cpm` \u002F `endpoint_resp_time` come from the `native-rpc-entry` rule's `source Service {}` \u002F `source Endpoint {}` (`tal\u002Fnative-rpc.yaml`). That only fires on **inbound entry spans**. Client-side \u002F edge layers only emit exit spans, so `native-rpc-entry` never sends these sources for them. Mini-programs get their entities from MAL \u002F LAL and use MAL metrics (`meter_\u003Clayer>_*`) for load \u002F latency; browser uses its own OAL sources and metrics (`browser_app_*` in `browser.oal`). iOS is the exception: `IOSHTTPSpanListener` sends the core `Service` \u002F `Endpoint` sources itself for each URLSession HTTP call (an `NSURLSession`-scope CLIENT span from an iOS \u002F iPadOS resource with a non-empty `service.name`, carrying an HTTP method and a peer domain), and `ServiceInstance` when the resource carries a non-empty `service.version`, so `service_cpm` \u002F `service_resp_time` \u002F `service_sla` \u002F `endpoint_cpm` do populate under `IOS`. Don't wire dashboards to OAL-derived metrics that the layer never produces.\n\nAdditional OAL rules go in the same file. OAL grammar: `oap-server\u002Foal-grammar\u002Fsrc\u002Fmain\u002Fantlr4\u002FOALParser.g4`. Syntax and examples: [`docs\u002Fen\u002Fconcepts-and-designs\u002Foal.md`](..\u002F..\u002F..\u002Fdocs\u002Fen\u002Fconcepts-and-designs\u002Foal.md).\n\n### 3.2 MAL (for meter \u002F metric samples)\n\n**File location**: `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fotel-rules\u002F\u003Cnamespace>\u002F*.yaml` (OTel scrape) or `meter-analyzer-config\u002F*.yaml` (SW native meter).\n\n**Registration** (MANDATORY): append the rule path to the default in `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fapplication.yml`:\n```yaml\nreceiver-otel:\n  default:\n    enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:\"...,\u003Cyour-namespace>\u002F*\"}\n```\nIf you skip this, the rule isn't loaded in the default starter and downstream `converters.forEach(...)` is a silent no-op — samples just disappear.\n\n**Also update** `test\u002Fe2e-v2\u002Fcases\u002Fstorage\u002Fexpected\u002Fconfig-dump.yml`. This e2e diffs `\u002Fdebugging\u002Fconfig\u002Fdump` output against a static file; every default-value change in `application.yml` must mirror here or the storage e2e fails.\n\n**Naming**: keep attribute scope in the metric name. If a metric reads from `foo.bar.foreground.exit_count`, call it `foo_foreground_exit_count`, not `foo_exit_count`. When two attributes (foreground vs background, ingress vs egress) share a theme, keep them as separate distinguishable metrics — don't flatten.\n\n### 3.3 Histogram pitfalls (both OAL and MAL)\n\n- **Per-bucket counts, not cumulative.** `histogram_percentile` accumulates internally. Feeding cumulative counts doubles the accumulator and shifts percentiles up. Each observation contributes `1` to exactly one `le` bucket.\n- **`le` label scale.** `SampleFamily.histogram()` rescales `le` by `defaultHistogramBucketUnit.toMillis(1)` (default `SECONDS` ⇒ ×1000). If your raw labels are already in ms, use `SampleFamilyBuilder.newBuilder(samples).defaultHistogramBucketUnit(TimeUnit.MILLISECONDS).build()`.\n- **Overflow bucket.** MAL parses `le=\"Infinity\"` as `Double.POSITIVE_INFINITY` → stored as `Long.MAX_VALUE`. If a percentile lands there, the UI renders ≈ 9.2 × 10¹⁸. Prefer a finite sentinel (e.g., 30 s for hang\u002Flaunch) so outliers remain visible without crashing the chart.\n- **`NamingControl`** — every `service` \u002F `serviceInstance` \u002F `endpoint` label that ends up as an entity key must go through `namingControl.formatServiceName()` \u002F `formatInstanceName()` \u002F `formatEndpointName()`. Raw resource attributes (long git-SHA build metadata, path-style endpoints) will otherwise exceed storage limits.\n\n---\n\n## 4. Logs — LAL\n\n**File location**: `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Flal\u002F\u003Cfeature>.yaml`.\n\n**Registration**: append to `SW_LOG_LAL_FILES` env var, or `log-analyzer.default.lalFiles` in `application.yml`.\n\n**Grammar**: `oap-server\u002Fanalyzer\u002Flog-analyzer\u002Fsrc\u002Fmain\u002Fantlr4\u002FLALParser.g4`. The compiler docs are thorough: [`oap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md`](..\u002F..\u002F..\u002Foap-server\u002Fanalyzer\u002Flog-analyzer\u002FCLAUDE.md) — covers:\n- `filter { … }` block with `parsed.*`, `tag()`, `sourceAttribute()`, `log.*`, `def` locals, `extractor {}`, `metrics {}`, `sink {}`\n- `layer: auto` mode (script decides layer from source attrs)\n- `inputType` \u002F `outputType` SPI (`LALSourceTypeProvider`) to bind typed proto to `parsed.*`\n- Test harness at `oap-server\u002Fanalyzer\u002Flog-analyzer\u002Fsrc\u002Ftest\u002Fresources\u002Fscripts\u002Flal\u002Ftest-lal\u002Foap-cases\u002F` — ship YAML + `.data.yaml` pair; `LALScriptExecutionTest` picks them up automatically.\n\n**`layer: auto` gotcha**: the extractor's `layer \"X\"` assignment must propagate to `LogMetadata.layer` (not just the output builder) — `FilterSpec.doSink` reads metadata to decide drop\u002Froute. The current codegen (`LALBlockCodegen.generateFieldToOutput`) handles this correctly; if you extend codegen, preserve that invariant.\n\n---\n\n## 5. UI — templates, submodule, and menu\n\n### 5.1 Dashboard templates\n\nJSON files under `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fui-initialized-templates\u002F\u003Cfeature>\u002F`. Conventional fileset:\n\n- `\u003Cfeature>-root.json` — `ServiceList` widget, layered service index.\n- `\u003Cfeature>-service.json` — per-service dashboard, usually a `Tab` with Overview \u002F Instance \u002F Endpoint \u002F Log sub-tabs.\n- `\u003Cfeature>-instance.json` — per-instance dashboard.\n- `\u003Cfeature>-endpoint.json` — per-endpoint dashboard.\n- Menu link in `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Fui-initialized-templates\u002Fmenu.yaml`.\n\n> **Collaborate with the developer on dashboard design.** Dashboard layout and metric selection are product decisions, not derivable from the MAL rules alone. Before authoring the JSON, check in with the developer on: which metric goes on which panel, which percentiles \u002F aggregations to show, what's on the root list vs. per-service vs. per-instance vs. per-endpoint pages, what stays in the Overview tab vs. a sub-tab, widget sizes \u002F ordering, and whether Trace \u002F Log tabs belong at all (depends on whether the feature emits traces and logs). Then **manually set up the feature against a live OAP** — emit a few real OTLP payloads, open each dashboard in the UI, confirm every widget renders non-empty, and walk the drill-down from root → service → instance → endpoint. Only after that eyes-on pass is the dashboard ready to ship.\n\nDetails that always bite:\n\n- **Tab chrome eats ~4 rows.** Set outer `Tab.h = content_max_y + 4` or you get an inner scrollbar on top of the page scrollbar.\n- **Widget heights consistent.** Use `h=15` per row for `Line` \u002F `Bar` widgets to match mesh \u002F general peer templates. Don't mix h=12 \u002F h=15 \u002F h=24 across a feature's dashboards.\n- **List widgets (ServiceList \u002F InstanceList \u002F EndpointList)** render two pieces per cell:\n  - `expressions` → the aggregated summary number (right side). Required. Wrap raw metrics in `avg(...)` \u002F `sum(...)`. Omitting it leaves the column empty.\n  - `subExpressions` → the sparkline trend (left side). Optional, falls back to `expressions` if absent.\n  - `metricConfig` pairs 1-to-1 with `expressions` (not `subExpressions`). Use for `label` (column header) and `unit`.\n  - Compound columns via MQE addition: `avg(metric_a + metric_b)` in `expressions`, `metric_a + metric_b` in `subExpressions`.\n  - Normal widgets with multiple expressions on one chart — always give a human label per expression in `metricConfig`, otherwise legend renders raw metric names.\n\n### 5.2 Hot-reload (edits to shipped templates take effect on restart)\n\n`UITemplateInitializer` normally uses `addIfNotExist` (keyed on JSON `id`), so on a plain restart your edits to shipped templates are ignored — the seeded copy in storage wins. For dev \u002F extension work, set the environment variable and restart:\n\n```bash\nSW_UI_TEMPLATE_FORCE_RELOAD=true\n# then restart OAP — shipped templates overwrite the seeded copies via addOrReplace.\n# Hard-refresh the browser (Cmd+Shift+R) because the booster UI caches config in memory.\n```\n\nThis is a dev \u002F extension knob only — read directly from the OS environment, not wired through `application.yml`. Unset or `false` preserves the default production behavior where operators' UI edits persist across restarts.\n\n### 5.3 New top-level menu group (e.g. \"Mobile\", \"Gateway\")\n\nNew **child entries** only need template JSON + `menu.yaml`. A new **top-level group** needs submodule work in `skywalking-ui` (git submodule at `skywalking-ui\u002F`):\n\n1. `skywalking-ui\u002Fsrc\u002Fassets\u002Ficons\u002F\u003Cname>.svg` — menu icon.\n2. `skywalking-ui\u002Fsrc\u002Flocales\u002Flang\u002Fmenus\u002F{en,es,zh}.ts` — i18n labels (`\u003Cname>`, `\u003Cname>_desc`, `\u003Cname>_\u003Cchild>`, `\u003Cname>_\u003Cchild>_desc`). Missing i18n falls back to raw keys on screen.\n3. Commit those upstream in `apache\u002Fskywalking-booster-ui`, bump the submodule pointer here.\n4. Rebuild with `-Pui,dist` (not `-Pbackend,dist`) and `make docker.ui`, otherwise the new assets aren't in the image. See [package skill](..\u002Fpackage\u002FSKILL.md).\n\n---\n\n## 6. Docs set — everything that moves together\n\n| File | What to add |\n|---|---|\n| `docs\u002Fen\u002Fsetup\u002Fbackend\u002F\u003Cfeature>-monitoring.md` | User-facing setup guide. Required SDK \u002F collector, required env vars, sample OTLP payload, metrics \u002F log tags list. |\n| `docs\u002Fmenu.yml` | Add the guide to the docs navigation tree. |\n| `docs\u002Fen\u002Fsecurity\u002FREADME.md` | **Client-side monitoring**: mobile \u002F browser \u002F mini-program endpoints accept data from the public internet without auth by default. Add a rate-limit + abuse-protection note. Agent-side monitoring over private VPC usually doesn't need this. |\n| `docs\u002Fen\u002Fchanges\u002Fchanges.md` | Per-release changelog. Add under `#### OAP Server` for features\u002Fbug-fixes, `#### UI` if the submodule bumps, `#### Documentation` if new guides. |\n| `docs\u002Fen\u002Fswip\u002FSWIP-\u003CN>.md` | Design doc for non-trivial features; keep reconciled with final code after review. |\n| `docs\u002Fen\u002Fswip\u002Freadme.md` | Move SWIP from \"Proposed\" to \"Accepted\" once merged. |\n\n---\n\n## 7. E2E test\n\nFolder: `test\u002Fe2e-v2\u002Fcases\u002F\u003Cfeature>\u002F`. See [run-e2e skill](..\u002Frun-e2e\u002FSKILL.md) for harness basics (setup \u002F trigger \u002F verify phases, template functions, expected-file authoring via `contains`).\n\nFeature-specific extras:\n\n1. **Docker-compose**: extend `..\u002F..\u002Fscript\u002Fdocker-compose\u002Fbase-compose.yml` oap + storage. Set `SW_OTEL_RECEIVER_ENABLED_HANDLERS`, `SW_LOG_LAL_FILES`, and any feature-specific env. Don't list default rules here — rely on `application.yml` so the config-dump test stays consistent.\n2. **Register in CI**: add an entry under the e2e matrix in `.github\u002Fworkflows\u002Fskywalking.yaml`.\n3. **OTLP data hygiene**:\n   - `spanId` is exactly 16 hex chars; `traceId` is exactly 32. Protobuf-JSON decodes them as base64; other lengths return HTTP 400.\n   - Setup-step curl loops must fail loudly: `curl -sS -f --retry 30 --retry-delay 5 --retry-connrefused --retry-all-errors --max-time 10 ...` + `set -e`. The pattern `for …; do curl && break || sleep 5; done` silently succeeds when every attempt connection-refused (the final `sleep` returns 0) because OAP takes ~50 s to start in CI.\n4. **Verify each case by hand before pushing** and use a published simulator image where the upstream SDK has one. See [`run-e2e` skill §6–§9](..\u002Frun-e2e\u002FSKILL.md) for verify-query triage, expected-file authoring, and the list of expected-file traps (YAML quoting on colons, nested `contains` fragility, `timeUnixNano: \"0\"`, setup-step curl loops that swallow failure).\n5. **Config dump**: when you modify any `application.yml` default (rule list, handler list, ...), also update `test\u002Fe2e-v2\u002Fcases\u002Fstorage\u002Fexpected\u002Fconfig-dump.yml`.\n\n---\n\n## 8. Pre-submit checklist\n\nRun in order; each has a dedicated skill:\n\n1. [`compile`](..\u002Fcompile\u002FSKILL.md) — `.\u002Fmvnw clean flatten:flatten install javadoc:javadoc -B -q -Pall -Dmaven.test.skip -Dcheckstyle.skip -Dgpg.skip`. Javadoc `[ERROR] ... error:` must be fixed; `[ERROR] ... warning:` is noise.\n2. Tests: `.\u002Fmvnw -pl oap-server\u002Fanalyzer\u002Flog-analyzer -am test -Dtest=LALScriptExecutionTest` for LAL changes, equivalent for MAL, OAL unit tests where the rule lives.\n3. [`license`](..\u002Flicense\u002FSKILL.md) — `license-eye header check` (Apache 2.0 headers). If the UI submodule bumped npm deps, also `license-eye dependency resolve --summary .\u002Fdist-material\u002Frelease-docs\u002FLICENSE.tpl` and commit the `LICENSE` diff.\n4. [`package`](..\u002Fpackage\u002FSKILL.md) — full `make docker` (or dist-rebuild + `make docker.oap` \u002F `make docker.ui`). Don't trust `docker.oap` alone to pick up code changes; the dist tarball is a file-level prereq.\n5. [`run-e2e`](..\u002Frun-e2e\u002FSKILL.md) — run the new case end-to-end locally before pushing.\n6. [`gh-pull-request`](..\u002Fgh-pull-request\u002FSKILL.md) — commit and open PR with the template, attach UI screenshots.\n\n---\n\n## 9. Cross-cutting traps collected from real extensions\n\n| Trap | Symptom | Fix |\n|---|---|---|\n| Layer folder named with hyphens | `ui-initialized-templates\u002Fmy-layer\u002F*.json` on disk but dashboard empty | Folder must be `Layer.name().toLowerCase()` (underscores). `wechat_mini_program\u002F`, not `wechat-mini-program\u002F` |\n| Extending the native trace rules for a client-side layer | Added componentId → layer mapping \"to get topology for free\"; then `service_cpm` etc. don't populate anyway | Client-side (exit-only) layers don't need trace-analysis changes. Ship `Layer` + `component-libraries.yml` + MAL + LAL. See §2 header for when extending trace analysis is actually justified |\n| Dashboards wired to OAL-derived metrics on a client-side layer | Charts stay empty — `service_cpm` \u002F `service_resp_time` \u002F `endpoint_cpm` never populate | Those come from the `native-rpc-entry` rule's inbound entry spans, which mini-program \u002F browser don't emit. Use the layer's own metrics instead: MAL `meter_\u003Clayer>_*` for mini-programs, `browser_app_*` (`browser.oal`) for browser. iOS does get them, from `IOSHTTPSpanListener`'s own `Service` \u002F `Endpoint` sources |\n| LAL `#` comment in DSL body | OAP boot fails with `extraneous input 'metrics'` at the `#` line | LAL grammar accepts `\u002F\u002F` and `\u002F* *\u002F` only. No `#` comments |\n| LAL bare def-var value access — `tag 'x': myvar` | OAP boot fails with `Cannot resolve expression '[myvar]'` | `LALValueCodegen` only treats def-vars as valid when there's a method chain (`myvar?.foo`). Inline `sourceAttribute(...)` or `tag(...)` in the three value-access sites instead of storing to a def, or add a no-op chain like `myvar.toString()`. Conditions (`if (myvar == ...)`) go through a different codegen path and DO handle bare def refs |\n| MAL rule file with multi-doc YAML (`---` separator) | `MALFilterExecutionTest` fails with `expected a single document`; OAP would also fail at startup via `Rules.loadRules` (calls `Yaml.loadAs`, single-doc) | Split into one file per top-level `expSuffix` \u002F `filter` \u002F `metricPrefix` combo. Reference each file by name in `application.yml` (`lalFiles`, `malFiles`, `enabledOtelMetricsRules`) |\n| Histogram bucket bounds in ms with default `.histogram()` call | Percentile values come out 1000× inflated (`P50 = 100000` for a 100 ms request) | MAL's default `defaultHistogramBucketUnit = SECONDS` multiplies `le` by 1000 when converting to stored ms. For SDKs that emit ms bounds directly (mini-program, most in-house JS SDKs), use `.histogram(\"le\", TimeUnit.MILLISECONDS)`. `TimeUnit` is registered in `MALCodegenHelper.ENUM_FQCN` so rule YAML can reference it |\n| `.rate()` \u002F `.increase()` on a DELTA counter \u002F histogram `_count` | Compute delta-of-delta — nonsense values | OTLP DELTA means each flush is already the per-flush count. MAL's per-minute aggregator sums directly to requests-per-minute. For CUMULATIVE (Prometheus default), use `.rate(\"PT1M\")` for per-sec or `.increase(\"PT1M\")` for per-min; they are **not** equivalent (differ by 60×) |\n| MAL bucket-family name with `_histogram` suffix | Rule compiles, metric stays empty | `PrometheusMetricConverter` emits the bucket family as `\u003Cmetric_name>` (no suffix), with `\u003Cmetric_name>_count` and `\u003Cmetric_name>_sum` as the companion families. Reference the bucket family by its bare name in the rule |\n| LAL `tag(\"key\") == null` check on a missing tag | Check never triggers — LAL's `tag()` returns empty string `\"\"` for absent tags, not `null`, so conditional abort is a no-op | Guard on both: `tag(\"key\") == null || tag(\"key\") == \"\"`. Same for `sourceAttribute(\"key\")` |\n| MAL rule file with `expSuffix: service(...)` mixing service-scoped and endpoint-scoped rules | Endpoint rules' metrics appear under the service entity (no endpoint_traffic rows created); `swctl endpoint ls` returns empty. `MetricConvert.formatExp:132` appends `(exp).expSuffix` to every rule, so `.endpoint(...)` in the rule's expression is followed by `.service(...)` and the service scope wins. | Follow the APISIX pattern — leave `expSuffix:` empty and explicitly chain `.service(...)` \u002F `.endpoint(...)` on each rule. If every rule in a file is same-scope (e.g. all instance-scoped in `*-instance.yaml`), `expSuffix` is still fine |\n\nE2E-side authoring traps (log-content YAML quoting, nested `contains` fragility, `timeUnixNano: \"0\"`, setup-step curl swallowing failure) live in the [`run-e2e` skill §9](..\u002Frun-e2e\u002FSKILL.md).\n| Missing layer-root template | Menu item lands on empty \"no dashboard\" view | `Layer.vue:41-44` requires a template with `isRoot: true`; ship a `\u003Clayer>-root.json` (precedent: `ios\u002Fios-root.json`) |\n| New layer not selectable in UI \u002F swctl | `service ly \u003CLAYER>` returns empty | Add the enum in `Layer.java` with a fresh id; see §0 |\n| Stale dist in image | `make docker.oap` succeeds but behavior is old | See [`package` skill](..\u002Fpackage\u002FSKILL.md); rebuild dist first, then image |\n| Shipped template edits don't render after restart | Edited JSON on disk, but OAP shows old copy | Set `SW_UI_TEMPLATE_FORCE_RELOAD=true` (environment variable, not `application.yml`) and restart; OAP writes `addOrReplace`. Unset in production so operator UI edits aren't clobbered. |\n| MAL service-scoped rule fragmented by instance | \"Overall app\" metric has N series per version | Keep `service_instance_id` out of service-scoped `.sum`\u002F`.avg`. Put instance-scoped metrics in a separate `\u003Cfeature>-instance.yaml` with `expSuffix: instance(...)` — see `otel-rules\u002Fios\u002F` pattern. |\n| Config dump drift | Storage e2e fails after `application.yml` edit | Mirror the change in `test\u002Fe2e-v2\u002Fcases\u002Fstorage\u002Fexpected\u002Fconfig-dump.yml` |\n| LAL `layer:auto` WARN drop | Every log silently dropped | Codegen must set `metadata.layer` alongside output builder's layer |\n| Histogram 1000× off | p50 = 2 000 000 ms | `defaultHistogramBucketUnit(MILLISECONDS)` on the `SampleFamily` |\n| Histogram shifted up | p50 = 5 000 on 500–2500 ms data | Emit per-bucket, not cumulative |\n| Empty list column | Dashboard instance table has no numbers | Add `expressions` (not just `subExpressions`) |\n| Double scroll | Inner tab scroll + page scroll | `Tab.h = content_max + 4` chrome buffer |\n| Raw metric name in legend | Multiple expressions on one widget | Add parallel `metricConfig[].label` entries |\n| `spanKind` mismatch | Span skipped silently | `OTLPSpanReaderImpl.spanKind()` returns enum `.name()` like `SPAN_KIND_CLIENT`, not `CLIENT` |\n| Negative latency | `*_sla` inflated | Clamp `rawLatency \u003C 0 ? 0 : (int) min(Integer.MAX_VALUE, rawLatency)` |\n| SLA too optimistic | Client-side DNS \u002F timeout \u002F connection-refused counted as success | Require a real response code for success: `status = code > 0 && code \u003C 400`. Do not treat missing (0) as success. |\n| Silent drop with no rules | `converters == null` NPE elsewhere | Init to empty list at declaration |\n| `+Inf` bucket surfaces as `9.2e18` | UI shows garbage when a sample lands in overflow | Use finite sentinel bucket (e.g. 30 s) |\n| OTel submodule bump misses UI image | New icons \u002F i18n not visible | Rebuild with `-Pui,dist` + `make docker.ui` |\n| LICENSE drift after submodule bump | CI `Dependency licenses` fails | `license-eye dependency resolve --summary .\u002Fdist-material\u002Frelease-docs\u002FLICENSE.tpl` and commit |\n| Setup-step curl loop swallows failure | e2e passes metrics but fails log verify | `curl -f --retry-connrefused --retry-all-errors` + `set -e` |\n| Verify case flag mismatch | CI fails early on `swctl` flag unknown to pinned commit | Fire every verify command locally first; bump `SW_CTL_COMMIT` if needed |\n\n---\n\n## 10. Reference extensions (read their code)\n\nBrowse these when you need a template to copy from. Grouped by the kind of data source, because the extension points differ significantly:\n\n### SW native trace (language agents) — the \"General\" layer and friends\n\n`Layer.GENERAL` is the default layer for services reporting via the SkyWalking native trace protocol (Java \u002F Python \u002F Go \u002F Node.js \u002F .NET \u002F PHP \u002F Lua \u002F Rust agents). It's the richest reference because the native pipeline is where the **full topology** lives:\n\n- **Service → service dependency** (topology): derived by `tal\u002Fnative-rpc.yaml` and `tal\u002Fnative-network-alias.yaml`. Entry\u002Fexit spans produce `ServiceRelation`, `ServiceInstanceRelation` sources.\n- **Endpoint → endpoint dependency**: `native-rpc-entry` emits `EndpointRelation` sources from an entry span's references, and `native-endpoint-cross-thread` from exit and local spans with references. This is unique to the native protocol — OTLP spans don't preserve upstream entry-span metadata the same way, so OTLP-only layers generally lose endpoint-level topology.\n- OAL aggregation: `oap-server\u002Fserver-starter\u002Fsrc\u002Fmain\u002Fresources\u002Foal\u002Fcore.oal` — all `service_relation_*`, `service_instance_relation_*`, `endpoint_relation_*` metrics are already defined there; a new native-tracing layer just needs to produce the right sources with the right `Layer`.\n\nRead order: `nativeTalFiles` in `application.yml` → `SwNativeRuleRunner` → the `tal\u002Fnative-*.yaml` rules → OAL rules.\n\n### Service-mesh (Istio \u002F Envoy ALS) — topology from proxy-side signals\n\nNo agent in the service; Envoy emits HTTP\u002FTCP access logs (ALS) and metrics. Service-to-service topology is reconstructed from ALS request\u002Fresponse pairs.\n\n- `oap-server\u002Fserver-receiver-plugin\u002Fenvoy-metrics-receiver-plugin\u002F` — ALS receiver + `EnvoyAccessLogBuilder` persistence (LAL output type).\n- `otel-rules\u002Fistio-*`, `otel-rules\u002Fenvoy\u002F*` — MAL rules for proxy-emitted OTel metrics.\n- `lal\u002Fenvoy-als.yaml` — LAL rule that extracts topology-generating fields from ALS.\n- Layer: `Layer.MESH`, plus `Layer.MESH_CP` (control plane) and `Layer.MESH_DP` (data plane) for split-plane topology.\n\nTopology parity with native traces: ALS gives you service-level relations; endpoint-level relations are approximate (proxy sees inbound + outbound, not the full call-chain that a native agent observes).\n\n### OTLP-only layers (OTel SDKs, iOS\u002FAndroid, AI gateway, …)\n\nOTLP spans feed a `SpanListener`. Topology is **partial by construction** — an OTel Swift SDK on a mobile device only knows about its own outbound requests; there's no upstream caller. So these layers typically produce only `Service` \u002F `ServiceInstance` \u002F `Endpoint` sources (no `*Relation`) and the Topology view shows isolated islands, by design.\n\n- **iOS** (`Layer.IOS`, OTLP traces + MetricKit + LAL diagnostics): `oap-server\u002Fanalyzer\u002Fios-analyzer\u002F` + `docs\u002Fen\u002Fswip\u002FSWIP-11.md`.\n- **Envoy AI Gateway** (`Layer.ENVOY_AI_GATEWAY`, LLM \u002F MCP, OTLP metrics + Zipkin sampled traces + LAL access-log sampling): `oap-server\u002Fanalyzer\u002Fgen-ai-analyzer\u002F`, `otel-rules\u002Fenvoy-ai-gateway\u002F*`, `lal\u002Fenvoy-ai-gateway.yaml`.\n\n### OTel-metrics-only layers (scraped exporters)\n\nNo spans, no logs — just MAL rules converting OTel metric scrapes.\n\n- **MySQL \u002F PostgreSQL**: `otel-rules\u002Fmysql\u002F*`, `otel-rules\u002Fpostgresql\u002F*`.\n- **Kubernetes**: `otel-rules\u002Fk8s\u002F*` — paired with `virtual-k8s-service` (`Layer.K8S_SERVICE`) sources for synthetic services.\n- **AWS**: `otel-rules\u002Faws-eks\u002F*`, `otel-rules\u002Faws-s3\u002F*`, `otel-rules\u002Faws-dynamodb\u002F*`.\n\n### SW native meter (custom protocol)\n\nNo OTel; the agent \u002F plugin sends pre-built meter samples on the SW `MeterProtocol`.\n\n- **Nginx Lua** (`Layer.NGINX`): `meter-analyzer-config\u002Fnginx-service.yaml`, with the Lua agent in `apache\u002Fskywalking-nginx-lua`.\n- **APISIX** (`Layer.APISIX`): `meter-analyzer-config\u002Fapisix-service.yaml` + LAL trace extraction.\n\n### Which to copy from\n\n| You're adding | Closest reference |\n|---|---|\n| Language agent (new JVM \u002F runtime) | The General \u002F Java layer flow; usually no change — the native TAL rules already analyze it. |\n| Service mesh \u002F proxy | `mesh-*` + `envoy-*` rules; LAL on ALS. |\n| Mobile \u002F browser \u002F mini-program SDK (OTLP) | `ios-analyzer` (OTLP client-side). Expect no topology relations. |\n| Database \u002F middleware (metrics only) | `mysql`, `postgresql` — pure MAL rules. |\n| Cloud service (no agent, just metrics) | `aws-eks` \u002F `aws-s3`. |\n| Custom protocol \u002F home-grown telemetry | `apisix` \u002F `nginx` — SW native meter. |\n",{"name":5,"description":7},"master","b0a3065c6eb0e0fa526566869ac1298b204a11a5",".claude\u002Fskills\u002Fnew-monitoring-feature",6628,"https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking","https:\u002F\u002Fskywalking.apache.org\u002F","https:\u002F\u002Fgithub.com\u002Fapache","https:\u002F\u002Fwww.apache.org\u002F","2026-11-11 08:56:29",{"source":8,"skillName":5,"commands":38,"prompt":48,"zipAvailable":49},[39,42,45],{"tool":40,"command":41},"npx","npx skills add apache\u002Fskywalking --skill new-monitoring-feature",{"tool":43,"command":44},"bunx","bunx skills add apache\u002Fskywalking --skill new-monitoring-feature",{"tool":46,"command":47},"pnpm","pnpm dlx skills add apache\u002Fskywalking --skill new-monitoring-feature","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Fnew-monitoring-feature\u002FSKILL.md",true,[],[52,69,79,95,102,118],{"skillNo":53,"name":54,"nameZh":6,"description":55,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":56,"categories":57,"keywords":63,"publishedAt":23},"01M6ACBTY89EEM81A0WWN0QQT3","test","Run unit tests, integration tests, or slow integration tests matching CI. Use to validate changes before submitting a PR.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Ftest\u002FSKILL.md",[58],{"categoryNo":59,"categoryName":60,"taxonomyType":61,"taxonomyTypeDesc":62},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",[54,64,65,66,67,68],"unit tests","validate","integration tests","pr","run",{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":76,"publishedAt":23},"01M6ACBRBNXZ57EN1G8GREKFY5","run-e2e","Run SkyWalking E2E tests locally","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Frun-e2e\u002FSKILL.md",[75],{"categoryNo":59,"categoryName":60,"taxonomyType":61,"taxonomyTypeDesc":62},[68,77,20,78,54],"run e2e","e2e tests",{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":91,"publishedAt":23},"01M6ACBNH0HSYFX2K9CYJSPAVM","package","Rebuild the SkyWalking distribution and OAP Docker image after source changes. Use before running e2e tests so the image reflects your code changes. Avoids the \"image looks updated but runtime has stale jars\" trap.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Fpackage\u002FSKILL.md",[85,88],{"categoryNo":86,"categoryName":87,"taxonomyType":61,"taxonomyTypeDesc":62},"01JBSEEDCAT000000000000011","运维与部署",{"categoryNo":89,"categoryName":90,"taxonomyType":61,"taxonomyTypeDesc":62},"01JBSEEDCAT000000000000016","图像与多媒体",[81,20,92,78,93,94],"code","docker","image",{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":23},"01M6ACBG5P0NY2A7AEX89BZNFH","license","Check and fix Apache 2.0 license headers and dependency licenses using skywalking-eyes. Use before submitting a PR.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Flicense\u002FSKILL.md",[],[97,20,9,67],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":106,"categories":107,"keywords":111,"publishedAt":23},"01M6ACBDKKTXZ8Y36YHBADT8JA","gh-pull-request","Verify, commit, and push changes on a PR branch. Runs pre-flight checks (compile, checkstyle, license headers) before every push. Also creates the PR if one doesn't exist yet.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Fgh-pull-request\u002FSKILL.md",[108],{"categoryNo":109,"categoryName":110,"taxonomyType":61,"taxonomyTypeDesc":62},"01JBSEEDCAT000000000000007","代码审查",[112,113,114,115,116,67,117],"gh pull request","pull","pull request","compile","push","commit",{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":124,"publishedAt":23},"01M6ACBB39DNW4G2SR836V0RA8","generate-classes","Generate bytecode classes from DSL scripts (MAL, OAL, LAL, Hierarchy). Runs the compiler and dumps .class files for inspection.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fskywalking\u002Fblob\u002Fmaster\u002F.claude\u002Fskills\u002Fgenerate-classes\u002FSKILL.md",[],[125,126],"generate classes","generate"]