[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6ACJHX8BES7ZNNRNVCQ4QWE":3,"skill-same:01M6ACJHX8BES7ZNNRNVCQ4QWE":68},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":45,"commitSha":46,"skillDirectory":47,"forksCount":48,"repositoryHtmlUrl":49,"repositoryHomepageUrl":50,"ownerHtmlUrl":51,"ownerBlogUrl":52,"ownerDescription":6,"updatedAt":53,"install":54,"resources":67},"01M6ACJHX8BES7ZNNRNVCQ4QWE","review-comet-pr",null,"Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, Iceberg writes, and documentation. Provides guidance to a human reviewer rather than posting comments.","apache\u002Fdatafusion-comet","apache","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F47359?v=4","01M6AC83E6K4FDKK7HZ8WNMQKE","The Apache Software Foundation","01M6ACHX42TPNCR8TQJEAR8J5J",1322,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-pr\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"review","review comet pr","pr","comments","review skills","use","for","posting","request","human","reviewer","number","when","documentation","workflow","datafusion","memory","pull request","iceberg","pull","2026-11-10 10:12:30",false,"---\nname: review-comet-pr\ndescription: Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, Iceberg writes, and documentation. Provides guidance to a human reviewer rather than posting comments.\nargument-hint: \u003Cpr-number>\n---\n\n\u003C!--\nLicensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License.\n-->\n\nReview Comet PR #$ARGUMENTS\n\nThis is the entry point for every Comet PR review. It covers what applies regardless of what the PR\ntouches. Depth for a specific subsystem lives in a sibling skill, and step 1 tells you which ones to\nload.\n\n## 1. Route to the Area Skills\n\n**Do this before reading the diff in detail.** Look at the list of changed files and load every\nsibling skill whose area the PR touches. More than one usually applies. A shuffle change that alters\nspilling is also a memory change. An expression that returns a new array type may also be an FFI\nchange.\n\n| The PR touches                                                                                                                                                                             | Also use                        |\n| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- |\n| `spark\u002Fsrc\u002Fmain\u002Fscala\u002Forg\u002Fapache\u002Fcomet\u002Fserde\u002F`, `QueryPlanSerde.scala`, `native\u002Fspark-expr\u002F`, `expr.proto`, `comet_scalar_funcs.rs`                                                        | `review-comet-expression-pr`    |\n| `CometExecIterator`, `CometNativeArrowSource`, `NativeUtil`, `CometVector` subclasses, `jni_api.rs`, `scan.rs`, anything using `FFI_Arrow*`                                                | `review-comet-ffi-pr`           |\n| `native\u002Fcore\u002Fsrc\u002Fexecution\u002Fmemory_pools\u002F`, `CometTaskMemoryManager`, `CometShuffleMemoryAllocator`, `MemoryConsumer` \u002F `try_grow` \u002F `reserve` call sites, pool configs                     | `review-comet-memory-pr`        |\n| `native\u002Fshuffle\u002F`, `spark\u002Fsrc\u002Fmain\u002Fscala\u002Forg\u002Fapache\u002Fspark\u002Fsql\u002Fcomet\u002Fexecution\u002Fshuffle\u002F`, `spark\u002Fsrc\u002Fmain\u002Fjava\u002Forg\u002Fapache\u002Fspark\u002Fshuffle\u002F`, `CometShuffleExchangeExec`                       | `review-comet-shuffle-pr`       |\n| `IcebergWriteStrategy`, `IcebergWriteExec`, `IcebergCommitExec`, `CometIcebergWriteExec`, `CometIcebergNativeWrite`, `iceberg_write.rs`, `iceberg_partition_path.rs`, the iceberg-rust pin | `review-comet-iceberg-write-pr` |\n| `docs\u002Fsource\u002F`, `README.md`, `AGENTS.md`, `.ai\u002Fskills\u002F**\u002FSKILL.md`, `.doc(...)` strings in config classes, doc comments                                                                    | `review-comet-docs`             |\n\nFor a new operator, read `docs\u002Fsource\u002Fcontributor-guide\u002Fadding_a_new_operator.md` alongside this\nskill. There is no dedicated operator review skill yet.\n\nFor a PR that deals with timestamps or the session timezone, read\n`docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md` as well, whichever areas it touches. Timezone handling\ncuts across the serde, the native kernels, the scans and the JVM\u002Fnative boundary, so no area skill\nowns it. \"Timestamps and timezones\" in step 5 says what to check.\n\nIf the PR falls outside all of these, for example build, CI, or release tooling, this skill on its\nown is the review.\n\n## 2. Gather PR Metadata\n\nUse `gh pr view` against `apache\u002Fdatafusion-comet` to fetch the title, body, author, draft state,\nstate, and changed files.\n\n## 3. Review Existing Comments First\n\nBefore forming your review:\n\n1. **Read all existing review comments** on the PR\n2. **Check the conversation tab** for any discussion\n3. **Avoid duplicating feedback** that others have already provided\n4. **Build on existing discussions** rather than starting new threads on the same topic\n5. **If you have no additional concerns beyond what is already discussed, say so**\n6. **Ignore Copilot reviews.** Do not reference or build upon comments from GitHub Copilot.\n\n`gh pr view \u003Cpr> --repo apache\u002Fdatafusion-comet --comments` shows them.\n\n## 4. Read the Diff\n\n`gh pr diff \u003Cpr> --repo apache\u002Fdatafusion-comet`\n\nRead the surrounding code, not just the diff hunks. Comet has strong local conventions, and the most\ncommon review finding is a change that works but does not match how the neighbouring code solves the\nsame problem.\n\n## 5. Checks That Apply to Every PR\n\n### Spark compatibility\n\nThis is the point of the project. Comet must produce the same results as Spark, or declare that it\ndoes not. Whatever the PR changes, ask what Spark does in the same situation and where the answer\ncame from. \"It looks right\" is not an answer. Reading the Spark source is.\n\n### Support levels\n\nAnything that reports `getSupportLevel()` must report it honestly:\n\n- `Compatible()`, matches Spark exactly\n- `Incompatible(Some(\"reason\"))`, differs in documented ways and requires\n  `spark.comet.expression.\u003CName>.allowIncompatible=true`\n- `Unsupported(Some(\"reason\"))`, cannot be implemented\n\nA PR that quietly marks something `Compatible` while the diff shows a known divergence is the single\nmost important thing to catch. Keep reasons concise and link a tracking issue when the behavior is\nknown to differ.\n\nTwo disguised forms of the same thing:\n\n- A divergence documented in the compatibility guide on a path that is enabled by default. If the\n  guide says the results differ, the default has to fall back for that case (#5469, #5507).\n- A test switched to `ignore` in the PR because Comet's answer now differs from Spark's. An ignored\n  test is a known divergence, so the code needs a fallback for that case (#5701).\n\n### Behavior change against the latest release\n\nUsers upgrade from a release, not from `main`. So the question is not only what the PR changes\nrelative to `main`, but whether a user moving from the latest release to a build with this PR sees\ndifferent behavior. `main` may already carry unreleased changes in the same code, and a PR that\nlooks harmless against `main` can finish turning a released behavior into a different one.\n\nFind the latest release branch, the highest `branch-X.Y`:\n\n```shell\ngit ls-remote --heads https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet 'branch-*' | sort -t- -k2 -V | tail -1\n```\n\nFetch the PR head and diff the files the PR touches against that branch, not against `main`:\n\n```shell\ngit fetch https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet branch-X.Y:release-X.Y pull\u002F\u003Cpr>\u002Fhead:pr-\u003Cpr>\ngit diff release-X.Y pr-\u003Cpr> -- \u003Cchanged files>\n```\n\nThen ask, for each code path the PR touches, whether any of these differ from the release:\n\n- the result for some input, including null, NaN, `-0.0`, empty, overflow, and timezone cases\n- whether a query raises an error, and which error\n- whether an operator or expression runs natively or falls back to Spark\n- a config default, a config name, or a support level\n- performance or memory use on an existing path\n\nThe tests the PR adds are a good probe. If a new test would fail on the release branch, the PR\nchanges released behavior, and you should know which of those two cases it is. Running the test\nagainst the release branch is the cheapest way to find out when the answer is not obvious from\nthe code.\n\nEvery behavior change against the release is one of two things:\n\n- **Intended.** A bug fix that makes Comet match Spark, or a deliberate change. The PR description\n  should say so, user-facing changes need a note in the user guide or compatibility docs, and a\n  correctness fix should be considered for backport to the release branches per\n  `docs\u002Fsource\u002Fcontributor-guide\u002Fbackporting.md`.\n- **Unintended.** The PR, alone or together with unreleased changes already on `main`, makes Comet\n  diverge from Spark where the release did not, or makes a released path slower. This is a\n  correctness or performance regression, and it falls under the request-changes rule below.\n\nLook hardest when the PR is one of the three kinds of change behind most of the regressions that the\n1.1.0 audit found ([#6399](https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fissues\u002F6399)):\n\n- **A path that becomes native by default.** The query used to fall back to Spark and was right.\n  Find the inputs where the native path differs from Spark, and either test them or fall back for\n  them. Making map and struct literals native exposed a type mismatch in the native `IF` (#6334).\n  Constant metadata columns went native with a per-split value that DataFusion and Spark assign\n  differently (#6505).\n- **A removed fallback or guard.** List everything the fallback was shielding, not just the case the\n  PR is about. Removing the Iceberg complex-type null-check fallback also exposed every `explode` of\n  an Iceberg array to a schema-evolution bug in iceberg-rust (#6504).\n- **A broad routing change.** A catch-all that sends more expressions down a path, such as the JVM\n  codegen dispatcher, admits shapes the path never handled. List the new shapes and test one of each\n  (#6424, #6425).\n\nReport the comparison in the review even when nothing changed, so the reviewer knows it was done.\n\n### Spark version coverage\n\nComet supports several Spark versions. The check on the Spark version belongs in Scala, either in a\nshim under `spark\u002Fsrc\u002Fmain\u002Fspark-*\u002F` or in one named predicate such as `isSpark42Plus` or\n`SparkFixes.slopeFiltersVarByPairNulls`. It does not belong in an ad hoc comparison on the\nversion string in shared code, or in native Rust. If the PR adds a shim for one 4.x version, check\nthat the sibling 4.x source sets got it too.\n\nProtobuf fields, native function arguments, and native function names only ever see the behavior,\nsuch as `wrap_second_millisecond_overflow`, never the version, such as `spark_420_plus` (#6740). A\nfork that backports a Spark fix then changes the one Scala check that sets the flag and leaves the\nprotobuf and Rust alone. Flag a version-named field, argument, or function, and comments that say\n\"Spark 4.2\" for behavior later releases inherit or that leave out the Spark JIRA.\n`Cast.is_spark4_plus` and `get_json_object_spark34` predate the rule and #6891 tracks renaming them,\nso flag them only when a PR adds behavior behind them.\n\nWhen Spark changed the behavior in a patch release, such as SPARK-55969 or SPARK-54918, a check on\nthe minor version is wrong for every earlier patch. CI builds only the newest patch of each line, so\nit can't catch that (#6042, #5701). The pull request CI also runs only the default Spark profile, so\nlogic that depends on the Spark version needs the matching `run-spark-*` labels (#6156).\n\n### Timestamps and timezones\n\nTimezone bugs are easy to miss in review and in tests. A mislabelled timestamp column passes any\ntest that only projects it, and a result computed in the wrong timezone looks plausible. A PR deals\nwith timezones if it touches a datetime expression, a cast to or from a timestamp, how a scan reads\ntimestamps, the timestamp type at the JVM\u002Fnative boundary, or anything that reads the session\ntimezone. Searching the diff flags most of these PRs:\n\n```shell\ngh pr diff \u003Cpr> --repo apache\u002Fdatafusion-comet | grep -inE 'time_?zone|zoneid|chrono_tz|timestamp(ntz)?type|timestampmicro|timestamp\\('\n```\n\nFor such a PR, hold the diff against \"The invariant\" and \"Guidelines\" in\n`docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md`. Look for these first:\n\n- A `TimestampType` value labelled with the session timezone, or with no timezone. Inside a native\n  plan every `TimestampType` value is labelled exactly `\"UTC\"`, and every `TimestampNTZType` value\n  has no timezone. Check the declared type and the arrays the code builds, not only the values.\n- A timezone taken from the JVM default or the host. An expression uses the `timeZoneId` Spark\n  stamped on it, passed through `CometTimeZone.nativeId`, not `SQLConf.get.sessionLocalTimeZone`.\n- A path gated on a UTC session. `Etc\u002FUTC` is the session default on Ubuntu and Debian images, so\n  check what the gate does with it, and that the output there is still labelled `\"UTC\"` rather than\n  `\"Etc\u002FUTC\"`.\n- A timezone applied to a `TimestampNTZType` value, other than to convert it to `TimestampType`.\n- Tests that use a single session timezone, or only project the result. \"Testing timezone-sensitive\n  code\" in the same page says what to ask for.\n\n`timezones.md` also describes specific code: where the `\"UTC\"` label is set, which serdes serialize\na timezone, how `CometTimeZone` rewrites timezone IDs, what `array_with_timezone` does, how the\nscans adapt timestamps, and which expressions go through the codegen dispatcher. A PR that changes\nany of these updates the page in the same PR. The page also documents some known limitations as\ncurrent behavior, such as chrono-tz's DST horizon and the timezone database versions. A fix for one\nof the bugs tracked in [#6335](https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fissues\u002F6335) usually\nchanges that text too.\n\n### Configuration\n\nNew configs go in `CometConf.scala` and must follow\n`docs\u002Fsource\u002Fcontributor-guide\u002Fconfig_conventions.md`. Check the naming, the default, the category,\nand that the description reads as user-facing documentation, because it is. `configs.md` is\ngenerated from it.\n\nA new behavior that is on by default and trades performance for some workloads needs a supported\nconfig, not a testing one, to turn it off before merge (#6466).\n\n### Tests\n\n- Does the PR test the thing it changed, or only that nothing else broke?\n- Are the tests in the right framework for the area? The area skills say which.\n- Does a bug fix come with a test that fails without the fix?\n- Do the tests compare against Spark? A comparison with Comet's own accumulator, or with a second\n  implementation such as iceberg-rust instead of the Iceberg Java that Spark runs, can agree on the\n  wrong answer (#6423, #6426).\n- New suites must be registered in both `.github\u002Fworkflows\u002Fpr_build_linux.yml` and\n  `pr_build_macos.yml`.\n\nFor a change on a default path, look for tests with the inputs that broke earlier changes:\n\n- `-0.0` next to `0.0`, and NaN with different payloads, including inside arrays and structs (#5469,\n  #5507, #5701)\n- Values at a type or unit boundary, including negative timestamps before 1970 (#6426)\n- A constant that binary floating point can't represent, such as 0.1, aggregated across partitions\n  (#6423)\n- A batch where every row takes the same branch (#6334)\n- Output that the native side slices, so it reaches the JVM past the first batch with a non-zero\n  offset (#6464)\n- A Parquet file that Spark splits into several partitions, an empty file, and an Iceberg file\n  written before a nested field was added (#6504, #6505, #6506)\n- Data that defeats a heuristic, such as distinct keys at the start of a task followed by repeats\n  (#6466)\n- One of a pair of aliases configured without the other (#5825)\n\n### CI\n\n`gh pr checks \u003Cpr> --repo apache\u002Fdatafusion-comet`. For only the failed checks, add\n`--json name,state,bucket,link --jq '.[] | select(.bucket == \"fail\")'`.\n\nSummarize failures in the review. Do not compare against failures on `main`.\n\nSome paths have no CI coverage at all. Nothing in CI reads from a real object store, runs Iceberg's\nforward-compatibility tables, or makes a spilled operator replay under a tight memory pool. A change\non one of those paths needs a local run, and the review should ask for one (#5759, #6254).\n\n### Dependency upgrades\n\nReview a dependency bump as a set of behavior changes, not an API migration. For DataFusion, arrow,\nparquet and iceberg-rust, diff the upstream source of what Comet calls on default paths. Both\nversions are in the local Cargo registry after a build of each side. A test that the bump changes to\n`ignore` or to a new expected value marks a behavior change that needs a fallback or an explicit\ndecision. The DataFusion 55 upgrade changed signed-zero handling in `array_distinct` and\n`array_union` (#5701), the spill replay of the final aggregate (#6254), and, through iceberg-rust,\ntask validation (#5759).\n\n## 6. Documentation Freshness\n\nEvery PR carries a documentation question, and it has two halves.\n\n**Half one: does the PR need to add user-facing documentation?** New user-visible behavior belongs in\nthe user guide.\n\n**Half two: does the PR make existing contributor documentation wrong?** This half is the one that\ngets missed. The contributor guide describes how subsystems actually work today. It contains class\ntables, file paths, config defaults, diagrams, and stated invariants. A PR that renames a class,\nmoves a file between crates, changes a default, adds a case to a selection rule, or changes an\nownership rule silently turns a paragraph of that guide into a lie. The doc update belongs in the\nsame PR, because a follow-up that is not filed as an issue does not happen.\n\nEach area skill names the doc for its subsystem and the specific claims in it that go stale. When you\nload an area skill, read its doc and hold the diff against it.\n\n**Do not ask contributors to hand-edit generated docs.** `GenerateDocs` fills in the config tables\nand the expression compatibility tables when the docs are built, and CI rebuilds and publishes the\ndocs when a merge to `main` changes their inputs. The\n**Generated Content** section of `docs\u002Fsource\u002Fcontributor-guide\u002Fstyle_guide.md` says exactly which\ntext is generated. Review the source that feeds it instead. The prose around the generated tables,\nthe `_category_template\u002F` files, and the `index.md` pages are written by hand and are reviewed\nnormally.\n\n## Review Bar\n\nHold a high bar. Several rounds of review and revision before merge are normal and expected. Prefer\niterating on the PR over merging it and trusting a follow-up, because follow-ups that are not filed\nas issues do not get done. Treat \"we can fix that later\" as \"that will not get fixed.\"\n\nEvery finding must be actionable. Before writing a comment, decide whether it is worth addressing\nbefore merge. If it is, raise it and expect a response. If it is not, cut it. There is no third tier.\n\n- Never label feedback as \"not a blocker\", \"nit\", \"minor\", \"optional\", \"low priority\", or \"feel free\n  to ignore\". Either raise it as something to address or drop it. That label tells the author to skip\n  it and leaves nobody accountable for it.\n- If a finding is real but genuinely belongs in separate work, say that plainly and ask for a\n  tracking issue, then reference the issue link in the review. Do not leave it as a floating remark.\n- Bikeshedding is worse than silence. A preference with no correctness, performance, compatibility,\n  or maintainability argument behind it does not go in the review.\n- This bar is about what gets raised, not about how it is worded. Keep the tone below. A question you\n  expect an answer to still counts as something the author needs to address.\n\n## Request Changes for Correctness and Performance Regressions\n\nTwo kinds of finding have to block the merge until the author addresses them: a correctness problem\nthat the PR introduces, and a performance regression. A correctness problem means Comet can return a\ndifferent answer from Spark, including a result where Spark raises an error, or `Compatible()` over a\nknown divergence. Crashes, hangs, leaks, and lost or corrupted data count too. A performance\nregression means an existing path gets slower or uses more memory. A tracking issue for a later fix\ndoes not address either one.\n\nWhen any finding is one of these, the review is submitted as **Request changes**. `main` needs a\nsingle approval to merge, and any committer's approval counts, including one given before this\nreview. A **Comment** review does not stop that approval from merging the PR over the finding. A\n**Request changes** review from a reviewer with write access blocks the merge until the same reviewer\napproves the PR or someone with write access dismisses the review.\n\nRequesting changes commits the reviewer to the re-review. Once the author has addressed the findings\nbehind it, the reviewer approves the PR or dismisses the review. A later **Comment** review leaves\nthe block in place.\n\n## Output Format\n\nPresent your review as guidance for the reviewer:\n\n1. **PR Summary**, brief description of what the PR does\n2. **Areas**, which sibling skills you loaded and why\n3. **CI Status**, summary of CI check results\n4. **Behavior vs Release**, which release branch you compared against, and every behavior change\n   you found, each marked intended or unintended. Say \"no change\" when there is none.\n5. **Findings**, organized by area\n6. **Suggested Review Comments**, specific comments the reviewer could leave, with file and line\n   references. Everything here is something you expect the author to address. Anything that did not\n   clear the bar above should not appear.\n7. **Review State**, how to submit the review. Use the first case that applies:\n   - **Request changes** when any finding is a correctness problem that the PR introduces or a\n     performance regression, including an unintended behavior change against the latest release.\n     Name those findings.\n   - **Approve**, or dismiss the earlier review, when this is a re-review and the findings behind the\n     reviewer's earlier **Request changes** review have all been addressed.\n   - **Comment** otherwise.\n\n## Review Tone and Style\n\nWrite reviews that sound human and conversational. Avoid:\n\n- Robotic or formulaic language\n- Em dashes. Use separate sentences instead.\n- Semicolons. Use separate sentences instead.\n\nInstead:\n\n- Write in flowing paragraphs using simple grammar\n- Keep sentences short and separate rather than joining them with punctuation\n- Be kind and constructive, even when raising concerns\n- Use backticks around any code references such as function names, file paths, class names, types,\n  and config keys\n- **Suggest** adding tests rather than stating tests are missing\n- **Ask questions** about edge cases rather than asserting they are not handled\n- Frame concerns as questions or suggestions when possible\n- Acknowledge what the PR does well before raising concerns\n\n## Do Not Post Comments\n\n**IMPORTANT: Never post comments or reviews on the PR directly.** This skill and all of its siblings\nare for providing guidance to a human reviewer. Present all findings and suggested comments to the\nuser. The user will decide what to post.\n\nWhen the user tells you to post the review for them, submit it in the review state from the output.\nFor **Request changes** that is\n`gh pr review \u003Cpr> --repo apache\u002Fdatafusion-comet --request-changes --body-file \u003Cfile>`. A PR comment\nor a **Comment** review does not block the merge.\n","\u003C!--\nLicensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License.\n-->\n\nReview Comet PR #$ARGUMENTS\n\nThis is the entry point for every Comet PR review. It covers what applies regardless of what the PR\ntouches. Depth for a specific subsystem lives in a sibling skill, and step 1 tells you which ones to\nload.\n\n## 1. Route to the Area Skills\n\n**Do this before reading the diff in detail.** Look at the list of changed files and load every\nsibling skill whose area the PR touches. More than one usually applies. A shuffle change that alters\nspilling is also a memory change. An expression that returns a new array type may also be an FFI\nchange.\n\n| The PR touches                                                                                                                                                                             | Also use                        |\n| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- |\n| `spark\u002Fsrc\u002Fmain\u002Fscala\u002Forg\u002Fapache\u002Fcomet\u002Fserde\u002F`, `QueryPlanSerde.scala`, `native\u002Fspark-expr\u002F`, `expr.proto`, `comet_scalar_funcs.rs`                                                        | `review-comet-expression-pr`    |\n| `CometExecIterator`, `CometNativeArrowSource`, `NativeUtil`, `CometVector` subclasses, `jni_api.rs`, `scan.rs`, anything using `FFI_Arrow*`                                                | `review-comet-ffi-pr`           |\n| `native\u002Fcore\u002Fsrc\u002Fexecution\u002Fmemory_pools\u002F`, `CometTaskMemoryManager`, `CometShuffleMemoryAllocator`, `MemoryConsumer` \u002F `try_grow` \u002F `reserve` call sites, pool configs                     | `review-comet-memory-pr`        |\n| `native\u002Fshuffle\u002F`, `spark\u002Fsrc\u002Fmain\u002Fscala\u002Forg\u002Fapache\u002Fspark\u002Fsql\u002Fcomet\u002Fexecution\u002Fshuffle\u002F`, `spark\u002Fsrc\u002Fmain\u002Fjava\u002Forg\u002Fapache\u002Fspark\u002Fshuffle\u002F`, `CometShuffleExchangeExec`                       | `review-comet-shuffle-pr`       |\n| `IcebergWriteStrategy`, `IcebergWriteExec`, `IcebergCommitExec`, `CometIcebergWriteExec`, `CometIcebergNativeWrite`, `iceberg_write.rs`, `iceberg_partition_path.rs`, the iceberg-rust pin | `review-comet-iceberg-write-pr` |\n| `docs\u002Fsource\u002F`, `README.md`, `AGENTS.md`, `.ai\u002Fskills\u002F**\u002FSKILL.md`, `.doc(...)` strings in config classes, doc comments                                                                    | `review-comet-docs`             |\n\nFor a new operator, read `docs\u002Fsource\u002Fcontributor-guide\u002Fadding_a_new_operator.md` alongside this\nskill. There is no dedicated operator review skill yet.\n\nFor a PR that deals with timestamps or the session timezone, read\n`docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md` as well, whichever areas it touches. Timezone handling\ncuts across the serde, the native kernels, the scans and the JVM\u002Fnative boundary, so no area skill\nowns it. \"Timestamps and timezones\" in step 5 says what to check.\n\nIf the PR falls outside all of these, for example build, CI, or release tooling, this skill on its\nown is the review.\n\n## 2. Gather PR Metadata\n\nUse `gh pr view` against `apache\u002Fdatafusion-comet` to fetch the title, body, author, draft state,\nstate, and changed files.\n\n## 3. Review Existing Comments First\n\nBefore forming your review:\n\n1. **Read all existing review comments** on the PR\n2. **Check the conversation tab** for any discussion\n3. **Avoid duplicating feedback** that others have already provided\n4. **Build on existing discussions** rather than starting new threads on the same topic\n5. **If you have no additional concerns beyond what is already discussed, say so**\n6. **Ignore Copilot reviews.** Do not reference or build upon comments from GitHub Copilot.\n\n`gh pr view \u003Cpr> --repo apache\u002Fdatafusion-comet --comments` shows them.\n\n## 4. Read the Diff\n\n`gh pr diff \u003Cpr> --repo apache\u002Fdatafusion-comet`\n\nRead the surrounding code, not just the diff hunks. Comet has strong local conventions, and the most\ncommon review finding is a change that works but does not match how the neighbouring code solves the\nsame problem.\n\n## 5. Checks That Apply to Every PR\n\n### Spark compatibility\n\nThis is the point of the project. Comet must produce the same results as Spark, or declare that it\ndoes not. Whatever the PR changes, ask what Spark does in the same situation and where the answer\ncame from. \"It looks right\" is not an answer. Reading the Spark source is.\n\n### Support levels\n\nAnything that reports `getSupportLevel()` must report it honestly:\n\n- `Compatible()`, matches Spark exactly\n- `Incompatible(Some(\"reason\"))`, differs in documented ways and requires\n  `spark.comet.expression.\u003CName>.allowIncompatible=true`\n- `Unsupported(Some(\"reason\"))`, cannot be implemented\n\nA PR that quietly marks something `Compatible` while the diff shows a known divergence is the single\nmost important thing to catch. Keep reasons concise and link a tracking issue when the behavior is\nknown to differ.\n\nTwo disguised forms of the same thing:\n\n- A divergence documented in the compatibility guide on a path that is enabled by default. If the\n  guide says the results differ, the default has to fall back for that case (#5469, #5507).\n- A test switched to `ignore` in the PR because Comet's answer now differs from Spark's. An ignored\n  test is a known divergence, so the code needs a fallback for that case (#5701).\n\n### Behavior change against the latest release\n\nUsers upgrade from a release, not from `main`. So the question is not only what the PR changes\nrelative to `main`, but whether a user moving from the latest release to a build with this PR sees\ndifferent behavior. `main` may already carry unreleased changes in the same code, and a PR that\nlooks harmless against `main` can finish turning a released behavior into a different one.\n\nFind the latest release branch, the highest `branch-X.Y`:\n\n```shell\ngit ls-remote --heads https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet 'branch-*' | sort -t- -k2 -V | tail -1\n```\n\nFetch the PR head and diff the files the PR touches against that branch, not against `main`:\n\n```shell\ngit fetch https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet branch-X.Y:release-X.Y pull\u002F\u003Cpr>\u002Fhead:pr-\u003Cpr>\ngit diff release-X.Y pr-\u003Cpr> -- \u003Cchanged files>\n```\n\nThen ask, for each code path the PR touches, whether any of these differ from the release:\n\n- the result for some input, including null, NaN, `-0.0`, empty, overflow, and timezone cases\n- whether a query raises an error, and which error\n- whether an operator or expression runs natively or falls back to Spark\n- a config default, a config name, or a support level\n- performance or memory use on an existing path\n\nThe tests the PR adds are a good probe. If a new test would fail on the release branch, the PR\nchanges released behavior, and you should know which of those two cases it is. Running the test\nagainst the release branch is the cheapest way to find out when the answer is not obvious from\nthe code.\n\nEvery behavior change against the release is one of two things:\n\n- **Intended.** A bug fix that makes Comet match Spark, or a deliberate change. The PR description\n  should say so, user-facing changes need a note in the user guide or compatibility docs, and a\n  correctness fix should be considered for backport to the release branches per\n  `docs\u002Fsource\u002Fcontributor-guide\u002Fbackporting.md`.\n- **Unintended.** The PR, alone or together with unreleased changes already on `main`, makes Comet\n  diverge from Spark where the release did not, or makes a released path slower. This is a\n  correctness or performance regression, and it falls under the request-changes rule below.\n\nLook hardest when the PR is one of the three kinds of change behind most of the regressions that the\n1.1.0 audit found ([#6399](https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fissues\u002F6399)):\n\n- **A path that becomes native by default.** The query used to fall back to Spark and was right.\n  Find the inputs where the native path differs from Spark, and either test them or fall back for\n  them. Making map and struct literals native exposed a type mismatch in the native `IF` (#6334).\n  Constant metadata columns went native with a per-split value that DataFusion and Spark assign\n  differently (#6505).\n- **A removed fallback or guard.** List everything the fallback was shielding, not just the case the\n  PR is about. Removing the Iceberg complex-type null-check fallback also exposed every `explode` of\n  an Iceberg array to a schema-evolution bug in iceberg-rust (#6504).\n- **A broad routing change.** A catch-all that sends more expressions down a path, such as the JVM\n  codegen dispatcher, admits shapes the path never handled. List the new shapes and test one of each\n  (#6424, #6425).\n\nReport the comparison in the review even when nothing changed, so the reviewer knows it was done.\n\n### Spark version coverage\n\nComet supports several Spark versions. The check on the Spark version belongs in Scala, either in a\nshim under `spark\u002Fsrc\u002Fmain\u002Fspark-*\u002F` or in one named predicate such as `isSpark42Plus` or\n`SparkFixes.slopeFiltersVarByPairNulls`. It does not belong in an ad hoc comparison on the\nversion string in shared code, or in native Rust. If the PR adds a shim for one 4.x version, check\nthat the sibling 4.x source sets got it too.\n\nProtobuf fields, native function arguments, and native function names only ever see the behavior,\nsuch as `wrap_second_millisecond_overflow`, never the version, such as `spark_420_plus` (#6740). A\nfork that backports a Spark fix then changes the one Scala check that sets the flag and leaves the\nprotobuf and Rust alone. Flag a version-named field, argument, or function, and comments that say\n\"Spark 4.2\" for behavior later releases inherit or that leave out the Spark JIRA.\n`Cast.is_spark4_plus` and `get_json_object_spark34` predate the rule and #6891 tracks renaming them,\nso flag them only when a PR adds behavior behind them.\n\nWhen Spark changed the behavior in a patch release, such as SPARK-55969 or SPARK-54918, a check on\nthe minor version is wrong for every earlier patch. CI builds only the newest patch of each line, so\nit can't catch that (#6042, #5701). The pull request CI also runs only the default Spark profile, so\nlogic that depends on the Spark version needs the matching `run-spark-*` labels (#6156).\n\n### Timestamps and timezones\n\nTimezone bugs are easy to miss in review and in tests. A mislabelled timestamp column passes any\ntest that only projects it, and a result computed in the wrong timezone looks plausible. A PR deals\nwith timezones if it touches a datetime expression, a cast to or from a timestamp, how a scan reads\ntimestamps, the timestamp type at the JVM\u002Fnative boundary, or anything that reads the session\ntimezone. Searching the diff flags most of these PRs:\n\n```shell\ngh pr diff \u003Cpr> --repo apache\u002Fdatafusion-comet | grep -inE 'time_?zone|zoneid|chrono_tz|timestamp(ntz)?type|timestampmicro|timestamp\\('\n```\n\nFor such a PR, hold the diff against \"The invariant\" and \"Guidelines\" in\n`docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md`. Look for these first:\n\n- A `TimestampType` value labelled with the session timezone, or with no timezone. Inside a native\n  plan every `TimestampType` value is labelled exactly `\"UTC\"`, and every `TimestampNTZType` value\n  has no timezone. Check the declared type and the arrays the code builds, not only the values.\n- A timezone taken from the JVM default or the host. An expression uses the `timeZoneId` Spark\n  stamped on it, passed through `CometTimeZone.nativeId`, not `SQLConf.get.sessionLocalTimeZone`.\n- A path gated on a UTC session. `Etc\u002FUTC` is the session default on Ubuntu and Debian images, so\n  check what the gate does with it, and that the output there is still labelled `\"UTC\"` rather than\n  `\"Etc\u002FUTC\"`.\n- A timezone applied to a `TimestampNTZType` value, other than to convert it to `TimestampType`.\n- Tests that use a single session timezone, or only project the result. \"Testing timezone-sensitive\n  code\" in the same page says what to ask for.\n\n`timezones.md` also describes specific code: where the `\"UTC\"` label is set, which serdes serialize\na timezone, how `CometTimeZone` rewrites timezone IDs, what `array_with_timezone` does, how the\nscans adapt timestamps, and which expressions go through the codegen dispatcher. A PR that changes\nany of these updates the page in the same PR. The page also documents some known limitations as\ncurrent behavior, such as chrono-tz's DST horizon and the timezone database versions. A fix for one\nof the bugs tracked in [#6335](https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fissues\u002F6335) usually\nchanges that text too.\n\n### Configuration\n\nNew configs go in `CometConf.scala` and must follow\n`docs\u002Fsource\u002Fcontributor-guide\u002Fconfig_conventions.md`. Check the naming, the default, the category,\nand that the description reads as user-facing documentation, because it is. `configs.md` is\ngenerated from it.\n\nA new behavior that is on by default and trades performance for some workloads needs a supported\nconfig, not a testing one, to turn it off before merge (#6466).\n\n### Tests\n\n- Does the PR test the thing it changed, or only that nothing else broke?\n- Are the tests in the right framework for the area? The area skills say which.\n- Does a bug fix come with a test that fails without the fix?\n- Do the tests compare against Spark? A comparison with Comet's own accumulator, or with a second\n  implementation such as iceberg-rust instead of the Iceberg Java that Spark runs, can agree on the\n  wrong answer (#6423, #6426).\n- New suites must be registered in both `.github\u002Fworkflows\u002Fpr_build_linux.yml` and\n  `pr_build_macos.yml`.\n\nFor a change on a default path, look for tests with the inputs that broke earlier changes:\n\n- `-0.0` next to `0.0`, and NaN with different payloads, including inside arrays and structs (#5469,\n  #5507, #5701)\n- Values at a type or unit boundary, including negative timestamps before 1970 (#6426)\n- A constant that binary floating point can't represent, such as 0.1, aggregated across partitions\n  (#6423)\n- A batch where every row takes the same branch (#6334)\n- Output that the native side slices, so it reaches the JVM past the first batch with a non-zero\n  offset (#6464)\n- A Parquet file that Spark splits into several partitions, an empty file, and an Iceberg file\n  written before a nested field was added (#6504, #6505, #6506)\n- Data that defeats a heuristic, such as distinct keys at the start of a task followed by repeats\n  (#6466)\n- One of a pair of aliases configured without the other (#5825)\n\n### CI\n\n`gh pr checks \u003Cpr> --repo apache\u002Fdatafusion-comet`. For only the failed checks, add\n`--json name,state,bucket,link --jq '.[] | select(.bucket == \"fail\")'`.\n\nSummarize failures in the review. Do not compare against failures on `main`.\n\nSome paths have no CI coverage at all. Nothing in CI reads from a real object store, runs Iceberg's\nforward-compatibility tables, or makes a spilled operator replay under a tight memory pool. A change\non one of those paths needs a local run, and the review should ask for one (#5759, #6254).\n\n### Dependency upgrades\n\nReview a dependency bump as a set of behavior changes, not an API migration. For DataFusion, arrow,\nparquet and iceberg-rust, diff the upstream source of what Comet calls on default paths. Both\nversions are in the local Cargo registry after a build of each side. A test that the bump changes to\n`ignore` or to a new expected value marks a behavior change that needs a fallback or an explicit\ndecision. The DataFusion 55 upgrade changed signed-zero handling in `array_distinct` and\n`array_union` (#5701), the spill replay of the final aggregate (#6254), and, through iceberg-rust,\ntask validation (#5759).\n\n## 6. Documentation Freshness\n\nEvery PR carries a documentation question, and it has two halves.\n\n**Half one: does the PR need to add user-facing documentation?** New user-visible behavior belongs in\nthe user guide.\n\n**Half two: does the PR make existing contributor documentation wrong?** This half is the one that\ngets missed. The contributor guide describes how subsystems actually work today. It contains class\ntables, file paths, config defaults, diagrams, and stated invariants. A PR that renames a class,\nmoves a file between crates, changes a default, adds a case to a selection rule, or changes an\nownership rule silently turns a paragraph of that guide into a lie. The doc update belongs in the\nsame PR, because a follow-up that is not filed as an issue does not happen.\n\nEach area skill names the doc for its subsystem and the specific claims in it that go stale. When you\nload an area skill, read its doc and hold the diff against it.\n\n**Do not ask contributors to hand-edit generated docs.** `GenerateDocs` fills in the config tables\nand the expression compatibility tables when the docs are built, and CI rebuilds and publishes the\ndocs when a merge to `main` changes their inputs. The\n**Generated Content** section of `docs\u002Fsource\u002Fcontributor-guide\u002Fstyle_guide.md` says exactly which\ntext is generated. Review the source that feeds it instead. The prose around the generated tables,\nthe `_category_template\u002F` files, and the `index.md` pages are written by hand and are reviewed\nnormally.\n\n## Review Bar\n\nHold a high bar. Several rounds of review and revision before merge are normal and expected. Prefer\niterating on the PR over merging it and trusting a follow-up, because follow-ups that are not filed\nas issues do not get done. Treat \"we can fix that later\" as \"that will not get fixed.\"\n\nEvery finding must be actionable. Before writing a comment, decide whether it is worth addressing\nbefore merge. If it is, raise it and expect a response. If it is not, cut it. There is no third tier.\n\n- Never label feedback as \"not a blocker\", \"nit\", \"minor\", \"optional\", \"low priority\", or \"feel free\n  to ignore\". Either raise it as something to address or drop it. That label tells the author to skip\n  it and leaves nobody accountable for it.\n- If a finding is real but genuinely belongs in separate work, say that plainly and ask for a\n  tracking issue, then reference the issue link in the review. Do not leave it as a floating remark.\n- Bikeshedding is worse than silence. A preference with no correctness, performance, compatibility,\n  or maintainability argument behind it does not go in the review.\n- This bar is about what gets raised, not about how it is worded. Keep the tone below. A question you\n  expect an answer to still counts as something the author needs to address.\n\n## Request Changes for Correctness and Performance Regressions\n\nTwo kinds of finding have to block the merge until the author addresses them: a correctness problem\nthat the PR introduces, and a performance regression. A correctness problem means Comet can return a\ndifferent answer from Spark, including a result where Spark raises an error, or `Compatible()` over a\nknown divergence. Crashes, hangs, leaks, and lost or corrupted data count too. A performance\nregression means an existing path gets slower or uses more memory. A tracking issue for a later fix\ndoes not address either one.\n\nWhen any finding is one of these, the review is submitted as **Request changes**. `main` needs a\nsingle approval to merge, and any committer's approval counts, including one given before this\nreview. A **Comment** review does not stop that approval from merging the PR over the finding. A\n**Request changes** review from a reviewer with write access blocks the merge until the same reviewer\napproves the PR or someone with write access dismisses the review.\n\nRequesting changes commits the reviewer to the re-review. Once the author has addressed the findings\nbehind it, the reviewer approves the PR or dismisses the review. A later **Comment** review leaves\nthe block in place.\n\n## Output Format\n\nPresent your review as guidance for the reviewer:\n\n1. **PR Summary**, brief description of what the PR does\n2. **Areas**, which sibling skills you loaded and why\n3. **CI Status**, summary of CI check results\n4. **Behavior vs Release**, which release branch you compared against, and every behavior change\n   you found, each marked intended or unintended. Say \"no change\" when there is none.\n5. **Findings**, organized by area\n6. **Suggested Review Comments**, specific comments the reviewer could leave, with file and line\n   references. Everything here is something you expect the author to address. Anything that did not\n   clear the bar above should not appear.\n7. **Review State**, how to submit the review. Use the first case that applies:\n   - **Request changes** when any finding is a correctness problem that the PR introduces or a\n     performance regression, including an unintended behavior change against the latest release.\n     Name those findings.\n   - **Approve**, or dismiss the earlier review, when this is a re-review and the findings behind the\n     reviewer's earlier **Request changes** review have all been addressed.\n   - **Comment** otherwise.\n\n## Review Tone and Style\n\nWrite reviews that sound human and conversational. Avoid:\n\n- Robotic or formulaic language\n- Em dashes. Use separate sentences instead.\n- Semicolons. Use separate sentences instead.\n\nInstead:\n\n- Write in flowing paragraphs using simple grammar\n- Keep sentences short and separate rather than joining them with punctuation\n- Be kind and constructive, even when raising concerns\n- Use backticks around any code references such as function names, file paths, class names, types,\n  and config keys\n- **Suggest** adding tests rather than stating tests are missing\n- **Ask questions** about edge cases rather than asserting they are not handled\n- Frame concerns as questions or suggestions when possible\n- Acknowledge what the PR does well before raising concerns\n\n## Do Not Post Comments\n\n**IMPORTANT: Never post comments or reviews on the PR directly.** This skill and all of its siblings\nare for providing guidance to a human reviewer. Present all findings and suggested comments to the\nuser. The user will decide what to post.\n\nWhen the user tells you to post the review for them, submit it in the review state from the output.\nFor **Request changes** that is\n`gh pr review \u003Cpr> --repo apache\u002Fdatafusion-comet --request-changes --body-file \u003Cfile>`. A PR comment\nor a **Comment** review does not block the merge.\n",{"name":5,"description":7,"argument-hint":44},"\u003Cpr-number>","main","2f9552d022eb84a79b31b3870c70be9725fdb41f",".ai\u002Fskills\u002Freview-comet-pr",418,"https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet","https:\u002F\u002Fdatafusion.apache.org\u002Fcomet","https:\u002F\u002Fgithub.com\u002Fapache","https:\u002F\u002Fwww.apache.org\u002F","2026-11-15 03:15:20",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add apache\u002Fdatafusion-comet --skill review-comet-pr",{"tool":60,"command":61},"bunx","bunx skills add apache\u002Fdatafusion-comet --skill review-comet-pr",{"tool":63,"command":64},"pnpm","pnpm dlx skills add apache\u002Fdatafusion-comet --skill review-comet-pr","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-pr\u002FSKILL.md",true,[],[69,98,124,139,150,165],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":80,"publishedAt":39},"01M6ACJR50ZS7F1QX58JEKCGVM","wire-datafusion-function","Use when wiring an existing DataFusion or datafusion-spark function into Comet for a Spark expression. Identifies the right wiring pattern (one-line passthrough, datafusion-spark UDF registration, or custom serde with input massaging \u002F restrictions), applies the Scala serde, registers the UDF in jni_api when needed, and adds SQL file tests. Assumes the function already exists upstream — if not, switch to `implement-comet-expression`.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Fwire-datafusion-function\u002FSKILL.md",[75],{"categoryNo":76,"categoryName":77,"taxonomyType":78,"taxonomyTypeDesc":79},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[34,81,82,83,84,85,86,87,24,88,89,90,91,92,93,94,25,95,96,97,31],"wire datafusion function","implement","sql","scala","spark","file","api","switch","registration","upstream","tests","name","expression","with","Existing","custom","wiring",{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":39},"01M6ACJPDDPND6FHE2PC1CJ4D3","suggest-native-expression","Use when picking the next Spark expression to implement natively in Comet, or when asked whether a specific codegen-dispatched expression is worth a native Rust implementation. Scores the candidate on how likely a native path can be 100% Spark-compatible and how much throughput or allocation it would save versus the JVM codegen dispatcher, records the verdict in the per-expression audit log so disqualified candidates are not re-litigated, and files a GitHub issue for the recommendation.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Fsuggest-native-expression\u002FSKILL.md",[],[105,93,106,107,108,82,85,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,31],"suggest native expression","native","github","rust","dispatcher","audit","save","how","log","issue","files","next","jvm","audit log","throughput","re","path","allocation","candidate",{"skillNo":125,"name":126,"nameZh":6,"description":127,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":39},"01M6ACJMHPEN7E76VPKC3P4X6C","review-comet-shuffle-pr","Use when reviewing a DataFusion Comet pull request that touches native or JVM columnar shuffle, the shuffle writers and readers, partitioning, the Arrow IPC block format, shuffle compression, or the Celeborn integration. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-shuffle-pr\u002FSKILL.md",[],[19,21,131,36,38,132,34,133,134,135,24,117,106,136,30,137,138,27,31],"review comet shuffle pr","arrow","integration","compression","format","block","ipc","load",{"skillNo":140,"name":141,"nameZh":6,"description":142,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":143,"categories":144,"keywords":145,"publishedAt":39},"01M6ACJFM0QS61Z1RW516HXN2M","review-comet-memory-pr","Use when reviewing a DataFusion Comet pull request that touches memory pools, memory reservations, spilling, the Spark TaskMemoryManager bridge, allocators, or memory-related configuration. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-memory-pr\u002FSKILL.md",[],[19,35,21,146,85,36,38,34,147,24,148,149,30,138,27,31],"review comet memory pr","bridge","configuration","reservations",{"skillNo":151,"name":152,"nameZh":6,"description":153,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":154,"categories":155,"keywords":156,"publishedAt":39},"01M6ACJCV64V0FZPMVXDE2K3DX","review-comet-iceberg-write-pr","Use when reviewing a DataFusion Comet pull request that touches Iceberg writes, the split-operator write plan (IcebergWriteStrategy, IcebergWriteExec, IcebergCommitExec), the native Iceberg writer (CometIcebergWriteExec, iceberg_write.rs, iceberg_partition_path.rs), its eligibility gate (CometIcebergNativeWrite), or the iceberg-rust pin. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-iceberg-write-pr\u002FSKILL.md",[],[19,21,37,157,158,108,36,38,34,159,160,161,24,162,163,106,164,30,138,31],"review comet iceberg write pr","write","plan","pin","partition","writer","write plan","split",{"skillNo":166,"name":167,"nameZh":6,"description":168,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":169,"categories":170,"keywords":171,"publishedAt":39},"01M6ACJA72KJRAM6F7WHMX01PN","review-comet-ffi-pr","Use when reviewing a DataFusion Comet pull request that crosses the JVM\u002Fnative boundary, touching Arrow C Data or C Stream interface code, batch export and import, CometExecIterator, ScanExec, NativeUtil, CometVector subclasses, or jni_api. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-ffi-pr\u002FSKILL.md",[],[19,21,172,173,36,174,38,132,175,34,176,87,24,117,177,106,178,30,138,27,31],"review comet ffi pr","ffi","code","batch","boundary","import","interface"]