[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6WFHBP7GZJTHP8X9QDARJSR":3,"skill-same:01M6WFHBP7GZJTHP8X9QDARJSR":68},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":45,"commitSha":46,"skillDirectory":47,"forksCount":48,"repositoryHtmlUrl":49,"repositoryHomepageUrl":50,"ownerHtmlUrl":51,"ownerBlogUrl":52,"ownerDescription":6,"updatedAt":53,"install":54,"resources":67},"01M6WFHBP7GZJTHP8X9QDARJSR","review-comet-docs",null,"Use when reviewing the documentation in a DataFusion Comet pull request or branch, or when auditing an existing page or directory under docs\u002Fsource or README.md. Checks wording against the documentation style guide and checks that config keys, operator names, links, images and generated content match the code. Advisory only, it never edits files or posts comments. Load alongside review-comet-pr when a PR changes docs.","apache\u002Fdatafusion-comet","apache","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F47359?v=4","01M6AC83E6K4FDKK7HZ8WNMQKE","The Apache Software Foundation","01M6ACHX42TPNCR8TQJEAR8J5J",1322,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-docs\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"review","review comet docs","docs","only","directory","when","request","source","number","posts","files","images","never","generated","branch","operator","datafusion","code","documentation","config","2026-11-10 10:12:30",false,"---\nname: review-comet-docs\ndescription: Use when reviewing the documentation in a DataFusion Comet pull request or branch, or when auditing an existing page or directory under docs\u002Fsource or README.md. Checks wording against the documentation style guide and checks that config keys, operator names, links, images and generated content match the code. Advisory only, it never edits files or posts comments. Load alongside review-comet-pr when a PR changes docs.\nargument-hint: \u003Cpr-number | branch | docs-path>\n---\n\n\u003C!--\nLicensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License.\n-->\n\nReview Comet docs: $ARGUMENTS\n\nPick the mode from the argument:\n\n- **PR mode**: a number. Review the docs a pull request changes.\n- **Audit mode**: a file or directory that exists in the working tree. Review it as it is now.\n- **Branch mode**: a branch, tag, or commit that `git rev-parse --verify` accepts. Review what it\n  changes compared with `main`, for example before opening a PR. Otherwise, ask which one is meant.\n\nIn PR mode, **REQUIRED BACKGROUND:** use `review-comet-pr` for its steps 2 to 4 (PR metadata,\nexisting comments, and the diff) and its CI check. This skill only covers documentation.\n\nThis skill gives advice to a human. Never edit files, commit, push, or post comments.\n\n**Shell notes.** Run every command from the repository root. Each tool call may start a new shell,\nso save the variables you set in a state file named for this run. Use a different name for each run\nand mode, and keep it the same in every call of that run:\n\n```bash\nSTATE=\"$(git rev-parse --git-dir)\u002Freview-comet-docs-$(printf %s \"\u003Cpr-or-name>\" | tr \u002F -).env\"\n{ printf 'R=%q\\n' \"$R\"; printf 'HEAD_SHA=%q\\n' \"$HEAD_SHA\"; printf 'BASE=%q\\n' \"$BASE\";\n  printf 'TARGET=%q\\n' \"$TARGET\"; printf 'PATHS=%q\\n' \"$PATHS\"; printf 'ADDED=%q\\n' \"$ADDED\"; } \\\n  > \"$STATE\"\n```\n\nStart every later call by loading it:\n\n```bash\nSTATE=\"$(git rev-parse --git-dir)\u002Freview-comet-docs-$(printf %s \"\u003Cpr-or-name>\" | tr \u002F -).env\"; . \"$STATE\"\n```\n\nIn zsh, `$HEAD_SHA:spark\u002F...` is read as a history modifier, so always write `\"${HEAD_SHA}:\u003Cpath>\"`.\nSave the state again after step 3 sets `PATHS` and step 4 sets `ADDED`. Stop and report the review as\nincomplete if a command in steps 1 to 4 reports an error. A search that finds nothing, which makes\n`git grep` exit with status 1, is not an error. Where a step gives a fallback, such as the guide\nlookup in step 2 or the docs build in step 4, use it and keep going.\n\n## 1. Select the Revisions\n\nFind the remote that points at `apache\u002Fdatafusion-comet`, because its name differs between clones:\n\n```bash\nR=$(git remote -v | awk '\u002Fapache\\\u002Fdatafusion-comet(\\.git)? \\(fetch\\)\u002F {print $1; exit}')\n```\n\nIf `R` is empty, use `https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet.git` in its place.\n\n**PR mode.** Read the PR's base branch and head commit, then fetch both without touching the working\ntree. The leading `+` lets the fetch replace a ref after a force push.\n\n```bash\ngh pr view \u003Cpr> --repo apache\u002Fdatafusion-comet --json baseRefName,headRefOid\ngit fetch \"$R\" \"+refs\u002Fheads\u002F\u003Cbase>:refs\u002Freview\u002Fbase-\u003Cpr>\" \"+pull\u002F\u003Cpr>\u002Fhead:refs\u002Freview\u002Fpr-\u003Cpr>\"\nTARGET=refs\u002Freview\u002Fbase-\u003Cpr>; HEAD_SHA=$(git rev-parse refs\u002Freview\u002Fpr-\u003Cpr>)\n```\n\nCheck that `HEAD_SHA` equals `headRefOid`. If it does not, the PR changed while you fetched it: run\nboth commands again, at most twice. From here on, read everything at `HEAD_SHA`, not at the ref,\nwhich a later fetch can move.\n\n**Branch mode.** Fetch `main` the same way. `TARGET=refs\u002Freview\u002Fmain` and\n`HEAD_SHA=$(git rev-parse --verify \u003Cbranch>)`:\n\n```bash\ngit fetch \"$R\" \"+refs\u002Fheads\u002Fmain:refs\u002Freview\u002Fmain\"\n```\n\n**Both modes.** Find the merge base. It fails in a shallow clone, so deepen the history and try again,\nor report the comparison as incomplete:\n\n```bash\nBASE=$(git merge-base \"$TARGET\" \"$HEAD_SHA\") ||\n  { git fetch --deepen=1000 \"$R\" && BASE=$(git merge-base \"$TARGET\" \"$HEAD_SHA\"); } ||\n  { echo \"no merge base: report the comparison as incomplete\" >&2; false; }\n```\n\nIf `git rev-list --count \"$BASE..$TARGET\"` is not zero, the target branch has moved since the change\nbranched. Check the claims against `$TARGET` too, and report what has changed there.\n\n**Audit mode.** There is no base. List the files, including new files that are not committed yet, and\nsave the list as `PATHS`, as step 3 does in the other modes. The mechanical checks run on every file\nin it; only the judgment checks in step 5 are capped:\n\n```bash\nPATHS=\"${STATE%.env}.paths\"\ngit -c core.quotePath=false ls-files --cached --others --exclude-standard -- \u003Cpath> > \"$PATHS\"\n```\n\n## 2. Read the Style Guide\n\nRead `docs\u002Fsource\u002Fcontributor-guide\u002Fstyle_guide.md` in full on every run, from the revision under\nreview: `git show \"${HEAD_SHA}:docs\u002Fsource\u002Fcontributor-guide\u002Fstyle_guide.md\"` in PR and branch mode,\nor the working tree in audit mode. If it does not exist there, as for a change made before the guide\nwas added, read it from `$TARGET`. If it does not exist there either, read the working tree's copy and say in\nthe report which copy you used. Stop and ask the user for a revision only when there is no copy at\nall.\n\nThe guide is the rulebook, and its first section says what it covers. Cite its rule IDs (`T3`, `N1`,\n`P16`, and so on), and do not restate or reinterpret its rules here. If the guide and this skill\ndisagree, the guide wins.\n\n## 3. Select the Files\n\nIn PR and branch mode, list the changed files from Git, which is complete even for large PRs:\n\n```bash\ngit -c core.quotePath=false diff --name-only \"$BASE\" \"$HEAD_SHA\"\n```\n\nFrom that list, `\u003Cpaths>` below means:\n\n- `.md` and `.rst` files under `docs\u002Fsource`, and `README.md`;\n- files that change `.doc(...)` strings in `CometConf.scala` or in contrib config classes;\n- files that change doc comments (Scaladoc, Javadoc, and Rust `\u002F\u002F\u002F` comments);\n- `AGENTS.md` and `.ai\u002Fskills\u002F**\u002FSKILL.md`;\n- `.svg` files under `docs\u002Fsource`, whose `\u003Ctitle>`, `\u003Cdesc>`, and `\u003Ctext>` elements are text readers\n  see.\n\nSave that list to a file, one path per line. Commands read it with `tr '\\n' '\\0' \u003C \"$PATHS\" | xargs -0 -r`,\nwhich passes each path as one argument, so a file name with a space stays one path. Code files are\nall kept here; step 4 keeps only their doc comments:\n\n```bash\nPATHS=\"${STATE%.env}.paths\"\ngit -c core.quotePath=false diff --name-only \"$BASE\" \"$HEAD_SHA\" \\\n  | grep -E '^(docs\u002Fsource\u002F.*\\.(md|rst|svg)|README\\.md|AGENTS\\.md|\\.ai\u002Fskills\u002F.*\u002FSKILL\\.md|.*\\.(scala|java|rs))$' \\\n  | grep -v '\u002Fsrc\u002Ftest\u002F' > \"$PATHS\"\n```\n\nFor doc comments, `AGENTS.md`, and skills, check the vocabulary and accuracy rules the guide names\nfor them, not the page conventions. Check the changed lines and the paragraphs around them, not the\nwhole page. Report a problem in unchanged text only when the change makes it wrong, or list it\nseparately as already on the page. Also read the code diff for the behavior the changed docs\ndescribe, because docs are usually only as accurate as the code change beside them.\n\nIn every mode:\n\n- Skip the content the guide's **Generated Content** section lists as generated or as records.\n  Review the hand-written text around the generated markers. On `main` the text between the markers\n  is empty, because the docs build fills it in, so any text a change adds between them is a hand\n  edit, and a finding (`P5`).\n- Do the judgment checks in step 5 for at most 10 files per run, counting each file that has lines\n  in `$ADDED` after the code-file filter in step 4. In audit mode, count the files step 1 lists, in\n  that order. List any pages you skipped so the user can run the skill again on them.\n\n## 4. Mechanical Checks\n\nThese checks produce candidates, not findings. Read each hit in context before reporting it. For the\nwording checks, ignore hits inside fenced code blocks, inline code, quoted plan output or logs, and\ntext that quotes a phrase in order to discuss it (`N6`), such as the style guide's own examples. The\naccuracy checks still apply to code font: config keys, operator names, and code examples are usually\nwritten that way. For each check, record what it checked, the number of hits, and how many of them\nare findings.\n\nThe commands use `git grep -E` to find candidate lines and `perl` to match them. They do not need a\nGit built with PCRE, and they avoid two traps: the system `grep` on macOS has no `-P`, and `\\b`\nmisses Markdown italics such as `_native execution_`, because `_` counts as a word character.\n\n**Lines to check.** In PR and branch mode, list the added lines with their file and line number, and\nrun the line checks below on this list only. In audit mode, run them on the files with\n`git grep -n --untracked`.\n\n```bash\nADDED=$(mktemp \"$(git rev-parse --git-dir)\u002Freview-added.XXXXXX\")\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false diff --no-ext-diff --src-prefix=a\u002F --dst-prefix=b\u002F -U0 \\\n    \"$BASE\" \"$HEAD_SHA\" -- \\\n  | awk '\u002F^--- \u002F{h=1; next}\n         h && \u002F^\\+\\+\\+ b\\\u002F\u002F{f=substr($0,7); sub(\u002F\\t$\u002F,\"\",f); h=0; next}\n         \u002F^@@\u002F{h=0; split($3,a,\u002F[+,]\u002F); n=a[2]; next}\n         \u002F^\\+\u002F{print f\":\"n\": \"substr($0,2); n++}' > \"$ADDED\"\n```\n\nIn a code file, only doc comments and `.doc(...)` strings are text to check. Before the wording checks,\nleave out its other added lines:\n\n```bash\nperl -i -ne 'print if !\u002F\\.(scala|java|rs):\\d+: \u002F || \u002F:\\d+: \\s*(\\\u002F\\*\\*|\\*|\\\u002F\\\u002F[\\\u002F!]|\\.doc\\(|\")\u002F' \"$ADDED\"\n```\n\n**\"Native\" and \"vectorized\".** Check each hit against `N1`-`N6` and `T10`. Which page the hit is on\nmatters, because `N3` allows more in the contributor guide:\n\n```bash\nP='(?:(?\u003C![A-Za-z0-9])native(?:ly)?(?![A-Za-z0-9])|vectori[sz]ed)'\nperl -ne \"print if \u002F^.+?:\\d+: .*$P\u002Fi\" \"$ADDED\"\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false grep -n -i --untracked -E 'native|vectori[sz]ed' -- \\\n  | perl -ne \"print if \u002F^.+?:\\d+:.*$P\u002Fi\"  # audit\n```\n\n**Config keys.** Every `spark.comet.*` key a page names must exist (`P13`). List the keys the text\nnames, then the keys the code defines at the revision under review. `\u003Crev>` is `\"$HEAD_SHA\"`. In\naudit mode, leave `\u003Crev>` out entirely: an empty `\"$HEAD_SHA\"` is an empty revision, which Git\nrejects. The second command reads definitions that span two lines, expands\n`createExecEnabledConfig`, and leaves out deprecated alternative names and unexpanded templates:\n\n```bash\nperl -ne 'print \"$1\\n\" while \u002F(spark\\.comet\\.[\\w.\u003C>]*[\\w>])\u002Fg' \"$ADDED\" | sort -u\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false grep -h --untracked -o \\\n    -E 'spark\\.comet\\.[A-Za-z0-9_.\u003C>]*[A-Za-z0-9_>]' -- | sort -u  # audit\ngit grep -h -A1 -E '(conf|ConfigBuilder|createExecEnabledConfig)\\(' \u003Crev> -- spark\u002Fsrc\u002Fmain contrib \\\n  | perl -0777 -ne '\n      while (\u002F\\b(?:conf|ConfigBuilder)\\(\\s*s?\"([^\"]+)\"\u002Fg) {\n        ($k = $1) =~ s\u002F\\$\\{?COMET_EXEC_CONFIG_PREFIX\\}?\u002Fspark.comet.exec\u002F;\n        $k =~ s\u002F\\$\\{?COMET_PREFIX\\}?\u002Fspark.comet\u002F;\n        print \"$k\\n\" if $k =~ \u002F^spark\\.comet\\.\u002F && $k !~ \u002F\\$\u002F;\n      }\n      while (\u002F\\bcreateExecEnabledConfig\\(\\s*\"([^\"]+)\"\u002Fg) { print \"spark.comet.exec.$1.enabled\\n\" }' \\\n  | sort -u\n```\n\nIf the second command prints nothing, it is broken: stop and report that, rather than flagging every\nkey. Some keys are built from a name at run time:\n\n- `spark.comet.expression.\u003CName>.enabled` and `.allowIncompatible`, also written with an uppercase\n  placeholder such as `EXPRNAME`: `\u003CName>` is the Spark expression\n  class name, as in the `QueryPlanSerde` serde maps, not a Comet or protobuf name. Some expressions\n  use their own opt-in key instead, set by `nativeOptInConfigKeyOverride`, such as\n  `spark.comet.caseConversion.enabled`.\n- `spark.comet.operator.\u003CName>.allowIncompatible`: `\u003CName>` is the Spark operator class name, such as\n  `ProjectExec`.\n- `spark.comet.datafusion.\u003Ckey>`: not a Comet config. Comet passes it to DataFusion only when\n  `spark.comet.exec.respectDataFusionConfigs=true`, which is off by default. Check the key against\n  DataFusion.\n\nBefore reporting a missing key, search the code for it, because a few keys are defined in other\nways, such as `spark.comet.version` in `Plugins.scala`. A deprecated key that a page still uses is a\nfinding (`P16`).\n\n**Operator names.** Check operator names against `O1`, class names against `O2`, and expressions\nagainst `O4`. List the concrete operator classes and the classes that override their printed name:\n\n```bash\ngit grep -h -E 'case class (Comet|Iceberg)' \u003Crev> -- spark\u002Fsrc\u002Fmain contrib \\\n  | perl -ne 'print \"$1\\n\" if \u002Fcase class ((?:Comet|Iceberg)[A-Za-z]+Exec)\\b\u002F' | sort -u\ngit grep -n -E 'override (def|val|lazy val) (nodeName|simpleString)' \u003Crev> -- spark\u002Fsrc\u002Fmain contrib\n```\n\n`spark\u002Fsrc\u002Fmain` includes the per-Spark-version source directories. If the first command prints\nnothing, it is broken: stop and report that. A class's `nodeName` is its name without `Exec`, unless\nit overrides `nodeName`. Compare each name in the text with these, as `O1` describes, not with a rule\nabout suffixes. When the page is an operator reference, also report concrete operators that it\nleaves out. Leave out planning markers such as `CometScanExec`, which never appear in a final plan,\nand contrib operators, which belong on their contrib page. A name the text calls a protobuf message,\nsuch as `ToJson`, is not an operator: check it in `native\u002Fproto` instead. When you search the code\nfor a class name, `git grep -E` has no `\\b`, so end the pattern with `([^A-Za-z0-9_]|$)`.\n\n**Docs build.** Links, anchors, images, toctrees, and page structure, in Markdown and\nreStructuredText, are checked by building the docs the way the published site is built. This\ncatches what a reading of the changed lines cannot, such as a link on an unchanged page that a\nrenamed heading breaks, or a page that no toctree includes. Each revision builds with its own\n`docs\u002Frequirements.txt`, so a change that updates a Sphinx extension is built with the new version.\n`venv` installs those requirements into a virtual environment named after their hash and prints its\npath, so revisions with the same requirements share one environment. Each tool call may start a new\nshell, so define `venv` and `build` in every call that uses them:\n\n```bash\nvenv() {  # $1 = revision, or worktree in audit mode; prints the environment's path\n  if [ \"$1\" = worktree ]; then req=$(cat docs\u002Frequirements.txt) || return 1\n  else req=$(git show \"$1:docs\u002Frequirements.txt\") || return 1; fi\n  V=\"$(git rev-parse --git-dir)\u002Freview-comet-docs-venv-$(printf '%s\\n' \"$req\" | git hash-object --stdin | cut -c1-12)\"\n  [ -x \"$V\u002Fbin\u002Fsphinx-build\" ] || {\n    PY=$(for p in python3.13 python3.12 python3.11 python3.10 python3; do\n      command -v $p >\u002Fdev\u002Fnull && $p -c 'import sys; sys.exit(sys.version_info \u003C (3, 10))' && { echo $p; break; }\n    done)\n    [ -n \"$PY\" ] && \"$PY\" -m venv --clear \"$V\" >&2 && printf '%s\\n' \"$req\" > \"$V.txt\" &&\n      \"$V\u002Fbin\u002Fpip\" install -q --disable-pip-version-check -r \"$V.txt\" >&2\n  } || { rm -rf \"$V\"; echo \"docs build not checked: needs Python 3.10 or later\" >&2; return 1; }\n  printf '%s\\n' \"$V\"\n}\n```\n\nThen build the base and the head and compare their warnings. `build` copies the expression templates\ninto the per-version pages, as `docs\u002Fbuild.sh` does, so links to those pages resolve. It returns an\nerror if any step fails, including Sphinx itself, so a failed build is never read as a clean one:\n\n```bash\nbuild() {  # $1 = revision, or worktree in audit mode; $2 = output directory\n  VENV=$(venv \"$1\") || return 1\n  mkdir -p \"$2\u002Fsrc\u002Fdocs\" || return 1\n  if [ \"$1\" = worktree ]; then\n    cp -R docs\u002Fsource \"$2\u002Fsrc\u002Fdocs\u002F\" && profiles=$(cat docs\u002Fbuild.sh) || return 1\n  else\n    git archive \"$1\" docs\u002Fsource | tar -x -C \"$2\u002Fsrc\" && profiles=$(git show \"$1:docs\u002Fbuild.sh\") ||\n      return 1\n  fi\n  T=\"$2\u002Fsrc\u002Fdocs\u002Fsource\u002Fuser-guide\u002Flatest\u002Fcompatibility\u002Fexpressions\"\n  for p in $(printf '%s\\n' \"$profiles\" | sed -n 's\u002F^SPARK_PROFILES=(\\(.*\\))\u002F\\1\u002Fp'); do\n    mkdir -p \"$T\u002F$p\" && cp \"$T\u002F_category_template\"\u002F*.md \"$T\u002F$p\u002F\" || return 1\n  done\n  rm -rf \"$T\u002F_category_template\"\n  \"$VENV\u002Fbin\u002Fsphinx-build\" -q -b html -n -D mermaid_output_format=raw \"$2\u002Fsrc\u002Fdocs\u002Fsource\" \\\n    \"$2\u002Fhtml\" > \"$2\u002Flog.txt\" 2>&1 || { echo \"sphinx-build failed: $2\u002Flog.txt\" >&2; return 1; }\n  sed -e \"s#$2\u002Fsrc\u002F##\" -e 's\u002F:[0-9]*: \u002F: \u002F' \"$2\u002Flog.txt\" | sort > \"$2\u002Fwarnings.txt\"\n}\nWORK=$(cd \"$(mktemp -d)\" && pwd -P)\nbuild \"$BASE\" \"$WORK\u002Fbase\" && build \"$HEAD_SHA\" \"$WORK\u002Fhead\" &&\n  comm -13 \"$WORK\u002Fbase\u002Fwarnings.txt\" \"$WORK\u002Fhead\u002Fwarnings.txt\"\nbuild worktree \"$WORK\u002Fhead\" && cat \"$WORK\u002Fhead\u002Fwarnings.txt\"  # audit\n```\n\nEvery warning that only the head build has is a finding: a broken link, anchor, or image, a page\nmissing from a toctree (`P6`), or a heading problem (`P2`). One exception: the published per-version\npages under `compatibility\u002Fexpressions\u002Fspark-*\u002F` also get a `## \u003CExpression>` heading from\n`GenerateDocs` for each expression whose serde returns compatibility notes, incompatible reasons, or\nunsupported reasons (`renderExpressionCompatNotes`). This build copies only the templates, so a link\nto one of those anchors warns here even when the published page has it. Before reporting such a link\nas checked by hand, confirm both: the serde map that `GenerateDocs.categoryPages` uses for the\nlinked page (`string.md` reads `QueryPlanSerde.stringExpressions`, `math.md` reads\n`mathExpressions`, and so on) contains the expression's Spark class at the linked Spark version,\nsince some are registered only in a version shim; and that class's serde returns one of those notes.\nThe heading is the class's simple name, such as `## Concat`, so a link to that heading on another\ncategory page is a broken link. In audit mode, report the warnings for the audited pages. If the\ndocs requirements cannot be installed or a build fails, say that the build was not checked, and\ncheck that each changed link's target file exists.\n\nSphinx does not count H1s. Check that each new or changed Markdown page has one H1 with the parser\nthe docs build uses, which knows where code blocks start and end. It uses `venv` from above;\nwithout an environment, count the H1s by reading the page and say so:\n\n```bash\nH1='import sys; from markdown_it import MarkdownIt\nprint(sum(t.type == \"heading_open\" and t.tag == \"h1\" for t in MarkdownIt(\"commonmark\").parse(sys.stdin.read())))'\ngit show \"${HEAD_SHA}:\u003Cfile>\" | \"$(venv \"$HEAD_SHA\")\u002Fbin\u002Fpython\" -c \"$H1\"\n\"$(venv worktree)\u002Fbin\u002Fpython\" -c \"$H1\" \u003C \u003Cfile>  # audit\n```\n\n**Other structure checks.** A new file has the ASF header (`P1`). A link to Comet's published docs by\nURL breaks `P4` on pages under `docs\u002Fsource`, but not in `README.md`. List the pages the change\nrenames or deletes; each one needs an entry in `redirects` in `docs\u002Fsource\u002Fconf.py` (`P17`). This\ncheck needs a base, so audit mode skips it:\n\n```bash\ngit diff --name-status --diff-filter=DR \"$BASE\" \"$HEAD_SHA\" -- docs\u002Fsource\n```\n\n**External links.** For external links on the lines under review (at most 20 in audit mode), check\nthe status with `curl -s -o \u002Fdev\u002Fnull -w '%{http_code}' -I -L --max-time 10 \u003Curl>`. Report a 404 as\na finding. If the request fails or the network is unavailable, say the link was not checked.\n\n**Version claims and markers.** Find version claims to check against `P8`, and `TODO` or `XXX`\nmarkers, which break `P7`:\n\n```bash\nV='(?:(?:Spark|Iceberg) [0-9]\\.[0-9x]+\\+?|\\b(?:TODO|XXX)\\b)'\nperl -ne \"print if \u002F^.+?:\\d+: .*$V\u002F\" \"$ADDED\"\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false grep -n --untracked -E 'Spark [0-9]|Iceberg [0-9]|TODO|XXX' -- \\\n  | perl -ne \"print if \u002F$V\u002F\"  # audit\n```\n\nCI runs prettier and the Apache RAT license check on PRs, so do not repeat them. If CI has not run\nthem yet, for example on a draft, say they were not checked. Mention them otherwise only when CI has\nfailed on them.\n\n## 5. Judgment Checks\n\n- **Is it true?** For every sentence that names a class, rule, config, default, or file, read the\n  code it describes at `HEAD_SHA`, or in the working tree in audit mode. A sentence can follow\n  every wording rule and still be wrong: a scan that no longer exists, a config default that has\n  changed, or an operator described as Rust-implemented that runs on the JVM. Check numbers, versions, and names first, and\n  check that a condition in the text is the whole condition in the code. Pages that describe\n  architecture or test infrastructure go stale most often. A false or outdated claim breaks `P16`.\n  This is the most valuable check in this skill.\n- **Does it agree with the rest of the docs?** A new sentence can contradict an older one on the\n  same page, or a count such as \"five configs\" can stop matching the list below it (`P16`). When a\n  change rewrites the description of a config or class, search `docs\u002Fsource` for the same name and\n  check the other descriptions.\n- **Which axis?** When you suggest a replacement for vague \"native\", choose the term the code\n  supports: _in the Comet pipeline_ (`T1`), _Rust-implemented_ (`T3`), or _JVM-side_ (`T4`). Never\n  suggest _Rust-implemented_ without checking the code.\n- **Page conventions.** Check `P3`, `P7`, `P8`, `P9`, `P10`, `P11`, `P12`, `P14`, and `P18` by\n  reading the page. For plan output (`P12`, `P16`), check that the operator names exist, that the\n  tree shape is one Spark could print, and that messages in it match the strings in the code. Do not\n  ask for new plan output to be invented.\n- **Same-change updates.** If the change adds an operator or expression, check the pages `P15`\n  lists. If it adds a config, check that its `.doc(...)` string follows the guide, since\n  `GenerateDocs` publishes it, and that any page that documents the config by hand mentions it.\n\n## 6. Severity\n\n- **Must-fix.** The page is wrong or broken: a false or outdated claim (`P16`), a config key, class,\n  or file that does not exist (`P13`), a broken link, anchor, or image, a hand edit to generated\n  content, a new file without the ASF header, or a new page missing from a toctree. When a sentence\n  is both false and badly worded, report it once, as must-fix, with a rewrite that fixes both.\n- **Should-fix.** The page breaks a style guide rule, but what it says is true. Most `T`, `N`, `O`,\n  and `P` findings belong here.\n- **Nit.** A one-off change that would make the text clearer, but that no rule requires. Nits carry\n  no rule ID.\n\nA problem that keeps coming up and that no rule covers is a **guide gap**, not a finding. Report it\nso the guide can be extended.\n\n## 7. Output\n\nWhen this skill runs as part of a `review-comet-pr` review, put its findings into that skill's\n**Findings** and **Suggested Review Comments**, and drop the nits. That skill's review bar has no\nthird tier, and its rules win for anything posted to a PR. On its own, use this format:\n\n1. **Scope.** The mode, the revisions reviewed (the base and head commits, and which guide version),\n   the files reviewed, and the files skipped, with the reason.\n2. **Mechanical checks.** A table with each check, its hit count, and how many hits are findings.\n3. **Findings**, grouped as must-fix, should-fix, and nit. For each one give the `path:line`, the\n   rule ID, the text quoted from the page, and a suggested rewrite. Keep rewrites short and plain,\n   and change only what the rule requires.\n4. **Guide gaps.**\n\nIf there are no findings, say so.\n\n## Checking This Skill\n\nAfter changing this skill or the style guide, run it on these cases and compare the results. The\nexpected results were checked on `main` in October 2026, so a later fix to a page can change them.\nEach row lists results the run must report and false positives it must not. A run can also report\nother real findings.\n\n| Run                                                                                               | Expected                                                                                                                     |\n| ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |\n| PR mode on #6786, which predates the guide, so step 2 reads it from `$TARGET` or the working tree | Must-fix: `datasources.md` and the new `.doc` string claim more than `shouldBridgeUnsupportedScan` does                      |\n| Audit `docs\u002Fsource\u002Fuser-guide\u002Flatest\u002Funderstanding-comet-plans.md`                                | No finding for Comet's _native shuffle_, `CometNativeScan`, `CometWindowExec`, or config keys                                |\n| Audit `docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md`                                                | No finding for _native code_ or _native plan_, or for `ToJson` and `ToCsv`, which are protobuf message names                 |\n| Audit `docs\u002Fsource\u002Fuser-guide\u002Flatest\u002Fcompatibility\u002Fexpressions`                                   | Lists all 16 files, reviews 10 of the templates and index pages, names the skipped ones, and skips only text between markers |\n| Audit a scratch page containing `_native execution_` and \"Spark's native shuffle\"                 | `N1` and `N5` findings                                                                                                       |\n","\u003C!--\nLicensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License.\n-->\n\nReview Comet docs: $ARGUMENTS\n\nPick the mode from the argument:\n\n- **PR mode**: a number. Review the docs a pull request changes.\n- **Audit mode**: a file or directory that exists in the working tree. Review it as it is now.\n- **Branch mode**: a branch, tag, or commit that `git rev-parse --verify` accepts. Review what it\n  changes compared with `main`, for example before opening a PR. Otherwise, ask which one is meant.\n\nIn PR mode, **REQUIRED BACKGROUND:** use `review-comet-pr` for its steps 2 to 4 (PR metadata,\nexisting comments, and the diff) and its CI check. This skill only covers documentation.\n\nThis skill gives advice to a human. Never edit files, commit, push, or post comments.\n\n**Shell notes.** Run every command from the repository root. Each tool call may start a new shell,\nso save the variables you set in a state file named for this run. Use a different name for each run\nand mode, and keep it the same in every call of that run:\n\n```bash\nSTATE=\"$(git rev-parse --git-dir)\u002Freview-comet-docs-$(printf %s \"\u003Cpr-or-name>\" | tr \u002F -).env\"\n{ printf 'R=%q\\n' \"$R\"; printf 'HEAD_SHA=%q\\n' \"$HEAD_SHA\"; printf 'BASE=%q\\n' \"$BASE\";\n  printf 'TARGET=%q\\n' \"$TARGET\"; printf 'PATHS=%q\\n' \"$PATHS\"; printf 'ADDED=%q\\n' \"$ADDED\"; } \\\n  > \"$STATE\"\n```\n\nStart every later call by loading it:\n\n```bash\nSTATE=\"$(git rev-parse --git-dir)\u002Freview-comet-docs-$(printf %s \"\u003Cpr-or-name>\" | tr \u002F -).env\"; . \"$STATE\"\n```\n\nIn zsh, `$HEAD_SHA:spark\u002F...` is read as a history modifier, so always write `\"${HEAD_SHA}:\u003Cpath>\"`.\nSave the state again after step 3 sets `PATHS` and step 4 sets `ADDED`. Stop and report the review as\nincomplete if a command in steps 1 to 4 reports an error. A search that finds nothing, which makes\n`git grep` exit with status 1, is not an error. Where a step gives a fallback, such as the guide\nlookup in step 2 or the docs build in step 4, use it and keep going.\n\n## 1. Select the Revisions\n\nFind the remote that points at `apache\u002Fdatafusion-comet`, because its name differs between clones:\n\n```bash\nR=$(git remote -v | awk '\u002Fapache\\\u002Fdatafusion-comet(\\.git)? \\(fetch\\)\u002F {print $1; exit}')\n```\n\nIf `R` is empty, use `https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet.git` in its place.\n\n**PR mode.** Read the PR's base branch and head commit, then fetch both without touching the working\ntree. The leading `+` lets the fetch replace a ref after a force push.\n\n```bash\ngh pr view \u003Cpr> --repo apache\u002Fdatafusion-comet --json baseRefName,headRefOid\ngit fetch \"$R\" \"+refs\u002Fheads\u002F\u003Cbase>:refs\u002Freview\u002Fbase-\u003Cpr>\" \"+pull\u002F\u003Cpr>\u002Fhead:refs\u002Freview\u002Fpr-\u003Cpr>\"\nTARGET=refs\u002Freview\u002Fbase-\u003Cpr>; HEAD_SHA=$(git rev-parse refs\u002Freview\u002Fpr-\u003Cpr>)\n```\n\nCheck that `HEAD_SHA` equals `headRefOid`. If it does not, the PR changed while you fetched it: run\nboth commands again, at most twice. From here on, read everything at `HEAD_SHA`, not at the ref,\nwhich a later fetch can move.\n\n**Branch mode.** Fetch `main` the same way. `TARGET=refs\u002Freview\u002Fmain` and\n`HEAD_SHA=$(git rev-parse --verify \u003Cbranch>)`:\n\n```bash\ngit fetch \"$R\" \"+refs\u002Fheads\u002Fmain:refs\u002Freview\u002Fmain\"\n```\n\n**Both modes.** Find the merge base. It fails in a shallow clone, so deepen the history and try again,\nor report the comparison as incomplete:\n\n```bash\nBASE=$(git merge-base \"$TARGET\" \"$HEAD_SHA\") ||\n  { git fetch --deepen=1000 \"$R\" && BASE=$(git merge-base \"$TARGET\" \"$HEAD_SHA\"); } ||\n  { echo \"no merge base: report the comparison as incomplete\" >&2; false; }\n```\n\nIf `git rev-list --count \"$BASE..$TARGET\"` is not zero, the target branch has moved since the change\nbranched. Check the claims against `$TARGET` too, and report what has changed there.\n\n**Audit mode.** There is no base. List the files, including new files that are not committed yet, and\nsave the list as `PATHS`, as step 3 does in the other modes. The mechanical checks run on every file\nin it; only the judgment checks in step 5 are capped:\n\n```bash\nPATHS=\"${STATE%.env}.paths\"\ngit -c core.quotePath=false ls-files --cached --others --exclude-standard -- \u003Cpath> > \"$PATHS\"\n```\n\n## 2. Read the Style Guide\n\nRead `docs\u002Fsource\u002Fcontributor-guide\u002Fstyle_guide.md` in full on every run, from the revision under\nreview: `git show \"${HEAD_SHA}:docs\u002Fsource\u002Fcontributor-guide\u002Fstyle_guide.md\"` in PR and branch mode,\nor the working tree in audit mode. If it does not exist there, as for a change made before the guide\nwas added, read it from `$TARGET`. If it does not exist there either, read the working tree's copy and say in\nthe report which copy you used. Stop and ask the user for a revision only when there is no copy at\nall.\n\nThe guide is the rulebook, and its first section says what it covers. Cite its rule IDs (`T3`, `N1`,\n`P16`, and so on), and do not restate or reinterpret its rules here. If the guide and this skill\ndisagree, the guide wins.\n\n## 3. Select the Files\n\nIn PR and branch mode, list the changed files from Git, which is complete even for large PRs:\n\n```bash\ngit -c core.quotePath=false diff --name-only \"$BASE\" \"$HEAD_SHA\"\n```\n\nFrom that list, `\u003Cpaths>` below means:\n\n- `.md` and `.rst` files under `docs\u002Fsource`, and `README.md`;\n- files that change `.doc(...)` strings in `CometConf.scala` or in contrib config classes;\n- files that change doc comments (Scaladoc, Javadoc, and Rust `\u002F\u002F\u002F` comments);\n- `AGENTS.md` and `.ai\u002Fskills\u002F**\u002FSKILL.md`;\n- `.svg` files under `docs\u002Fsource`, whose `\u003Ctitle>`, `\u003Cdesc>`, and `\u003Ctext>` elements are text readers\n  see.\n\nSave that list to a file, one path per line. Commands read it with `tr '\\n' '\\0' \u003C \"$PATHS\" | xargs -0 -r`,\nwhich passes each path as one argument, so a file name with a space stays one path. Code files are\nall kept here; step 4 keeps only their doc comments:\n\n```bash\nPATHS=\"${STATE%.env}.paths\"\ngit -c core.quotePath=false diff --name-only \"$BASE\" \"$HEAD_SHA\" \\\n  | grep -E '^(docs\u002Fsource\u002F.*\\.(md|rst|svg)|README\\.md|AGENTS\\.md|\\.ai\u002Fskills\u002F.*\u002FSKILL\\.md|.*\\.(scala|java|rs))$' \\\n  | grep -v '\u002Fsrc\u002Ftest\u002F' > \"$PATHS\"\n```\n\nFor doc comments, `AGENTS.md`, and skills, check the vocabulary and accuracy rules the guide names\nfor them, not the page conventions. Check the changed lines and the paragraphs around them, not the\nwhole page. Report a problem in unchanged text only when the change makes it wrong, or list it\nseparately as already on the page. Also read the code diff for the behavior the changed docs\ndescribe, because docs are usually only as accurate as the code change beside them.\n\nIn every mode:\n\n- Skip the content the guide's **Generated Content** section lists as generated or as records.\n  Review the hand-written text around the generated markers. On `main` the text between the markers\n  is empty, because the docs build fills it in, so any text a change adds between them is a hand\n  edit, and a finding (`P5`).\n- Do the judgment checks in step 5 for at most 10 files per run, counting each file that has lines\n  in `$ADDED` after the code-file filter in step 4. In audit mode, count the files step 1 lists, in\n  that order. List any pages you skipped so the user can run the skill again on them.\n\n## 4. Mechanical Checks\n\nThese checks produce candidates, not findings. Read each hit in context before reporting it. For the\nwording checks, ignore hits inside fenced code blocks, inline code, quoted plan output or logs, and\ntext that quotes a phrase in order to discuss it (`N6`), such as the style guide's own examples. The\naccuracy checks still apply to code font: config keys, operator names, and code examples are usually\nwritten that way. For each check, record what it checked, the number of hits, and how many of them\nare findings.\n\nThe commands use `git grep -E` to find candidate lines and `perl` to match them. They do not need a\nGit built with PCRE, and they avoid two traps: the system `grep` on macOS has no `-P`, and `\\b`\nmisses Markdown italics such as `_native execution_`, because `_` counts as a word character.\n\n**Lines to check.** In PR and branch mode, list the added lines with their file and line number, and\nrun the line checks below on this list only. In audit mode, run them on the files with\n`git grep -n --untracked`.\n\n```bash\nADDED=$(mktemp \"$(git rev-parse --git-dir)\u002Freview-added.XXXXXX\")\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false diff --no-ext-diff --src-prefix=a\u002F --dst-prefix=b\u002F -U0 \\\n    \"$BASE\" \"$HEAD_SHA\" -- \\\n  | awk '\u002F^--- \u002F{h=1; next}\n         h && \u002F^\\+\\+\\+ b\\\u002F\u002F{f=substr($0,7); sub(\u002F\\t$\u002F,\"\",f); h=0; next}\n         \u002F^@@\u002F{h=0; split($3,a,\u002F[+,]\u002F); n=a[2]; next}\n         \u002F^\\+\u002F{print f\":\"n\": \"substr($0,2); n++}' > \"$ADDED\"\n```\n\nIn a code file, only doc comments and `.doc(...)` strings are text to check. Before the wording checks,\nleave out its other added lines:\n\n```bash\nperl -i -ne 'print if !\u002F\\.(scala|java|rs):\\d+: \u002F || \u002F:\\d+: \\s*(\\\u002F\\*\\*|\\*|\\\u002F\\\u002F[\\\u002F!]|\\.doc\\(|\")\u002F' \"$ADDED\"\n```\n\n**\"Native\" and \"vectorized\".** Check each hit against `N1`-`N6` and `T10`. Which page the hit is on\nmatters, because `N3` allows more in the contributor guide:\n\n```bash\nP='(?:(?\u003C![A-Za-z0-9])native(?:ly)?(?![A-Za-z0-9])|vectori[sz]ed)'\nperl -ne \"print if \u002F^.+?:\\d+: .*$P\u002Fi\" \"$ADDED\"\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false grep -n -i --untracked -E 'native|vectori[sz]ed' -- \\\n  | perl -ne \"print if \u002F^.+?:\\d+:.*$P\u002Fi\"  # audit\n```\n\n**Config keys.** Every `spark.comet.*` key a page names must exist (`P13`). List the keys the text\nnames, then the keys the code defines at the revision under review. `\u003Crev>` is `\"$HEAD_SHA\"`. In\naudit mode, leave `\u003Crev>` out entirely: an empty `\"$HEAD_SHA\"` is an empty revision, which Git\nrejects. The second command reads definitions that span two lines, expands\n`createExecEnabledConfig`, and leaves out deprecated alternative names and unexpanded templates:\n\n```bash\nperl -ne 'print \"$1\\n\" while \u002F(spark\\.comet\\.[\\w.\u003C>]*[\\w>])\u002Fg' \"$ADDED\" | sort -u\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false grep -h --untracked -o \\\n    -E 'spark\\.comet\\.[A-Za-z0-9_.\u003C>]*[A-Za-z0-9_>]' -- | sort -u  # audit\ngit grep -h -A1 -E '(conf|ConfigBuilder|createExecEnabledConfig)\\(' \u003Crev> -- spark\u002Fsrc\u002Fmain contrib \\\n  | perl -0777 -ne '\n      while (\u002F\\b(?:conf|ConfigBuilder)\\(\\s*s?\"([^\"]+)\"\u002Fg) {\n        ($k = $1) =~ s\u002F\\$\\{?COMET_EXEC_CONFIG_PREFIX\\}?\u002Fspark.comet.exec\u002F;\n        $k =~ s\u002F\\$\\{?COMET_PREFIX\\}?\u002Fspark.comet\u002F;\n        print \"$k\\n\" if $k =~ \u002F^spark\\.comet\\.\u002F && $k !~ \u002F\\$\u002F;\n      }\n      while (\u002F\\bcreateExecEnabledConfig\\(\\s*\"([^\"]+)\"\u002Fg) { print \"spark.comet.exec.$1.enabled\\n\" }' \\\n  | sort -u\n```\n\nIf the second command prints nothing, it is broken: stop and report that, rather than flagging every\nkey. Some keys are built from a name at run time:\n\n- `spark.comet.expression.\u003CName>.enabled` and `.allowIncompatible`, also written with an uppercase\n  placeholder such as `EXPRNAME`: `\u003CName>` is the Spark expression\n  class name, as in the `QueryPlanSerde` serde maps, not a Comet or protobuf name. Some expressions\n  use their own opt-in key instead, set by `nativeOptInConfigKeyOverride`, such as\n  `spark.comet.caseConversion.enabled`.\n- `spark.comet.operator.\u003CName>.allowIncompatible`: `\u003CName>` is the Spark operator class name, such as\n  `ProjectExec`.\n- `spark.comet.datafusion.\u003Ckey>`: not a Comet config. Comet passes it to DataFusion only when\n  `spark.comet.exec.respectDataFusionConfigs=true`, which is off by default. Check the key against\n  DataFusion.\n\nBefore reporting a missing key, search the code for it, because a few keys are defined in other\nways, such as `spark.comet.version` in `Plugins.scala`. A deprecated key that a page still uses is a\nfinding (`P16`).\n\n**Operator names.** Check operator names against `O1`, class names against `O2`, and expressions\nagainst `O4`. List the concrete operator classes and the classes that override their printed name:\n\n```bash\ngit grep -h -E 'case class (Comet|Iceberg)' \u003Crev> -- spark\u002Fsrc\u002Fmain contrib \\\n  | perl -ne 'print \"$1\\n\" if \u002Fcase class ((?:Comet|Iceberg)[A-Za-z]+Exec)\\b\u002F' | sort -u\ngit grep -n -E 'override (def|val|lazy val) (nodeName|simpleString)' \u003Crev> -- spark\u002Fsrc\u002Fmain contrib\n```\n\n`spark\u002Fsrc\u002Fmain` includes the per-Spark-version source directories. If the first command prints\nnothing, it is broken: stop and report that. A class's `nodeName` is its name without `Exec`, unless\nit overrides `nodeName`. Compare each name in the text with these, as `O1` describes, not with a rule\nabout suffixes. When the page is an operator reference, also report concrete operators that it\nleaves out. Leave out planning markers such as `CometScanExec`, which never appear in a final plan,\nand contrib operators, which belong on their contrib page. A name the text calls a protobuf message,\nsuch as `ToJson`, is not an operator: check it in `native\u002Fproto` instead. When you search the code\nfor a class name, `git grep -E` has no `\\b`, so end the pattern with `([^A-Za-z0-9_]|$)`.\n\n**Docs build.** Links, anchors, images, toctrees, and page structure, in Markdown and\nreStructuredText, are checked by building the docs the way the published site is built. This\ncatches what a reading of the changed lines cannot, such as a link on an unchanged page that a\nrenamed heading breaks, or a page that no toctree includes. Each revision builds with its own\n`docs\u002Frequirements.txt`, so a change that updates a Sphinx extension is built with the new version.\n`venv` installs those requirements into a virtual environment named after their hash and prints its\npath, so revisions with the same requirements share one environment. Each tool call may start a new\nshell, so define `venv` and `build` in every call that uses them:\n\n```bash\nvenv() {  # $1 = revision, or worktree in audit mode; prints the environment's path\n  if [ \"$1\" = worktree ]; then req=$(cat docs\u002Frequirements.txt) || return 1\n  else req=$(git show \"$1:docs\u002Frequirements.txt\") || return 1; fi\n  V=\"$(git rev-parse --git-dir)\u002Freview-comet-docs-venv-$(printf '%s\\n' \"$req\" | git hash-object --stdin | cut -c1-12)\"\n  [ -x \"$V\u002Fbin\u002Fsphinx-build\" ] || {\n    PY=$(for p in python3.13 python3.12 python3.11 python3.10 python3; do\n      command -v $p >\u002Fdev\u002Fnull && $p -c 'import sys; sys.exit(sys.version_info \u003C (3, 10))' && { echo $p; break; }\n    done)\n    [ -n \"$PY\" ] && \"$PY\" -m venv --clear \"$V\" >&2 && printf '%s\\n' \"$req\" > \"$V.txt\" &&\n      \"$V\u002Fbin\u002Fpip\" install -q --disable-pip-version-check -r \"$V.txt\" >&2\n  } || { rm -rf \"$V\"; echo \"docs build not checked: needs Python 3.10 or later\" >&2; return 1; }\n  printf '%s\\n' \"$V\"\n}\n```\n\nThen build the base and the head and compare their warnings. `build` copies the expression templates\ninto the per-version pages, as `docs\u002Fbuild.sh` does, so links to those pages resolve. It returns an\nerror if any step fails, including Sphinx itself, so a failed build is never read as a clean one:\n\n```bash\nbuild() {  # $1 = revision, or worktree in audit mode; $2 = output directory\n  VENV=$(venv \"$1\") || return 1\n  mkdir -p \"$2\u002Fsrc\u002Fdocs\" || return 1\n  if [ \"$1\" = worktree ]; then\n    cp -R docs\u002Fsource \"$2\u002Fsrc\u002Fdocs\u002F\" && profiles=$(cat docs\u002Fbuild.sh) || return 1\n  else\n    git archive \"$1\" docs\u002Fsource | tar -x -C \"$2\u002Fsrc\" && profiles=$(git show \"$1:docs\u002Fbuild.sh\") ||\n      return 1\n  fi\n  T=\"$2\u002Fsrc\u002Fdocs\u002Fsource\u002Fuser-guide\u002Flatest\u002Fcompatibility\u002Fexpressions\"\n  for p in $(printf '%s\\n' \"$profiles\" | sed -n 's\u002F^SPARK_PROFILES=(\\(.*\\))\u002F\\1\u002Fp'); do\n    mkdir -p \"$T\u002F$p\" && cp \"$T\u002F_category_template\"\u002F*.md \"$T\u002F$p\u002F\" || return 1\n  done\n  rm -rf \"$T\u002F_category_template\"\n  \"$VENV\u002Fbin\u002Fsphinx-build\" -q -b html -n -D mermaid_output_format=raw \"$2\u002Fsrc\u002Fdocs\u002Fsource\" \\\n    \"$2\u002Fhtml\" > \"$2\u002Flog.txt\" 2>&1 || { echo \"sphinx-build failed: $2\u002Flog.txt\" >&2; return 1; }\n  sed -e \"s#$2\u002Fsrc\u002F##\" -e 's\u002F:[0-9]*: \u002F: \u002F' \"$2\u002Flog.txt\" | sort > \"$2\u002Fwarnings.txt\"\n}\nWORK=$(cd \"$(mktemp -d)\" && pwd -P)\nbuild \"$BASE\" \"$WORK\u002Fbase\" && build \"$HEAD_SHA\" \"$WORK\u002Fhead\" &&\n  comm -13 \"$WORK\u002Fbase\u002Fwarnings.txt\" \"$WORK\u002Fhead\u002Fwarnings.txt\"\nbuild worktree \"$WORK\u002Fhead\" && cat \"$WORK\u002Fhead\u002Fwarnings.txt\"  # audit\n```\n\nEvery warning that only the head build has is a finding: a broken link, anchor, or image, a page\nmissing from a toctree (`P6`), or a heading problem (`P2`). One exception: the published per-version\npages under `compatibility\u002Fexpressions\u002Fspark-*\u002F` also get a `## \u003CExpression>` heading from\n`GenerateDocs` for each expression whose serde returns compatibility notes, incompatible reasons, or\nunsupported reasons (`renderExpressionCompatNotes`). This build copies only the templates, so a link\nto one of those anchors warns here even when the published page has it. Before reporting such a link\nas checked by hand, confirm both: the serde map that `GenerateDocs.categoryPages` uses for the\nlinked page (`string.md` reads `QueryPlanSerde.stringExpressions`, `math.md` reads\n`mathExpressions`, and so on) contains the expression's Spark class at the linked Spark version,\nsince some are registered only in a version shim; and that class's serde returns one of those notes.\nThe heading is the class's simple name, such as `## Concat`, so a link to that heading on another\ncategory page is a broken link. In audit mode, report the warnings for the audited pages. If the\ndocs requirements cannot be installed or a build fails, say that the build was not checked, and\ncheck that each changed link's target file exists.\n\nSphinx does not count H1s. Check that each new or changed Markdown page has one H1 with the parser\nthe docs build uses, which knows where code blocks start and end. It uses `venv` from above;\nwithout an environment, count the H1s by reading the page and say so:\n\n```bash\nH1='import sys; from markdown_it import MarkdownIt\nprint(sum(t.type == \"heading_open\" and t.tag == \"h1\" for t in MarkdownIt(\"commonmark\").parse(sys.stdin.read())))'\ngit show \"${HEAD_SHA}:\u003Cfile>\" | \"$(venv \"$HEAD_SHA\")\u002Fbin\u002Fpython\" -c \"$H1\"\n\"$(venv worktree)\u002Fbin\u002Fpython\" -c \"$H1\" \u003C \u003Cfile>  # audit\n```\n\n**Other structure checks.** A new file has the ASF header (`P1`). A link to Comet's published docs by\nURL breaks `P4` on pages under `docs\u002Fsource`, but not in `README.md`. List the pages the change\nrenames or deletes; each one needs an entry in `redirects` in `docs\u002Fsource\u002Fconf.py` (`P17`). This\ncheck needs a base, so audit mode skips it:\n\n```bash\ngit diff --name-status --diff-filter=DR \"$BASE\" \"$HEAD_SHA\" -- docs\u002Fsource\n```\n\n**External links.** For external links on the lines under review (at most 20 in audit mode), check\nthe status with `curl -s -o \u002Fdev\u002Fnull -w '%{http_code}' -I -L --max-time 10 \u003Curl>`. Report a 404 as\na finding. If the request fails or the network is unavailable, say the link was not checked.\n\n**Version claims and markers.** Find version claims to check against `P8`, and `TODO` or `XXX`\nmarkers, which break `P7`:\n\n```bash\nV='(?:(?:Spark|Iceberg) [0-9]\\.[0-9x]+\\+?|\\b(?:TODO|XXX)\\b)'\nperl -ne \"print if \u002F^.+?:\\d+: .*$V\u002F\" \"$ADDED\"\ntr '\\n' '\\0' \u003C \"$PATHS\" \\\n  | xargs -0 -r git -c core.quotePath=false grep -n --untracked -E 'Spark [0-9]|Iceberg [0-9]|TODO|XXX' -- \\\n  | perl -ne \"print if \u002F$V\u002F\"  # audit\n```\n\nCI runs prettier and the Apache RAT license check on PRs, so do not repeat them. If CI has not run\nthem yet, for example on a draft, say they were not checked. Mention them otherwise only when CI has\nfailed on them.\n\n## 5. Judgment Checks\n\n- **Is it true?** For every sentence that names a class, rule, config, default, or file, read the\n  code it describes at `HEAD_SHA`, or in the working tree in audit mode. A sentence can follow\n  every wording rule and still be wrong: a scan that no longer exists, a config default that has\n  changed, or an operator described as Rust-implemented that runs on the JVM. Check numbers, versions, and names first, and\n  check that a condition in the text is the whole condition in the code. Pages that describe\n  architecture or test infrastructure go stale most often. A false or outdated claim breaks `P16`.\n  This is the most valuable check in this skill.\n- **Does it agree with the rest of the docs?** A new sentence can contradict an older one on the\n  same page, or a count such as \"five configs\" can stop matching the list below it (`P16`). When a\n  change rewrites the description of a config or class, search `docs\u002Fsource` for the same name and\n  check the other descriptions.\n- **Which axis?** When you suggest a replacement for vague \"native\", choose the term the code\n  supports: _in the Comet pipeline_ (`T1`), _Rust-implemented_ (`T3`), or _JVM-side_ (`T4`). Never\n  suggest _Rust-implemented_ without checking the code.\n- **Page conventions.** Check `P3`, `P7`, `P8`, `P9`, `P10`, `P11`, `P12`, `P14`, and `P18` by\n  reading the page. For plan output (`P12`, `P16`), check that the operator names exist, that the\n  tree shape is one Spark could print, and that messages in it match the strings in the code. Do not\n  ask for new plan output to be invented.\n- **Same-change updates.** If the change adds an operator or expression, check the pages `P15`\n  lists. If it adds a config, check that its `.doc(...)` string follows the guide, since\n  `GenerateDocs` publishes it, and that any page that documents the config by hand mentions it.\n\n## 6. Severity\n\n- **Must-fix.** The page is wrong or broken: a false or outdated claim (`P16`), a config key, class,\n  or file that does not exist (`P13`), a broken link, anchor, or image, a hand edit to generated\n  content, a new file without the ASF header, or a new page missing from a toctree. When a sentence\n  is both false and badly worded, report it once, as must-fix, with a rewrite that fixes both.\n- **Should-fix.** The page breaks a style guide rule, but what it says is true. Most `T`, `N`, `O`,\n  and `P` findings belong here.\n- **Nit.** A one-off change that would make the text clearer, but that no rule requires. Nits carry\n  no rule ID.\n\nA problem that keeps coming up and that no rule covers is a **guide gap**, not a finding. Report it\nso the guide can be extended.\n\n## 7. Output\n\nWhen this skill runs as part of a `review-comet-pr` review, put its findings into that skill's\n**Findings** and **Suggested Review Comments**, and drop the nits. That skill's review bar has no\nthird tier, and its rules win for anything posted to a PR. On its own, use this format:\n\n1. **Scope.** The mode, the revisions reviewed (the base and head commits, and which guide version),\n   the files reviewed, and the files skipped, with the reason.\n2. **Mechanical checks.** A table with each check, its hit count, and how many hits are findings.\n3. **Findings**, grouped as must-fix, should-fix, and nit. For each one give the `path:line`, the\n   rule ID, the text quoted from the page, and a suggested rewrite. Keep rewrites short and plain,\n   and change only what the rule requires.\n4. **Guide gaps.**\n\nIf there are no findings, say so.\n\n## Checking This Skill\n\nAfter changing this skill or the style guide, run it on these cases and compare the results. The\nexpected results were checked on `main` in October 2026, so a later fix to a page can change them.\nEach row lists results the run must report and false positives it must not. A run can also report\nother real findings.\n\n| Run                                                                                               | Expected                                                                                                                     |\n| ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |\n| PR mode on #6786, which predates the guide, so step 2 reads it from `$TARGET` or the working tree | Must-fix: `datasources.md` and the new `.doc` string claim more than `shouldBridgeUnsupportedScan` does                      |\n| Audit `docs\u002Fsource\u002Fuser-guide\u002Flatest\u002Funderstanding-comet-plans.md`                                | No finding for Comet's _native shuffle_, `CometNativeScan`, `CometWindowExec`, or config keys                                |\n| Audit `docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md`                                                | No finding for _native code_ or _native plan_, or for `ToJson` and `ToCsv`, which are protobuf message names                 |\n| Audit `docs\u002Fsource\u002Fuser-guide\u002Flatest\u002Fcompatibility\u002Fexpressions`                                   | Lists all 16 files, reviews 10 of the templates and index pages, names the skipped ones, and skips only text between markers |\n| Audit a scratch page containing `_native execution_` and \"Spark's native shuffle\"                 | `N1` and `N5` findings                                                                                                       |\n",{"name":5,"description":7,"argument-hint":44},"\u003Cpr-number | branch | docs-path>","main","4de6704391c9a7832a2079fc4e8d80fc09e9b209",".ai\u002Fskills\u002Freview-comet-docs",418,"https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet","https:\u002F\u002Fdatafusion.apache.org\u002Fcomet","https:\u002F\u002Fgithub.com\u002Fapache","https:\u002F\u002Fwww.apache.org\u002F","2026-11-15 03:15:20",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add apache\u002Fdatafusion-comet --skill review-comet-docs",{"tool":60,"command":61},"bunx","bunx skills add apache\u002Fdatafusion-comet --skill review-comet-docs",{"tool":63,"command":64},"pnpm","pnpm dlx skills add apache\u002Fdatafusion-comet --skill review-comet-docs","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-docs\u002FSKILL.md",true,[],[69,100,125,143,159,170],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":80,"publishedAt":39},"01M6ACJR50ZS7F1QX58JEKCGVM","wire-datafusion-function","Use when wiring an existing DataFusion or datafusion-spark function into Comet for a Spark expression. Identifies the right wiring pattern (one-line passthrough, datafusion-spark UDF registration, or custom serde with input massaging \u002F restrictions), applies the Scala serde, registers the UDF in jni_api when needed, and adds SQL file tests. Assumes the function already exists upstream — if not, switch to `implement-comet-expression`.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Fwire-datafusion-function\u002FSKILL.md",[75],{"categoryNo":76,"categoryName":77,"taxonomyType":78,"taxonomyTypeDesc":79},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[35,81,82,83,84,85,86,87,88,89,90,91,92,93,94,24,95,96,97,98,99],"wire datafusion function","sql","scala","implement","spark","registration","switch","for","upstream","custom","expression","tests","use","Existing","file","api","wiring","with","name",{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":106,"publishedAt":39},"01M6ACJPDDPND6FHE2PC1CJ4D3","suggest-native-expression","Use when picking the next Spark expression to implement natively in Comet, or when asked whether a specific codegen-dispatched expression is worth a native Rust implementation. Scores the candidate on how likely a native path can be 100% Spark-compatible and how much throughput or allocation it would save versus the JVM codegen dispatcher, records the verdict in the per-expression audit log so disqualified candidates are not re-litigated, and files a GitHub issue for the recommendation.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Fsuggest-native-expression\u002FSKILL.md",[],[107,91,108,109,110,111,84,85,112,29,24,113,114,115,116,117,118,119,120,121,122,123,124],"suggest native expression","native","dispatcher","github","rust","re","candidate","log","jvm","next","how","audit","audit log","allocation","issue","throughput","path","save",{"skillNo":126,"name":127,"nameZh":6,"description":128,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":129,"categories":130,"keywords":131,"publishedAt":39},"01M6ACJMHPEN7E76VPKC3P4X6C","review-comet-shuffle-pr","Use when reviewing a DataFusion Comet pull request that touches native or JVM columnar shuffle, the shuffle writers and readers, partitioning, the Arrow IPC block format, shuffle compression, or the Celeborn integration. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-shuffle-pr\u002FSKILL.md",[],[132,133,19,134,135,35,136,93,137,24,138,27,115,108,25,139,140,141,142],"review comet shuffle pr","pr","pull","pull request","arrow","load","ipc","format","integration","block","compression",{"skillNo":144,"name":145,"nameZh":6,"description":146,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":147,"categories":148,"keywords":149,"publishedAt":39},"01M6ACJHX8BES7ZNNRNVCQ4QWE","review-comet-pr","Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, Iceberg writes, and documentation. Provides guidance to a human reviewer rather than posting comments.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-pr\u002FSKILL.md",[],[133,19,150,134,151,135,152,35,153,37,24,27,154,155,25,156,88,93,157,158],"review comet pr","iceberg","memory","workflow","reviewer","human","posting","review skills","comments",{"skillNo":160,"name":161,"nameZh":6,"description":162,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":163,"categories":164,"keywords":165,"publishedAt":39},"01M6ACJFM0QS61Z1RW516HXN2M","review-comet-memory-pr","Use when reviewing a DataFusion Comet pull request that touches memory pools, memory reservations, spilling, the Spark TaskMemoryManager bridge, allocators, or memory-related configuration. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-memory-pr\u002FSKILL.md",[],[133,152,166,19,134,135,35,85,167,168,93,137,24,27,25,169],"review comet memory pr","bridge","reservations","configuration",{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":174,"categories":175,"keywords":176,"publishedAt":39},"01M6ACJCV64V0FZPMVXDE2K3DX","review-comet-iceberg-write-pr","Use when reviewing a DataFusion Comet pull request that touches Iceberg writes, the split-operator write plan (IcebergWriteStrategy, IcebergWriteExec, IcebergCommitExec), the native Iceberg writer (CometIcebergWriteExec, iceberg_write.rs, iceberg_partition_path.rs), its eligibility gate (CometIcebergNativeWrite), or the iceberg-rust pin. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-iceberg-write-pr\u002FSKILL.md",[],[133,151,177,19,178,134,135,35,111,179,180,181,93,137,182,24,27,183,184,108],"review comet iceberg write pr","write","writer","write plan","partition","pin","split","plan"]