[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7B5WJHB4D787EZCY4RNN44T":3,"skill-same:01M7B5WJHB4D787EZCY4RNN44T":69},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":6,"updatedAt":32,"install":33,"resources":46},"01M7B5WJHB4D787EZCY4RNN44T","modelopt-ptq",null,"Use when the user asks to \"quantize a model\", \"run PTQ\", \"post-training quantization\", \"NVFP4 quantization\", \"FP8 quantization\", \"INT8 quantization\", \"INT4 AWQ\", \"quantize LLM\", \"quantize MoE\", \"quantize VLM\", or needs to produce a quantized HuggingFace checkpoint from a pretrained model using ModelOpt. Do NOT use for multi-candidate recipe exploration or optimization (use modelopt-quant-recipe-search).","NVIDIA\u002FModel-Optimizer","NVIDIA","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1728152?v=4","01M6BXXZYTRNBXSTRXNNBB356H","NVIDIA Corporation","01M6BXY02X3PRVHNNATDHA2D0W",5604,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: modelopt-ptq\ndescription: >-\n  Use when the user asks to \"quantize a model\", \"run PTQ\", \"post-training\n  quantization\", \"NVFP4 quantization\", \"FP8 quantization\", \"INT8\n  quantization\", \"INT4 AWQ\", \"quantize LLM\", \"quantize MoE\", \"quantize VLM\",\n  or needs to produce a quantized HuggingFace checkpoint from a pretrained\n  model using ModelOpt. Do NOT use for multi-candidate recipe\n  exploration or optimization (use modelopt-quant-recipe-search).\n---\n\n# ModelOpt Post-Training Quantization\n\nProduce a quantized checkpoint from a pretrained model. **Read `examples\u002Fhf_ptq\u002FREADME.md` first** — it has the support matrix, CLI flags, and accuracy guidance.\n\nUse `modelopt-quant-recipe-search` for multi-candidate recipe exploration or\noptimization. Use this skill for each selected recipe's PTQ run.\n\n## Step 1 — Environment\n\nUse the common skill's `environment-setup.md` and `workspace-management.md`.\nAfter completing them you should know:\n\n- ModelOpt source is available\n- Local or remote (+ cluster config if remote)\n- SLURM \u002F Docker+GPU \u002F bare GPU\n- Launcher available?\n- Which workspace to use\n\n## Step 2 — Is the model supported?\n\nCheck the support table in `examples\u002Fhf_ptq\u002FREADME.md` for verified HF models.\n\n- **Listed** → supported, use `hf_ptq.py` (step 4A\u002F4B)\n- **Not listed** → read `references\u002Funsupported-models.md` to determine if `hf_ptq.py` can still work or if a custom script is needed (step 4C)\n\n## Step 2.5 — Check for model-specific dependencies\n\nIf the model uses `trust_remote_code` (check `config.json` for `auto_map`), inspect its custom Python files for imports not present in the container:\n\n```bash\ngrep -h \"^from \\|^import \" \u003Cmodel_path>\u002Fmodeling_*.py | sort -u\n```\n\n**Known dependency patterns:**\n\n| Import found | Packages to install |\n| --- | --- |\n| `from mamba_ssm` \u002F `from causal_conv1d` | `mamba-ssm causal-conv1d` (Mamba\u002Fhybrid models: NemotronH, Jamba) |\n\nIf extra deps are needed:\n- **Launcher (4B)**: set `EXTRA_PIP_DEPS` in the task's `environment` section — `ptq.sh` installs them automatically\n- **Manual (4A)**: `unset PIP_CONSTRAINT && pip install \u003Cdeps>` before running `hf_ptq.py`\n\n## Step 3 — Choose quantization format\n\n**First**, check for a model-specific recipe:\n\n```bash\nls modelopt_recipes\u002Fmodels\u002F 2>\u002Fdev\u002Fnull\nls modelopt_recipes\u002Fmodel_type\u002F\u003Cmodel_type>\u002Fptq\u002F 2>\u002Fdev\u002Fnull  # per-arch; \u003Cmodel_type> from local config.json (Hub ID: AutoConfig.from_pretrained)\n```\n\nIf a model-specific recipe exists, prefer `--recipe \u003Cpath>` — but **inspect its include\u002Fexclude patterns** rather than assuming (e.g. for VLMs, confirm the vision tower is actually excluded).\n\n**If no model-specific recipe**, choose a format based on GPU (details in `examples\u002Fhf_ptq\u002FREADME.md`):\n\n- **Blackwell** (B100\u002FB200\u002FGB200): `nvfp4` variants\n- **Hopper** (H100\u002FH200) or older: `fp8` or `int4_awq`\n\nUse `--qformat \u003Cname>` (e.g., `--qformat nvfp4`). Format definitions: `modelopt\u002Ftorch\u002Fquantization\u002Fconfig.py`. General PTQ recipes in `modelopt_recipes\u002Fgeneral\u002Fptq\u002F` correspond to the same formats — `--qformat` is the simpler way to use them.\n\nBefore running PTQ, sanity-check the selected qformat\u002Frecipe against the model structure. Inspect the recipe's include\u002Fexclude patterns and summarize which layer groups will be quantized and approximately how many modules\u002Flayers match (attention projections, MLP projections, experts, etc.). If the match count is 0, or far smaller than expected for the model, stop and fix the recipe or ask the user before launching calibration.\n\n**VLMs:** generic `*mlp*`\u002F`*experts*` recipes also match the vision tower (`model.visual.*`); quantizing the ViT silently breaks image benchmarks. Use the `model_type\u002F\u003Cmodel_type>\u002Fptq\u002F` recipe or add `*visual*`\u002F`*vision_tower*` excludes, then verify in Step 5 — see `references\u002Fcheckpoint-validation.md`.\n\nIf the source checkpoint is already quantized and the requested recipe\u002Fconfig reduces quantization coverage, confirm that intent with the user before running. For example, if an FP8 checkpoint is used as input and the recipe excludes some layers so they would fall back to BF16 instead of staying quantized, call out the affected layer groups and ask whether that FP8-to-BF16 fallback is intended.\n\n> NVFP4 can be calibrated on Hopper but requires Blackwell for inference.\n\n## Step 4 — Run PTQ\n\n**Goal: checkpoint on disk** (`.safetensors` + `config.json`).\n\nFor **listed models** (4A\u002F4B): run full calibration directly (`--calib_size 512`).\nFor **unlisted models** (4C): run a smoke test first (`--calib_size 4`), wait for success, then full calibration.\n\n### Recommended calibration datasets\n\n- **Text-only LLM PTQ:** Prefer the representative `nemotron-post-training-v3` blend. `modelopt\u002Ftorch\u002Futils\u002Fdataset_utils.py` expands it to seven registered Nemotron SFT domains. Configure Hugging Face credentials where required.\n\n  ```bash\n  python examples\u002Fhf_ptq\u002Fhf_ptq.py ... \\\n      --dataset nemotron-post-training-v3\n  ```\n\n- **VLM PTQ:** Include image-text calibration with `--calib_with_images`. This path uses `nemotron_vlm_dataset_v2` with the current default subsets `sparsetables`, `plotqa_cot`, and `wiki_en`; `examples\u002Fhf_ptq\u002Fhf_ptq.py` and `modelopt\u002Ftorch\u002Futils\u002Fvlm_dataset_utils.py` are the source of truth.\n\n  ```bash\n  python examples\u002Fhf_ptq\u002Fhf_ptq.py ... \\\n      --calib_with_images\n  ```\n\n`--dataset` selects text-only calibration and cannot substitute for multimodal examples. Use `--dataset cnn_dailymail` only as a fallback when representative data is unavailable, such as without gated-data access or with only a local public cache.\n\n### Which path?\n\n```text\nIn README table? ─→ YES ──→ SLURM (local or remote)? ──→ LAUNCHER (4B)\n                  │          Local Docker + GPU? ────────→ LAUNCHER (4B)\n                  │          Remote Docker (no SLURM)? ──→ MANUAL (4A)\n                  │          Bare GPU (local or remote)? → MANUAL (4A)\n                  │\n                  └→ NOT LISTED ──→ UNLISTED MODEL (4C)\n```\n\n### 4A — Direct: supported model, manual execution\n\n```bash\npip install --no-build-isolation \"nvidia-modelopt[hf]\"\npip install -r examples\u002Fhf_ptq\u002Frequirements.txt\n\npython examples\u002Fhf_ptq\u002Fhf_ptq.py \\\n    --pyt_ckpt_path \u003Cmodel> \\\n    --qformat \u003Cformat> \\\n    --calib_size 512 \\\n    --export_path \u003Coutput>\n```\n\nRun `--help` for all options.\n\nFor remote, use `remote_run` from the common skill's `remote_exec.sh`; see its\n`remote-execution.md`.\n\n### 4B — Launcher: supported model on SLURM or local Docker\n\nWrite a YAML config using `common\u002Fhf\u002Fptq.sh`. See `references\u002Flauncher-guide.md` for the full template.\n\n```bash\ncd tools\u002Flauncher\n# SLURM (remote or local):\nSLURM_HOST=\u003Chost> SLURM_ACCOUNT=\u003Cacct> uv run launch.py --yaml \u003Cconfig.yaml> user=\u003Cssh_user> identity=\u003Cssh_key> --yes\n# Local Docker:\nuv run launch.py --yaml \u003Cconfig.yaml> hf_local=\u003Chf_cache> --yes\n```\n\nThe launcher blocks and tails logs until the job completes. If the launcher fails (missing deps, config errors), fall back to path 4A (manual execution).\n\n### 4C — Unlisted model\n\nFollow `references\u002Funsupported-models.md`. It walks through investigating the model, patching ModelOpt if needed, and running `hf_ptq.py`. Run manually (like 4A) for easier monitoring and debugging.\n\nFor SLURM, see the common skill's `slurm-setup.md` and this skill's\n`references\u002Fslurm-setup-ptq.md`.\n\n### Monitoring\n\nAfter job submission, register the job and set up monitoring per the **modelopt-monitor skill**.\n\n## Step 5 — Verify output\n\n```bash\nls -lh \u003Coutput_path>\u002F\n# Expect: config.json, tokenizer files, model-*.safetensors\n```\n\nReport the path and size to the user.\n\n### Post-quantization validation\n\nThis is a required gate before any deployment or evaluation submission. Do not submit an eval, start a production serving job, or hand off the checkpoint as ready until the gate, including its serving canary, has passed.\n\nRead `references\u002Fcheckpoint-validation.md` and perform all four validation groups on the exact checkpoint path that will be deployed\u002Fevaluated:\n\n1. Check output size and estimated bits per weight against the baseline\u002Fsource checkpoint.\n2. Check quantized-weight coverage against the requested qformat\u002Frecipe\u002Fconfig.\n3. Check metadata consistency against the baseline\u002Fsource model.\n4. Complete the required downstream handoff and serving-readiness validation.\n\nReport the gate result before moving on. Follow the canonical report format and all blocking conditions in `references\u002Fcheckpoint-validation.md`; do not hand off a checkpoint unless every required check passes.\n\n## Key API Rules\n\n- `mtq.register()` classes **must** define `_setup()` and call it from `__init__`\n- Call `mto.enable_huggingface_checkpointing()` **before** quantization\n- Wildcard `*gate*` matches too broadly — use `*mlp.gate*` or `*router*`\n- VLMs: `hf_ptq.py` auto-extracts the language model via `extract_and_prepare_language_model_from_vl()` — no manual VLM handling needed in most cases\n- FP8 checkpoints: prefer `_QuantFP8Linear` (lazy dequant) over `FineGrainedFP8Config(dequantize=True)` which wastes ~2x memory. See `references\u002Funsupported-models.md` for details\n- Custom quantizer names must end with `_input_quantizer` or `_weight_quantizer`\n\n## Common Pitfalls\n\n- **Model-specific dependencies**: Models with `trust_remote_code` may import packages not in the container (e.g., `mamba-ssm` for hybrid Mamba models). See Step 2.5. Use `EXTRA_PIP_DEPS` env var with the launcher, or install manually before running `hf_ptq.py`\n- **Linear-attention models** (Qwen3-Next\u002F3.5\u002F3.8, GLM-5.x Flash, Kimi Linear, OLMo Hybrid) fall back to slow, memory-hungry PyTorch kernels without `pip install \"fla-core>=0.5.2\"`; install it before calibrating on GPU (it breaks CPU runs of these models)\n- **Transformers version**: New models may need a newer version of transformers than what's installed. Check `config.json` for `transformers_version`. In containers, beware of `PIP_CONSTRAINT` blocking upgrades — see `references\u002Fslurm-setup-ptq.md` for workarounds\n- **Gated datasets**: Some calibration datasets require HF authentication. Set `HF_TOKEN` in the job environment. Use `--dataset cnn_dailymail` only as the constrained-environment fallback described in Step 4, not as the preferred calibration set\n- **NFS root_squash + Docker**: See the common skill's `slurm-setup.md` section 5\n\n## References\n\n| Reference | When to read |\n| --- | --- |\n| common skill: `environment-setup.md` | Step 1: always |\n| common skill: `workspace-management.md` | Step 1: always |\n| `references\u002Flauncher-guide.md` | Step 4B only (launcher path) |\n| `tools\u002Flauncher\u002FCLAUDE.md` | Step 4B only, if you need more launcher detail |\n| `references\u002Funsupported-models.md` | Step 4C only (unlisted model) |\n| `references\u002Fcheckpoint-validation.md` | Step 5: mandatory post-PTQ gate before deployment\u002Fevaluation |\n| common skill: `remote-execution.md` | Step 4A\u002F4C only, if target is remote |\n| common skill: `slurm-setup.md` | Step 4A\u002F4C only, if using SLURM manually (not launcher) |\n| `references\u002Fslurm-setup-ptq.md` | Step 4A\u002F4C only, PTQ-specific SLURM (container, GPU sizing, FSDP2) |\n| `examples\u002Fhf_ptq\u002FREADME.md` | Step 3: support matrix, CLI flags, accuracy |\n| `modelopt\u002Ftorch\u002Fquantization\u002Fconfig.py` | Step 3: format definitions |\n| `modelopt\u002Ftorch\u002Fexport\u002Fmodel_utils.py` | Step 4C: TRT-LLM export type mapping |\n| `modelopt_recipes\u002F` | Step 3: pre-built recipes |\n","# ModelOpt Post-Training Quantization\n\nProduce a quantized checkpoint from a pretrained model. **Read `examples\u002Fhf_ptq\u002FREADME.md` first** — it has the support matrix, CLI flags, and accuracy guidance.\n\nUse `modelopt-quant-recipe-search` for multi-candidate recipe exploration or\noptimization. Use this skill for each selected recipe's PTQ run.\n\n## Step 1 — Environment\n\nUse the common skill's `environment-setup.md` and `workspace-management.md`.\nAfter completing them you should know:\n\n- ModelOpt source is available\n- Local or remote (+ cluster config if remote)\n- SLURM \u002F Docker+GPU \u002F bare GPU\n- Launcher available?\n- Which workspace to use\n\n## Step 2 — Is the model supported?\n\nCheck the support table in `examples\u002Fhf_ptq\u002FREADME.md` for verified HF models.\n\n- **Listed** → supported, use `hf_ptq.py` (step 4A\u002F4B)\n- **Not listed** → read `references\u002Funsupported-models.md` to determine if `hf_ptq.py` can still work or if a custom script is needed (step 4C)\n\n## Step 2.5 — Check for model-specific dependencies\n\nIf the model uses `trust_remote_code` (check `config.json` for `auto_map`), inspect its custom Python files for imports not present in the container:\n\n```bash\ngrep -h \"^from \\|^import \" \u003Cmodel_path>\u002Fmodeling_*.py | sort -u\n```\n\n**Known dependency patterns:**\n\n| Import found | Packages to install |\n| --- | --- |\n| `from mamba_ssm` \u002F `from causal_conv1d` | `mamba-ssm causal-conv1d` (Mamba\u002Fhybrid models: NemotronH, Jamba) |\n\nIf extra deps are needed:\n- **Launcher (4B)**: set `EXTRA_PIP_DEPS` in the task's `environment` section — `ptq.sh` installs them automatically\n- **Manual (4A)**: `unset PIP_CONSTRAINT && pip install \u003Cdeps>` before running `hf_ptq.py`\n\n## Step 3 — Choose quantization format\n\n**First**, check for a model-specific recipe:\n\n```bash\nls modelopt_recipes\u002Fmodels\u002F 2>\u002Fdev\u002Fnull\nls modelopt_recipes\u002Fmodel_type\u002F\u003Cmodel_type>\u002Fptq\u002F 2>\u002Fdev\u002Fnull  # per-arch; \u003Cmodel_type> from local config.json (Hub ID: AutoConfig.from_pretrained)\n```\n\nIf a model-specific recipe exists, prefer `--recipe \u003Cpath>` — but **inspect its include\u002Fexclude patterns** rather than assuming (e.g. for VLMs, confirm the vision tower is actually excluded).\n\n**If no model-specific recipe**, choose a format based on GPU (details in `examples\u002Fhf_ptq\u002FREADME.md`):\n\n- **Blackwell** (B100\u002FB200\u002FGB200): `nvfp4` variants\n- **Hopper** (H100\u002FH200) or older: `fp8` or `int4_awq`\n\nUse `--qformat \u003Cname>` (e.g., `--qformat nvfp4`). Format definitions: `modelopt\u002Ftorch\u002Fquantization\u002Fconfig.py`. General PTQ recipes in `modelopt_recipes\u002Fgeneral\u002Fptq\u002F` correspond to the same formats — `--qformat` is the simpler way to use them.\n\nBefore running PTQ, sanity-check the selected qformat\u002Frecipe against the model structure. Inspect the recipe's include\u002Fexclude patterns and summarize which layer groups will be quantized and approximately how many modules\u002Flayers match (attention projections, MLP projections, experts, etc.). If the match count is 0, or far smaller than expected for the model, stop and fix the recipe or ask the user before launching calibration.\n\n**VLMs:** generic `*mlp*`\u002F`*experts*` recipes also match the vision tower (`model.visual.*`); quantizing the ViT silently breaks image benchmarks. Use the `model_type\u002F\u003Cmodel_type>\u002Fptq\u002F` recipe or add `*visual*`\u002F`*vision_tower*` excludes, then verify in Step 5 — see `references\u002Fcheckpoint-validation.md`.\n\nIf the source checkpoint is already quantized and the requested recipe\u002Fconfig reduces quantization coverage, confirm that intent with the user before running. For example, if an FP8 checkpoint is used as input and the recipe excludes some layers so they would fall back to BF16 instead of staying quantized, call out the affected layer groups and ask whether that FP8-to-BF16 fallback is intended.\n\n> NVFP4 can be calibrated on Hopper but requires Blackwell for inference.\n\n## Step 4 — Run PTQ\n\n**Goal: checkpoint on disk** (`.safetensors` + `config.json`).\n\nFor **listed models** (4A\u002F4B): run full calibration directly (`--calib_size 512`).\nFor **unlisted models** (4C): run a smoke test first (`--calib_size 4`), wait for success, then full calibration.\n\n### Recommended calibration datasets\n\n- **Text-only LLM PTQ:** Prefer the representative `nemotron-post-training-v3` blend. `modelopt\u002Ftorch\u002Futils\u002Fdataset_utils.py` expands it to seven registered Nemotron SFT domains. Configure Hugging Face credentials where required.\n\n  ```bash\n  python examples\u002Fhf_ptq\u002Fhf_ptq.py ... \\\n      --dataset nemotron-post-training-v3\n  ```\n\n- **VLM PTQ:** Include image-text calibration with `--calib_with_images`. This path uses `nemotron_vlm_dataset_v2` with the current default subsets `sparsetables`, `plotqa_cot`, and `wiki_en`; `examples\u002Fhf_ptq\u002Fhf_ptq.py` and `modelopt\u002Ftorch\u002Futils\u002Fvlm_dataset_utils.py` are the source of truth.\n\n  ```bash\n  python examples\u002Fhf_ptq\u002Fhf_ptq.py ... \\\n      --calib_with_images\n  ```\n\n`--dataset` selects text-only calibration and cannot substitute for multimodal examples. Use `--dataset cnn_dailymail` only as a fallback when representative data is unavailable, such as without gated-data access or with only a local public cache.\n\n### Which path?\n\n```text\nIn README table? ─→ YES ──→ SLURM (local or remote)? ──→ LAUNCHER (4B)\n                  │          Local Docker + GPU? ────────→ LAUNCHER (4B)\n                  │          Remote Docker (no SLURM)? ──→ MANUAL (4A)\n                  │          Bare GPU (local or remote)? → MANUAL (4A)\n                  │\n                  └→ NOT LISTED ──→ UNLISTED MODEL (4C)\n```\n\n### 4A — Direct: supported model, manual execution\n\n```bash\npip install --no-build-isolation \"nvidia-modelopt[hf]\"\npip install -r examples\u002Fhf_ptq\u002Frequirements.txt\n\npython examples\u002Fhf_ptq\u002Fhf_ptq.py \\\n    --pyt_ckpt_path \u003Cmodel> \\\n    --qformat \u003Cformat> \\\n    --calib_size 512 \\\n    --export_path \u003Coutput>\n```\n\nRun `--help` for all options.\n\nFor remote, use `remote_run` from the common skill's `remote_exec.sh`; see its\n`remote-execution.md`.\n\n### 4B — Launcher: supported model on SLURM or local Docker\n\nWrite a YAML config using `common\u002Fhf\u002Fptq.sh`. See `references\u002Flauncher-guide.md` for the full template.\n\n```bash\ncd tools\u002Flauncher\n# SLURM (remote or local):\nSLURM_HOST=\u003Chost> SLURM_ACCOUNT=\u003Cacct> uv run launch.py --yaml \u003Cconfig.yaml> user=\u003Cssh_user> identity=\u003Cssh_key> --yes\n# Local Docker:\nuv run launch.py --yaml \u003Cconfig.yaml> hf_local=\u003Chf_cache> --yes\n```\n\nThe launcher blocks and tails logs until the job completes. If the launcher fails (missing deps, config errors), fall back to path 4A (manual execution).\n\n### 4C — Unlisted model\n\nFollow `references\u002Funsupported-models.md`. It walks through investigating the model, patching ModelOpt if needed, and running `hf_ptq.py`. Run manually (like 4A) for easier monitoring and debugging.\n\nFor SLURM, see the common skill's `slurm-setup.md` and this skill's\n`references\u002Fslurm-setup-ptq.md`.\n\n### Monitoring\n\nAfter job submission, register the job and set up monitoring per the **modelopt-monitor skill**.\n\n## Step 5 — Verify output\n\n```bash\nls -lh \u003Coutput_path>\u002F\n# Expect: config.json, tokenizer files, model-*.safetensors\n```\n\nReport the path and size to the user.\n\n### Post-quantization validation\n\nThis is a required gate before any deployment or evaluation submission. Do not submit an eval, start a production serving job, or hand off the checkpoint as ready until the gate, including its serving canary, has passed.\n\nRead `references\u002Fcheckpoint-validation.md` and perform all four validation groups on the exact checkpoint path that will be deployed\u002Fevaluated:\n\n1. Check output size and estimated bits per weight against the baseline\u002Fsource checkpoint.\n2. Check quantized-weight coverage against the requested qformat\u002Frecipe\u002Fconfig.\n3. Check metadata consistency against the baseline\u002Fsource model.\n4. Complete the required downstream handoff and serving-readiness validation.\n\nReport the gate result before moving on. Follow the canonical report format and all blocking conditions in `references\u002Fcheckpoint-validation.md`; do not hand off a checkpoint unless every required check passes.\n\n## Key API Rules\n\n- `mtq.register()` classes **must** define `_setup()` and call it from `__init__`\n- Call `mto.enable_huggingface_checkpointing()` **before** quantization\n- Wildcard `*gate*` matches too broadly — use `*mlp.gate*` or `*router*`\n- VLMs: `hf_ptq.py` auto-extracts the language model via `extract_and_prepare_language_model_from_vl()` — no manual VLM handling needed in most cases\n- FP8 checkpoints: prefer `_QuantFP8Linear` (lazy dequant) over `FineGrainedFP8Config(dequantize=True)` which wastes ~2x memory. See `references\u002Funsupported-models.md` for details\n- Custom quantizer names must end with `_input_quantizer` or `_weight_quantizer`\n\n## Common Pitfalls\n\n- **Model-specific dependencies**: Models with `trust_remote_code` may import packages not in the container (e.g., `mamba-ssm` for hybrid Mamba models). See Step 2.5. Use `EXTRA_PIP_DEPS` env var with the launcher, or install manually before running `hf_ptq.py`\n- **Linear-attention models** (Qwen3-Next\u002F3.5\u002F3.8, GLM-5.x Flash, Kimi Linear, OLMo Hybrid) fall back to slow, memory-hungry PyTorch kernels without `pip install \"fla-core>=0.5.2\"`; install it before calibrating on GPU (it breaks CPU runs of these models)\n- **Transformers version**: New models may need a newer version of transformers than what's installed. Check `config.json` for `transformers_version`. In containers, beware of `PIP_CONSTRAINT` blocking upgrades — see `references\u002Fslurm-setup-ptq.md` for workarounds\n- **Gated datasets**: Some calibration datasets require HF authentication. Set `HF_TOKEN` in the job environment. Use `--dataset cnn_dailymail` only as the constrained-environment fallback described in Step 4, not as the preferred calibration set\n- **NFS root_squash + Docker**: See the common skill's `slurm-setup.md` section 5\n\n## References\n\n| Reference | When to read |\n| --- | --- |\n| common skill: `environment-setup.md` | Step 1: always |\n| common skill: `workspace-management.md` | Step 1: always |\n| `references\u002Flauncher-guide.md` | Step 4B only (launcher path) |\n| `tools\u002Flauncher\u002FCLAUDE.md` | Step 4B only, if you need more launcher detail |\n| `references\u002Funsupported-models.md` | Step 4C only (unlisted model) |\n| `references\u002Fcheckpoint-validation.md` | Step 5: mandatory post-PTQ gate before deployment\u002Fevaluation |\n| common skill: `remote-execution.md` | Step 4A\u002F4C only, if target is remote |\n| common skill: `slurm-setup.md` | Step 4A\u002F4C only, if using SLURM manually (not launcher) |\n| `references\u002Fslurm-setup-ptq.md` | Step 4A\u002F4C only, PTQ-specific SLURM (container, GPU sizing, FSDP2) |\n| `examples\u002Fhf_ptq\u002FREADME.md` | Step 3: support matrix, CLI flags, accuracy |\n| `modelopt\u002Ftorch\u002Fquantization\u002Fconfig.py` | Step 3: format definitions |\n| `modelopt\u002Ftorch\u002Fexport\u002Fmodel_utils.py` | Step 4C: TRT-LLM export type mapping |\n| `modelopt_recipes\u002F` | Step 3: pre-built recipes |\n",{"name":5,"description":7},"main","e169b4c2bc62a32c45aaef793cde7599dee2c1b2","skills\u002Fmodelopt-ptq",806,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer","https:\u002F\u002Fnvidia.github.io\u002FModel-Optimizer\u002F","https:\u002F\u002Fgithub.com\u002FNVIDIA","https:\u002F\u002Fnvidia.com","2026-11-15 02:45:59",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add NVIDIA\u002FModel-Optimizer --skill modelopt-ptq",{"tool":39,"command":40},"bunx","bunx skills add NVIDIA\u002FModel-Optimizer --skill modelopt-ptq",{"tool":42,"command":43},"pnpm","pnpm dlx skills add NVIDIA\u002FModel-Optimizer --skill modelopt-ptq","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002FSKILL.md",true,[47,53,57,61,65],{"path":48,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":51,"sourceHtmlUrl":52},"references\u002Fcheckpoint-validation.md","REFERENCE","参考文档",13970,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002Freferences\u002Fcheckpoint-validation.md",{"path":54,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":55,"sourceHtmlUrl":56},"references\u002Flauncher-guide.md",4615,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002Freferences\u002Flauncher-guide.md",{"path":58,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":59,"sourceHtmlUrl":60},"references\u002Fslurm-setup-ptq.md",3757,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002Freferences\u002Fslurm-setup-ptq.md",{"path":62,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":63,"sourceHtmlUrl":64},"references\u002Funsupported-models.md",18908,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002Freferences\u002Funsupported-models.md",{"path":66,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":67,"sourceHtmlUrl":68},"tests.json",4428,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-ptq\u002Ftests.json",[70,77,84,91,98,105],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M7B5XD9M965H2TAV4ZJMPF6N","modelopt-speculative-decoding","Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or wants to check whether a run's acceptance rate meets threshold. Triggers on \"EAGLE3\", \"DFlash\", \"DSpark\", \"Domino\", \"draft model\", \"acceptance rate\", \"speculative decoding pipeline\". Do NOT use for quantizing a model (use modelopt-ptq) or serving a checkpoint (use modelopt-deployment).","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-speculative-decoding\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M7B5X9SG5FHW174VFK979DQ9","modelopt-run-agentx","Run the AgentX agentic serving benchmark with the SemiAnalysis harness. Use for AgentX setup, dataset downloads, concurrency sweeps, or serving latency and throughput comparisons.","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-run-agentx\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M7B5X4C0XGVA8N0BXF9N4HCX","modelopt-release-cherry-pick","Audit merged bug-fix PRs and release NVBugs for missing cherry-pick labels, then cherry-pick labeled PRs into a release branch and open a PR. Use when asked to \"cherry-pick PRs for release\u002FX.Y.Z\", \"pick PRs to release branch\", \"verify cherry-pick labels\", or \"cherry-pick labeled PRs\".","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-release-cherry-pick\u002FSKILL.md",[],[],{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":19},"01M7B5WXMDYXCK43YA5ZEF7ECY","modelopt-quant-recipe-search","Use when the user asks to find, search for, or optimize the best quantization recipe for a model, including direct requests like \"find the best quantization recipe and generate a PTQ checkpoint.\" Guides the multi-candidate loop: choose compute-vs-memory success metrics, select ModelOpt recipe baselines, design AutoQuant\u002Fmanual recipe deltas, interpret sensitivity, and decide next candidates. Do NOT use for a single known PTQ recipe run (use modelopt-ptq), serving (use modelopt-deployment), creating\u002Frunning evals (use modelopt-evaluation or modelopt-launching-evals), monitoring jobs (use modelopt-monitor), MLflow browsing (use modelopt-accessing-mlflow), or comparing completed baseline-vs-candidate scores only (use modelopt-compare-results).","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-quant-recipe-search\u002FSKILL.md",[],[],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M7B5WRMPZE23YXCFMZBM8KJA","modelopt-qad","Run explicitly requested ModelOpt Quantization-Aware Distillation (QAD) through Megatron Bridge (recommended) or Hugging Face Transformers to recover a measured BF16-to-PTQ accuracy gap. Use only when the user explicitly asks for QAD, including its topology, data preparation, launch, resume, checkpoint export, or recovery decisions.","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-qad\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M7B5WDB7KYNXD84PQRA69RPT","modelopt-monitor","Monitor submitted jobs (PTQ, evaluation, deployment) on SLURM clusters. Use when the user asks \"check job status\", \"is my job done\", \"monitor my evaluation\", \"what's the status of the PTQ\", \"check on job\" followed by a SLURM job ID, or after any skill submits a long-running job. Also triggers on \"nel status\", \"squeue\", or any request to check progress of a previously submitted job.","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FModel-Optimizer\u002Fblob\u002Fmain\u002Fskills\u002Fmodelopt-monitor\u002FSKILL.md",[],[]]