quantized-export
Export a promoted fine-tuned model in the right deployment format — merged safetensors, LoRA-only, GGUF with imatrix, or FP8. Use after a checkpoint passes promotion, when choosing a quantization format for a target device, or when an exported model fails its smoke test.
- License 许可
- MIT
- 最近更新
- 2026-08-19
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- description
- Export a promoted fine-tuned model in the right deployment format — merged safetensors, LoRA-only, GGUF with imatrix, or FP8. Use after a checkpoint passes promotion, when choosing a quantization format for a target device, or when an exported model fails its smoke test.
02 资源文件 1
- 参考文档references/export-commands.md12.4 KB
03 SKILL.md 正文
Quantized Export
The last stop after checkpoint-promotion
hands off a PROMOTE verdict: a checkpoint
that cleared the four-stage gate still isn't
deployed until it's exported in the right
format for its target runtime and proven to
still work post-export. A REJECT verdict
never reaches this skill — export starts only
from a promoted checkpoint.
Input: a promoted checkpoint (or LoRA adapter) plus the target deployment surface — GPU class, serving stack, and whether long-context/code/math workloads are in scope. Output format: an exported artifact in the chosen format plus a smoke-test diff report comparing 3–5 golden outputs pre-export and post-export.
Format Map
Pick format by hardware and deployment shape, not by habit — the wrong pick either wastes throughput headroom or breaks silently on specific workloads (see Workload Overrides).
- FP8 is the default on Hopper-class GPUs and newer. It preserves near-bf16 quality at roughly half the memory, and it's the safe first choice whenever the target GPU supports it and no edge-device constraint applies.
- AWQ INT4 targets older GPUs that predate FP8 hardware support. GPTQ is superseded for new deployments — don't reach for it on a fresh export; AWQ has better accuracy retention at the same bit width and wider current tooling support.
- GGUF with Q4_K_M quantization, built from an imatrix, is the edge/llama.cpp format. Use it for local or CPU-adjacent deployment, not for GPU-serving throughput — it optimizes for footprint, not tokens/sec on a datacenter GPU.
- NVFP4 is for Blackwell-at-scale
deployments only — and explicitly NOT on
GB10. NVFP4 on SM121 (GB10) runs ~32%
slower than FP8 because the hardware
lacks a native
cvt.e2m1x2path unless the kernel is compiledsm_121a. Choosing NVFP4 on a GB10 target is a regression, not an upgrade — pick FP8 there instead. - Merged vs. LoRA-only is a separate axis from quant format. A merged export folds the adapter into the base weights: larger artifact, no base-model dependency at serve time. LoRA-only keeps the adapter separate: much smaller artifact, but the serving stack must load the exact same base model alongside it — a mismatched or wrong-revision base silently changes outputs. Pick merged when artifact portability matters more than storage; pick LoRA-only when disk footprint or multi-adapter serving matters more.
Worked Picks
The core format-selection tradeoff, read as a lookup table for common scenarios:
| Target | Workload | Format |
|---|---|---|
| Datacenter GPU | generic chat | FP8 |
| Datacenter GPU | long-context/code/math | FP8 or W8A8 — never INT4 |
| Older GPU generation | generic | AWQ INT4 |
| Edge device / laptop | llama.cpp serving | GGUF Q4_K_M + imatrix |
| GB10 | any workload | FP8 via vLLM nightly, or GGUF via llama.cpp locally — skip NVFP4 |
# quick decision snippet — see the table above for the full map
hopper_or_newer: fp8
older_gpu: awq-int4
edge_llama_cpp: gguf-q4_k_m+imatrix
gb10_any_workload: fp8-vllm-nightly # never nvfp4 on GB10
Workload Overrides
The Format Map above is a default, not a rule that survives every workload. Long-context, code, and math workloads break at INT4 — quantization error compounds across long sequences and precise token-level reasoning in ways that don't show up on short, generic prompts. For any of these three workload classes, stay on FP8 or W8A8 even if the target hardware would otherwise justify INT4 on cost grounds.
- Don't validate this override with MMLU or
similar broad-knowledge benchmarks — they
don't stress the failure mode. Measure
with the actual task evals — the goldens
and graders from
eval-harness-first, run through the exported artifact — because INT4 degradation on long-context, code, or math shows up as task-specific failures (dropped context, broken syntax, arithmetic errors) well before it moves a knowledge benchmark. - If a task eval regresses after an INT4 export on one of these three workload classes, the fix is switching format, not re-tuning the quantization recipe — AWQ and GPTQ variants at the same bit width share the same compounding-error failure mode on these workloads.
The Smoke Test
Export bugs are silent at the file level — a malformed export still produces a loadable artifact, so file-existence checks prove nothing. The smoke test is mandatory for every export, with no exception for a format that "should just work":
- Load the exported artifact in its actual target runtime — vLLM for FP8/AWQ, llama.cpp for GGUF, not a quick sanity load in a different framework than the one that will serve it in production.
- Run 3–5 golden prompts through it —
pull these from the same
eval/goldens.jsonleval-harness-firstmaintains, not a fresh ad hoc set. - Compare each output against the
pre-export generation for the same
prompt, same deterministic sampling
settings — greedy decoding (temperature 0)
and a fixed seed, persisted and reused
between the pre- and post-export runs, not
just nominally identical config. For a
lossless export, byte match is the gate —
any diff is a bug. For a lossy
(quantized) export, byte match is expected
to fail; the gate is task-grader verdict
agreement instead — see
references/export-commands.md's Smoke-Test Script Skeleton.
Run this as a gate, not a manual check:
python smoke_test.py "$EXPORT_PATH" \
eval/goldens.jsonl pre-export-outputs.jsonl
# non-zero exit on any pre/post mismatch
Failure Signatures
What export bugs actually look like, not a clean pass/fail flag:
- Template mismatch presents as garbled or run-on output — the chat template baked into the export doesn't match the one the checkpoint was trained and evaluated against, so turn boundaries or special tokens land in the wrong place.
- Wrong quantization applied to
lm_headpresents as off-template or semantically nonsensical output that still looks fluent — the output head lost precision it needed even though the rest of the network quantized cleanly.
Never ship an export that skipped this step —
a checkpoint's PROMOTE verdict says the
un-exported checkpoint is good; it says
nothing about the export pipeline. Re-run on
any quant-method or runtime version bump, not
only after the first export. Runnable command
sequences for every format plus the
smoke-test script skeleton:
references/export-commands.md.
Related Skills
checkpoint-promotion— the only valid upstream source for this skill. A checkpoint without aPROMOTEverdict doesn't reach export.eval-harness-first— owns theeval/goldens.jsonlthis skill's smoke test draws its 3–5 prompts from, and the task evals the Workload Overrides section requires for long-context/code/math validation.finetuning-method-selection— itsreferences/model-catalog.mdis the place to check hardware-class assumptions (which GPU generations a base model targets) before picking a format off the Format Map above.
Spark users: on GB10, GGUF via llama.cpp
works well for local serving, and FP8 serving
via vLLM nightly builds is the other proven
path — NVFP4 is the one format to avoid there
(see the Format Map exception above). Once the
dgx-spark-ops plugin is installed, defer
Spark-specific serving and thermal questions to
its skills rather than re-deriving them here.
04 同仓库更多 Skills
youtube-transcript
Extract transcripts from YouTube videos. Use when the user asks for a transcript, subtitles, or captions of a YouTube video and provides a YouTube URL (youtube.com/watch?v=, youtu.be/, or similar). Supports output with or without timestamps.
youtube-thumbnail-design
YouTube thumbnail design with specific dimensions, contrast rules, and mobile preview optimization. Covers safe zones, text placement, face expression psychology, and A/B testing. Use for: YouTube thumbnails, video cover images, click-through optimization. Triggers: youtube thumbnail, thumbnail design, video thumbnail, click through rate, ctr optimization, youtube cover, video cover image, thumbnail maker, thumbnail tips, youtube design, video preview image
youtube-short-maker
Make YouTube Shorts that hold attention with Pexo — vertical 9:16, fast-paced, hook in the first 3 seconds. Describe your topic and Pexo writes the script, generates the shots, picks the models, and assembles a finished Short with music and captions. Use for YouTube Shorts: "youtube short", "make a youtube short", "shorts video", "short-form video". NOT for long-form or landscape video.
xcode-project-setup
Safely modifies Xcode projects (.pbxproj) to add Swift Packages and link files. Use this skill whenever an iOS project needs dependencies installed (e.g. Firebase, Alamofire).
x402
Set up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key. Two setups it works out up front — "just use it" (set up a wallet so you or Claude Code can run cloud browser tasks paid from the wallet — Claude writes and runs throwaway scripts, nothing touches your codebase) or "build it in" (install the SDK and write the key + code into your project). Walks through wallet setup, funding, .env, and a ~$1 test run. Use when the user asks about x402, pay-per-use, USDC payments, or wants Browser Use Cloud without an API key. For the free-tier signup (reverse-CAPTCHA → API key), use `browser-use cloud signup` or the `cloud` skill instead.
x-twitter-scraper
Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation.