[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6BY55P8GVX66H0FG8JHXHWF":3,"skill-same:01M6BY55P8GVX66H0FG8JHXHWF":65},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":46,"repositoryHtmlUrl":47,"repositoryHomepageUrl":48,"ownerHtmlUrl":49,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":50,"install":51,"resources":64},"01M6BY55P8GVX66H0FG8JHXHWF","gpu-clean-conversion",null,"Convert a PyTorch or Hugging Face model into a LiteRT model that runs fully on the GPU via the CompiledModel API with verified-correct output, and lay it out as a model recipe. Use when converting a new model, or when a converted model is rejected by the GPU, falls back to CPU, or returns wrong numbers on device.","google-ai-edge\u002Flitert-samples","google-ai-edge","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F150697620?v=4","01M6BY4NHMZF88PJAC1E1QJY8N","01M6BY4NNXP8PC95Y6CABCJXP9",494,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Fgpu-clean-conversion\u002FSKILL.md",[],[18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"gpu","gpu clean conversion","clean","conversion","hugging face","verified","CPU","via","with","output","use","recipe","as","back","api","convert","pytorch","correct","model","litert","2026-11-10 10:12:30",false,"---\nname: gpu-clean-conversion\ndescription: Convert a PyTorch or Hugging Face model into a LiteRT model that runs fully on the GPU via the CompiledModel API with verified-correct output, and lay it out as a model recipe. Use when converting a new model, or when a converted model is rejected by the GPU, falls back to CPU, or returns wrong numbers on device.\n---\n\n# GPU-clean conversion\n\nA conversion is done when three things hold, in this order:\n\n1. it converts,\n2. every node runs on the GPU,\n3. **the on-device output matches the source model.**\n\nStep 3 is not implied by step 2. The delegate can report full residency and\nstill return wrong numbers — that is the failure mode most of the rewrites\nbelow exist for. Never call a model done without a numerical check against\nCPU or PyTorch on the actual device.\n\n## Loop\n\n**1. Convert plain first.** No patches. This tells you what the model\nactually needs rather than what you assumed.\n\n**2. Verify through the CompiledModel API before touching a device.**\n\n```python\nfrom litert_gpu_toolkit import check_gpu_compatibility, print_report\nprint_report(check_gpu_compatibility(\"model.tflite\"))\n```\n\nIt compiles the model for the GPU accelerator, runs every signature on\nrandom inputs, and compares the outputs against a CPU-compiled reference.\nA failed GPU compile names the offending op — route it through the table\nbelow. A pass with a CPU-fallback warning means some ops fell back to the\nCPU; treat them the same way if you need full residency. This exercises the\nhost GPU, so step 5 on the actual device is still required.\n\nWhen no GPU runtime is at hand (a CI job, a laptop without the accelerator),\na static pre-flight gives the first read: [edge-lint](https:\u002F\u002Fgithub.com\u002Fjohn-rocky\u002Fedge-compat)\nparses the `.tflite` and reports, per op, what a measured compatibility matrix\nsays for the target backend and runtime version — delegated, fallback,\nincorrect, or `unknown` (not yet measured, not \"unsupported\") — with the\nmatrix's rewrite hint next to each finding. It needs no runtime or device and\ndoes not replace the compile-and-compare above; use it to route symptoms\nthrough the table below before a device is available.\n\n**3. Map each symptom to a rewrite.** The rewrites live in\n`utilities\u002Flitert_gpu_toolkit` — plain Python, no build step. Outside this\nrepo, clone litert-samples and put `utilities\u002F` on `PYTHONPATH`, or vendor\nthe directory:\n\n| What you see | Rewrite |\n|---|---|\n| `GATHER_ND` named in the compile error | Find its source. Stride-2 slicing (`x[:, ::2]`, Focus stems, patch merging), `grid_sample`, `MaxPool` padding, bicubic `interpolate`, and reflect-mode `F.pad` all lower to it. `patch_grid_sample`, `patch_maxpool_zeropad`, `patch_interpolate`, `patch_patch_merging` |\n| A rank-5+ tensor named in the compile error | `PixelShuffle` (rank-6 reshape), windowed attention, `einops.rearrange`, packed-QKV attention head splits. `pixelshuffle_to_conv_transpose`, `patch_window_attention`, `patch_einops` |\n| `TRANSPOSE_CONV` rejected | Version skew, not a missing op. `ZeroStuffConvT1d` \u002F `ZeroStuffConvT2d` — zero-stuff plus a plain conv, exact to ~1e-7 |\n| `SELECT` \u002F `SELECT_V2` | `PReLU`, `ELU`, in-place index assignment, and `torch.where` masking. Replace with arithmetic: `x*(1-m) + v*m`. Keep v finite in fp16: −1e9 is −inf there, and (1 − m) × −1e9 is 0 × −inf = NaN on every valid position; −30000 is exact in fp16 and gives the same softmax in fp32 when the logits are bounded |\n| `BROADCAST_TO` | Two cases. On a compile-time constant: an outer product or `.expand` that did not fold — bake the result as a constant at its target shape. On a **runtime** tensor the GPU delegate rejects it outright, even at rank 4 — the canonical case is GQA's `repeat_kv` (`x[:,:,None].expand(...)`, which is also rank-5, so two walls in one line). Exact rewrite: `torch.cat([x[:, i:i+1].expand(b, n_rep, s, d) for i in range(n_kv)], dim=1)` — same head order, bit-exact. Tracked upstream: google-ai-edge\u002FLiteRT#9191 |\n| Masked attention wrong only on device: token 0 bit-exact, every later token wrong | Broadcast `ADD` whose LHS is a `BATCH_MATMUL` result (the `scores + mask[1,1,S,S]` idiom) silently miscomputed on ai-edge-litert 2.1.3 and 2.1.5 and is fixed in 2.1.6 (head-axis size-1 broadcast only; google-ai-edge\u002FLiteRT#8593, closed as fixed; the 2.1.6 release notes list GPU broadcast fixes). The signature mimics broken RoPE — tap the rope output before blaming it. Rewrites on the older runtimes: pre-expand the mask to `[1,H,S,S]`, or materialize the BMM as a second output |\n| An `ADD` result that is both a graph output **and** consumed downstream comes back wrong | Output aliasing: the returned tensor holds an *operand*, not the sum — `ADD` with two runtime operands (`SUB`\u002F`MUL` exact, `x + 1.0` exact). This is the shape of every explicit state update in a streaming\u002Frecurrent graph. Workaround: emit `acc * one` where `one` is a **runtime** input holding 1.0 — a constant folds straight back into the pattern. Tracked upstream: google-ai-edge\u002FLiteRT#8599 |\n| `RELU_0_TO_1` rejected by the GPU delegate | Emitted by hard-sigmoid \u002F `nn.Hardtanh(0,1)`. Accepted in litert 2.1.3, rejected from 2.1.5 on — a model at full residency on an older runtime hard-fails `CompiledModel` creation after an upgrade. Rewrite: `relu(x) - relu(x-1)`, exact. Tracked upstream: google-ai-edge\u002FLiteRT#8598 |\n| `DIV: No support of few identical inputs` \u002F `Expected 1 const input tensor(s)`, device only | The delegate declines an op whose two inputs are the same tensor, and ops whose inputs are all constants — together these split a perceiver-style block (softmax over a length-1 axis of a constant latent bank) into several partitions. Fixes: special-case the degenerate axis (a softmax over a length-1 axis is identically 1), or make one input non-constant. Note the sibling LayerNorm-over-a-constant pattern no longer reaches the delegate at all — the converter folds it to a single `MUL`. Tracked upstream: google-ai-edge\u002FLiteRT#9192 |\n| `NHWC node rewriter not found: amax` | `x.amax(...)`\u002F`x.max(dim)` in stable-softmax, adaptive norms, qk-norm. Rewrite channel reduce-max as `max_pool2d(x.reshape(N,1,C,H*W), kernel=(C,1))` — numerically identical — or drop the norm to 3D |\n| `Lowering not found: aten._fft_r2c` \u002F `aten.complex` | `torch.stft`\u002F`istft` and complex views have no lowering (fails before any GPU check). A DFT is a fixed linear map: windowed-DFT as `Conv1d` with the cos\u002Fsin basis baked into kernels (stride = hop), iSTFT as inverse-DFT matmul + overlap-add via zero-stuffed conv-transpose — exact. Model-selection corollary: prefer time-domain vocoder branches over iSTFT-based ones. ⚠ Library STFT-as-conv stacks (torchlibrosa-style) have **numerically mis-converted** (corr 0.83) while the op check looks clean — verify the spectrogram numerically or compute log-mel host-side |\n| Compiles, runs, output is wrong or NaN | The fp16 reduction family — and the trigger is the **fp16 accumulator passing 65504**, so a plain single-axis mean\u002Fsum over enough elements overflows just like variance does. `patch_safe_layernorm`, `patch_rmsnorm`, `patch_instance_norm`, `hierarchical_mean`. Caveats: at extreme magnitudes (\\|x\\| in the thousands) even the adaptive safe-LN form overflows when it reconstructs the large variance — the robust form stays entirely in the down-scaled domain (`xs = x\u002FS`, normalize `xs`, never multiply the variance back by `S²`); `hierarchical_mean` is exact only for power-of-two spatial dims (for arbitrary dims, cascade `\u002F2` avg-pools with `ceil_mode` so each stage averages ≤~49 elements). A static form keeps fp32 bit-exact and puts no reduce-max in the graph: scale each norm's input by a power of two 2^-k chosen per site from a calibration over the whole check set (headroom for the row's sum of squares *and* for the largest single x² — both overflowed in the case on record, the mean of squares did not), and eps by 4^-k. When eps·4^-k drops out of fp16's range (eps 1e-6 at k = 3), an all-zero row (padding) becomes rsqrt(0) = inf and 0 × inf = NaN: clamp the rsqrt at 2^k\u002F√eps — the zero row's own value, never reached in fp32. Diagnostic split: **all-zero\u002Fall-blank output = an overflow in one block; a result that starts near-correct and degrades with depth = precision compounding**, which no overflow patch (and no fp32-precision flag) fixes |\n| Head outputs exactly zero | RMSNorm `Σx²` overflowed fp16 to `inf`. `patch_rmsnorm` |\n| GPU compile fails only at fp16 precision with a kernel-source error (`write_imageh` on Adreno and Mali OpenCL, `write` on Metal, at the same source line), while fp32 precision compiles | A boolean value on the way to a graph output. In the case on record an output mask computed as `(mask > 0.5)` and cast back to float made all three GPUs fail; the same output computed from the float input mask by float ops (a strided slice) compiled at fp16 everywhere and stayed bit-exact in fp32. Small graphs with the same GREATER → CAST pair compiled, so bisect on the real graph. Keep masks float (1 = valid) from input to output |\n\n**4. Re-convert and re-verify.** Repeat 2–3 until the check passes.\n\n**5. Verify on device, numerically.** Run the same input through the source\nmodel and the on-device graph and compare. Correlation on the output tensor,\nplus a task-level check where one exists (argmax match, IoU, mask foreground\ncount). Record the device and the residency line (`N\u002FN` nodes) alongside the\nnumber — a result without both is not reproducible.\n\nIf residency is full and the output is wrong, bisect by intermediate: dump\ntensors at block boundaries and find the first one that diverges. The cause is\nusually a reduction, not the op you suspect.\n\n## Watch for\n\n- **fp16 is used even for an fp32 graph.** The delegate reduces in fp16\n  regardless of tensor dtype by default (the GPU options carry a precision\n  setting — `CompiledModel.GpuOptions(precision = …)` in Kotlin — that can\n  request fp32 at a speed cost). Anything that sums many large values —\n  variance, `Σx²`, multi-axis mean — is a candidate.\n- **Approximation choices are not free.** Substituting a GELU or Swish\n  approximation changes numerics. A head with a wide output range can lose\n  real accuracy to the coarser form.\n- **`inplace=True` activations mutate a residual.** `out = self.act(x)` then\n  `x + out` adds `relu(x)`, not `x`. Swapping the activation silently changes\n  the math.\n- **A silent CPU fallback looks like success.** If the GPU output is\n  bit-identical to CPU fp32, it probably did not run on the GPU. Genuine fp16\n  execution drifts in the last digits. The runtime's own count is the line\n  `N operations will run on the GPU, and the remaining M operations will run\n  on the CPU.`; a node the delegate declines while building (`TopK requires\n  src tensor C dimension to be divisible by 4`) leaves only that log behind\n  — the checker's report can read fully-on-GPU while the op ran on the CPU\n  (`ai-edge-litert` 2.2.0 and 2.3.0).\n- **Reflect padding routes through `F.pad` even at `padding=0`** — every\n  conv with `padding_mode='reflect'` is affected, not just padded ones.\n  Slice+concat reflect pads are cheap for small pad widths.\n- **Channel-attention through a `Linear` confuses layout handling at\n  convert time** (`tfl.mul operands don't have broadcast-compatible\n  shapes`) — express channel attention as a 1×1 conv.\n- **`.chunk()` lowers to `SPLIT`** (GPU-rejected) — slice directly\n  instead; bit-exact.\n- **Constant folding can explode the file**: a frozen-param × constant\n  product materializes at full size per block (fp16 casting skips\n  non-weight constants, so it cannot rescue it). Feed one factor as a\n  runtime input — param × input never folds.\n\n## Output layout\n\nShip the result as a model recipe, not a one-off script:\n\n```\nmodels\u002F\u003Cfamily>\u002F\u003Cmodel>\u002F\n  README.md                 what the model is, what was produced\n  converted\u002F\n    README.md               environments, pipeline, verification results\n    export_*.py             one script per graph\n    dump_*_ref.py           reference dumps from the source model\n    verify_*.py             parity check against those references\n```\n\nKeep the export, the reference dump, and the verification separate so each\ncan be re-run alone. State the verification numbers in the README with the\ndevice they came from. Weights are not committed.\n\nThe tree above is this repo's convention. In your own project the directory\nnames matter less than the split — keep the same three separately-runnable\npieces wherever your models live.\n\nDo the model first. A demo can follow, and reusable inference code matters\nmore in it than a full UI.\n","# GPU-clean conversion\n\nA conversion is done when three things hold, in this order:\n\n1. it converts,\n2. every node runs on the GPU,\n3. **the on-device output matches the source model.**\n\nStep 3 is not implied by step 2. The delegate can report full residency and\nstill return wrong numbers — that is the failure mode most of the rewrites\nbelow exist for. Never call a model done without a numerical check against\nCPU or PyTorch on the actual device.\n\n## Loop\n\n**1. Convert plain first.** No patches. This tells you what the model\nactually needs rather than what you assumed.\n\n**2. Verify through the CompiledModel API before touching a device.**\n\n```python\nfrom litert_gpu_toolkit import check_gpu_compatibility, print_report\nprint_report(check_gpu_compatibility(\"model.tflite\"))\n```\n\nIt compiles the model for the GPU accelerator, runs every signature on\nrandom inputs, and compares the outputs against a CPU-compiled reference.\nA failed GPU compile names the offending op — route it through the table\nbelow. A pass with a CPU-fallback warning means some ops fell back to the\nCPU; treat them the same way if you need full residency. This exercises the\nhost GPU, so step 5 on the actual device is still required.\n\nWhen no GPU runtime is at hand (a CI job, a laptop without the accelerator),\na static pre-flight gives the first read: [edge-lint](https:\u002F\u002Fgithub.com\u002Fjohn-rocky\u002Fedge-compat)\nparses the `.tflite` and reports, per op, what a measured compatibility matrix\nsays for the target backend and runtime version — delegated, fallback,\nincorrect, or `unknown` (not yet measured, not \"unsupported\") — with the\nmatrix's rewrite hint next to each finding. It needs no runtime or device and\ndoes not replace the compile-and-compare above; use it to route symptoms\nthrough the table below before a device is available.\n\n**3. Map each symptom to a rewrite.** The rewrites live in\n`utilities\u002Flitert_gpu_toolkit` — plain Python, no build step. Outside this\nrepo, clone litert-samples and put `utilities\u002F` on `PYTHONPATH`, or vendor\nthe directory:\n\n| What you see | Rewrite |\n|---|---|\n| `GATHER_ND` named in the compile error | Find its source. Stride-2 slicing (`x[:, ::2]`, Focus stems, patch merging), `grid_sample`, `MaxPool` padding, bicubic `interpolate`, and reflect-mode `F.pad` all lower to it. `patch_grid_sample`, `patch_maxpool_zeropad`, `patch_interpolate`, `patch_patch_merging` |\n| A rank-5+ tensor named in the compile error | `PixelShuffle` (rank-6 reshape), windowed attention, `einops.rearrange`, packed-QKV attention head splits. `pixelshuffle_to_conv_transpose`, `patch_window_attention`, `patch_einops` |\n| `TRANSPOSE_CONV` rejected | Version skew, not a missing op. `ZeroStuffConvT1d` \u002F `ZeroStuffConvT2d` — zero-stuff plus a plain conv, exact to ~1e-7 |\n| `SELECT` \u002F `SELECT_V2` | `PReLU`, `ELU`, in-place index assignment, and `torch.where` masking. Replace with arithmetic: `x*(1-m) + v*m`. Keep v finite in fp16: −1e9 is −inf there, and (1 − m) × −1e9 is 0 × −inf = NaN on every valid position; −30000 is exact in fp16 and gives the same softmax in fp32 when the logits are bounded |\n| `BROADCAST_TO` | Two cases. On a compile-time constant: an outer product or `.expand` that did not fold — bake the result as a constant at its target shape. On a **runtime** tensor the GPU delegate rejects it outright, even at rank 4 — the canonical case is GQA's `repeat_kv` (`x[:,:,None].expand(...)`, which is also rank-5, so two walls in one line). Exact rewrite: `torch.cat([x[:, i:i+1].expand(b, n_rep, s, d) for i in range(n_kv)], dim=1)` — same head order, bit-exact. Tracked upstream: google-ai-edge\u002FLiteRT#9191 |\n| Masked attention wrong only on device: token 0 bit-exact, every later token wrong | Broadcast `ADD` whose LHS is a `BATCH_MATMUL` result (the `scores + mask[1,1,S,S]` idiom) silently miscomputed on ai-edge-litert 2.1.3 and 2.1.5 and is fixed in 2.1.6 (head-axis size-1 broadcast only; google-ai-edge\u002FLiteRT#8593, closed as fixed; the 2.1.6 release notes list GPU broadcast fixes). The signature mimics broken RoPE — tap the rope output before blaming it. Rewrites on the older runtimes: pre-expand the mask to `[1,H,S,S]`, or materialize the BMM as a second output |\n| An `ADD` result that is both a graph output **and** consumed downstream comes back wrong | Output aliasing: the returned tensor holds an *operand*, not the sum — `ADD` with two runtime operands (`SUB`\u002F`MUL` exact, `x + 1.0` exact). This is the shape of every explicit state update in a streaming\u002Frecurrent graph. Workaround: emit `acc * one` where `one` is a **runtime** input holding 1.0 — a constant folds straight back into the pattern. Tracked upstream: google-ai-edge\u002FLiteRT#8599 |\n| `RELU_0_TO_1` rejected by the GPU delegate | Emitted by hard-sigmoid \u002F `nn.Hardtanh(0,1)`. Accepted in litert 2.1.3, rejected from 2.1.5 on — a model at full residency on an older runtime hard-fails `CompiledModel` creation after an upgrade. Rewrite: `relu(x) - relu(x-1)`, exact. Tracked upstream: google-ai-edge\u002FLiteRT#8598 |\n| `DIV: No support of few identical inputs` \u002F `Expected 1 const input tensor(s)`, device only | The delegate declines an op whose two inputs are the same tensor, and ops whose inputs are all constants — together these split a perceiver-style block (softmax over a length-1 axis of a constant latent bank) into several partitions. Fixes: special-case the degenerate axis (a softmax over a length-1 axis is identically 1), or make one input non-constant. Note the sibling LayerNorm-over-a-constant pattern no longer reaches the delegate at all — the converter folds it to a single `MUL`. Tracked upstream: google-ai-edge\u002FLiteRT#9192 |\n| `NHWC node rewriter not found: amax` | `x.amax(...)`\u002F`x.max(dim)` in stable-softmax, adaptive norms, qk-norm. Rewrite channel reduce-max as `max_pool2d(x.reshape(N,1,C,H*W), kernel=(C,1))` — numerically identical — or drop the norm to 3D |\n| `Lowering not found: aten._fft_r2c` \u002F `aten.complex` | `torch.stft`\u002F`istft` and complex views have no lowering (fails before any GPU check). A DFT is a fixed linear map: windowed-DFT as `Conv1d` with the cos\u002Fsin basis baked into kernels (stride = hop), iSTFT as inverse-DFT matmul + overlap-add via zero-stuffed conv-transpose — exact. Model-selection corollary: prefer time-domain vocoder branches over iSTFT-based ones. ⚠ Library STFT-as-conv stacks (torchlibrosa-style) have **numerically mis-converted** (corr 0.83) while the op check looks clean — verify the spectrogram numerically or compute log-mel host-side |\n| Compiles, runs, output is wrong or NaN | The fp16 reduction family — and the trigger is the **fp16 accumulator passing 65504**, so a plain single-axis mean\u002Fsum over enough elements overflows just like variance does. `patch_safe_layernorm`, `patch_rmsnorm`, `patch_instance_norm`, `hierarchical_mean`. Caveats: at extreme magnitudes (\\|x\\| in the thousands) even the adaptive safe-LN form overflows when it reconstructs the large variance — the robust form stays entirely in the down-scaled domain (`xs = x\u002FS`, normalize `xs`, never multiply the variance back by `S²`); `hierarchical_mean` is exact only for power-of-two spatial dims (for arbitrary dims, cascade `\u002F2` avg-pools with `ceil_mode` so each stage averages ≤~49 elements). A static form keeps fp32 bit-exact and puts no reduce-max in the graph: scale each norm's input by a power of two 2^-k chosen per site from a calibration over the whole check set (headroom for the row's sum of squares *and* for the largest single x² — both overflowed in the case on record, the mean of squares did not), and eps by 4^-k. When eps·4^-k drops out of fp16's range (eps 1e-6 at k = 3), an all-zero row (padding) becomes rsqrt(0) = inf and 0 × inf = NaN: clamp the rsqrt at 2^k\u002F√eps — the zero row's own value, never reached in fp32. Diagnostic split: **all-zero\u002Fall-blank output = an overflow in one block; a result that starts near-correct and degrades with depth = precision compounding**, which no overflow patch (and no fp32-precision flag) fixes |\n| Head outputs exactly zero | RMSNorm `Σx²` overflowed fp16 to `inf`. `patch_rmsnorm` |\n| GPU compile fails only at fp16 precision with a kernel-source error (`write_imageh` on Adreno and Mali OpenCL, `write` on Metal, at the same source line), while fp32 precision compiles | A boolean value on the way to a graph output. In the case on record an output mask computed as `(mask > 0.5)` and cast back to float made all three GPUs fail; the same output computed from the float input mask by float ops (a strided slice) compiled at fp16 everywhere and stayed bit-exact in fp32. Small graphs with the same GREATER → CAST pair compiled, so bisect on the real graph. Keep masks float (1 = valid) from input to output |\n\n**4. Re-convert and re-verify.** Repeat 2–3 until the check passes.\n\n**5. Verify on device, numerically.** Run the same input through the source\nmodel and the on-device graph and compare. Correlation on the output tensor,\nplus a task-level check where one exists (argmax match, IoU, mask foreground\ncount). Record the device and the residency line (`N\u002FN` nodes) alongside the\nnumber — a result without both is not reproducible.\n\nIf residency is full and the output is wrong, bisect by intermediate: dump\ntensors at block boundaries and find the first one that diverges. The cause is\nusually a reduction, not the op you suspect.\n\n## Watch for\n\n- **fp16 is used even for an fp32 graph.** The delegate reduces in fp16\n  regardless of tensor dtype by default (the GPU options carry a precision\n  setting — `CompiledModel.GpuOptions(precision = …)` in Kotlin — that can\n  request fp32 at a speed cost). Anything that sums many large values —\n  variance, `Σx²`, multi-axis mean — is a candidate.\n- **Approximation choices are not free.** Substituting a GELU or Swish\n  approximation changes numerics. A head with a wide output range can lose\n  real accuracy to the coarser form.\n- **`inplace=True` activations mutate a residual.** `out = self.act(x)` then\n  `x + out` adds `relu(x)`, not `x`. Swapping the activation silently changes\n  the math.\n- **A silent CPU fallback looks like success.** If the GPU output is\n  bit-identical to CPU fp32, it probably did not run on the GPU. Genuine fp16\n  execution drifts in the last digits. The runtime's own count is the line\n  `N operations will run on the GPU, and the remaining M operations will run\n  on the CPU.`; a node the delegate declines while building (`TopK requires\n  src tensor C dimension to be divisible by 4`) leaves only that log behind\n  — the checker's report can read fully-on-GPU while the op ran on the CPU\n  (`ai-edge-litert` 2.2.0 and 2.3.0).\n- **Reflect padding routes through `F.pad` even at `padding=0`** — every\n  conv with `padding_mode='reflect'` is affected, not just padded ones.\n  Slice+concat reflect pads are cheap for small pad widths.\n- **Channel-attention through a `Linear` confuses layout handling at\n  convert time** (`tfl.mul operands don't have broadcast-compatible\n  shapes`) — express channel attention as a 1×1 conv.\n- **`.chunk()` lowers to `SPLIT`** (GPU-rejected) — slice directly\n  instead; bit-exact.\n- **Constant folding can explode the file**: a frozen-param × constant\n  product materializes at full size per block (fp16 casting skips\n  non-weight constants, so it cannot rescue it). Feed one factor as a\n  runtime input — param × input never folds.\n\n## Output layout\n\nShip the result as a model recipe, not a one-off script:\n\n```\nmodels\u002F\u003Cfamily>\u002F\u003Cmodel>\u002F\n  README.md                 what the model is, what was produced\n  converted\u002F\n    README.md               environments, pipeline, verification results\n    export_*.py             one script per graph\n    dump_*_ref.py           reference dumps from the source model\n    verify_*.py             parity check against those references\n```\n\nKeep the export, the reference dump, and the verification separate so each\ncan be re-run alone. State the verification numbers in the README with the\ndevice they came from. Weights are not committed.\n\nThe tree above is this repo's convention. In your own project the directory\nnames matter less than the split — keep the same three separately-runnable\npieces wherever your models live.\n\nDo the model first. A demo can follow, and reusable inference code matters\nmore in it than a full UI.\n",{"name":5,"description":7},"main","fb7bd2edd795f0c61a09757f94456b5d19e66bc7","skills\u002Fgpu-clean-conversion",134,"https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples","https:\u002F\u002Fdevelopers.google.com\u002Fedge\u002Flitert","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge","2026-11-13 14:36:51",{"source":8,"skillName":5,"commands":52,"prompt":62,"zipAvailable":63},[53,56,59],{"tool":54,"command":55},"npx","npx skills add google-ai-edge\u002Flitert-samples --skill gpu-clean-conversion",{"tool":57,"command":58},"bunx","bunx skills add google-ai-edge\u002Flitert-samples --skill gpu-clean-conversion",{"tool":60,"command":61},"pnpm","pnpm dlx skills add google-ai-edge\u002Flitert-samples --skill gpu-clean-conversion","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Fgpu-clean-conversion\u002FSKILL.md",true,[],[66,79,102,128,145,158],{"skillNo":67,"name":68,"nameZh":6,"description":69,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":70,"categories":71,"keywords":72,"publishedAt":38},"01M6BY5T6RQENC1DMWZ8NZ8JFF","on-device-verification","Prove a converted or quantized LiteRT model on the actual device via the CompiledModel API - confirm GPU residency, compare device output against the source model, and diagnose device-only failures such as silent CPU fallback, whole-graph compile ceilings, and fp16 range breaks. Use after conversion or quantization, when device output is wrong or NaN, when a clean graph fails to compile only on device, or when GPU and CPU outputs are suspiciously identical.","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Fon-device-verification\u002FSKILL.md",[],[73,74,75,32,76,77,78,36,37,18],"on device verification","verification","graph","FP16","compile","diagnose",{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":38},"01M6BY5F447DRN3YDJVRWDG9MZ","litert-google-tensor","Compiles LiteRT (.tflite) models ahead-of-time for Google Tensor TPUs (Tensor G5 on Pixel 10, Tensor G6 on Pixel 11) locally on Linux x86_64 or remotely via Google Colab (--colab on macOS, Windows, or ARM hosts), benchmarks them on a USB-attached Pixel with LiteRT's benchmark_model (NPU latency, peak memory, TPU tile time, TPU energy, optional CPU baseline), and packages compiled models for an Android Gradle app as a Google Play AI pack with the Tensor NPU runtime module and a Kotlin loader, applies it to the app and verifies it on a Pixel. Use when the user asks to compile, AOT-compile, or prepare a .tflite model for the Pixel TPU\u002FNPU, check whether a .tflite is already Tensor-compiled, measure on-device NPU performance of a compiled model, or package or deploy a compiled model into their Android app (or replace the one it ships). Don't use for Qualcomm, MediaTek, or Samsung NPUs, GPU-only benchmarking, LiteRT-LM or LLM (.litertlm) models, on-device JIT compilation, non-Gradle builds, or publishing to Google Play.","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Flitert-google-tensor\u002FSKILL.md",[],[86,87,88,37,36,89,90,91,92,93,94,95,96,18,97,98,99,100,101,77],"google","litert google tensor","tensor","windows","benchmarking","benchmark","baseline","tpu","memory","packages","android","play","package","kotlin","linux","performance",{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":106,"categories":107,"keywords":113,"publishedAt":38},"01M6BY5BY9H6SDG9GRRTT04EWA","litert-conversion-workflow","Convert a Hugging Face LLM or vision-language model checkpoint into a .litertlm bundle that runs on the LiteRT-LM runtime with verified quality - classify the architecture against known runtime walls, pick the recipe family (dense, reasoning, hybrid SSM, VLM), export, quantize, gate the result against the source model, and publish. Use when converting a new LLM or VLM to LiteRT-LM, when a converted bundle crashes on the first message or dies at engine creation, when a quantized model answers worse than its source, or when deciding whether a model is convertible at all.","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Flitert-conversion-workflow\u002FSKILL.md",[108],{"categoryNo":109,"categoryName":110,"taxonomyType":111,"taxonomyTypeDesc":112},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[114,37,115,21,29,116,117,118,119,120,22,23,121,122,123,124,125,36,126,127],"litert conversion workflow","workflow","message","classify","result","face","source","export","quantize","at","architecture","pick","publish","creation",{"skillNo":129,"name":130,"nameZh":6,"description":131,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":132,"categories":133,"keywords":134,"publishedAt":38},"01M6BY5835F3RMMW9KCCZ006JM","litert-compiled-model-migration","Rapidly migrate an Android application from legacy TensorFlow Lite (TFLite) to the modern LiteRT CompiledModel API (2.x) in Open Source GitHub repositories, keeping the TensorFlow Lite Support Library as LiteRT Support (litert-support 2.3.0). Covers buffer reuse, NPU JIT compilation, a reference implementation, and a two-stage verification gate (compile check, then an on-device parity test against a golden output).","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Flitert-compiled-model-migration\u002FSKILL.md",[],[135,36,37,74,136,137,138,139,140,96,32,141,142,143,77,144],"litert compiled model migration","library","jit","support","github","test","migrate","open source","open","tensorflow",{"skillNo":146,"name":147,"nameZh":6,"description":148,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":149,"categories":150,"keywords":151,"publishedAt":38},"01M6BY52X9KX3VRXQJS622DZH3","compiled-model-app-scaffolding","Build a new Android app (Kotlin, Compose) around a verified LiteRT model using the CompiledModel API - the app architecture, the inference-layer lifecycle rules, model delivery, and the UI traps that masquerade as model bugs. Use when turning a converted and device-verified model into a demo or product app, when an app's inference layer leaks memory or blocks the UI, or when a model that verified clean looks wrong inside an app.","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Fcompiled-model-app-scaffolding\u002FSKILL.md",[],[36,152,124,153,99,154,94,155,37,96,32,156,157],"compiled model app scaffolding","ui","lifecycle","product","inference","demo",{"skillNo":159,"name":160,"nameZh":6,"description":161,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":162,"categories":163,"keywords":164,"publishedAt":38},"01M6BY501NESZTA4NSEKX9BSDE","benchmark-on-ddp","Measure a LiteRT .tflite model with `benchmark_model` on Developer Device Platform (DDP) lab phones through `litert benchmark --ddp`, or on the Mac you run it from, and turn the session into rows of the performance leaderboard under benchmark\u002Fleaderboard - one matrix entry per model, one session per accelerator, collect results.pb and runtime_info.pb, rebuild board.json, check the page, commit the data. Use when a model needs latency and memory numbers per platform, device and accelerator, or when a new benchmark_model release means re-measuring the board.","https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Fblob\u002Fmain\u002Fskills\u002Fbenchmark-on-ddp\u002FSKILL.md",[],[165,91,166,167,168,169,170,101,171,172,173,94,36,37],"benchmark on ddp","data","developer","commit","release","collect","json","run","page"]