runpod-flash

Runpod SDK apps (Flash)

For an explicit Dockerfile/template project request, use runpod-build-template. Do not replace that deliverable with Flash unless the user chooses its code-first workflow.

Write Python locally, run it on remote Runpod GPUs/CPUs with rp flash dev, then rp flash deploy to ship. Flash is part of the runpod package (2.0): one SDK, one rp CLI, one login. The standalone runpod-flash package is deprecated and gets no fixes after October 29, 2026.

rp --help and rp flash <command> --help are authoritative for the command surface. This skill is the mental model, the decision rules, and the gotchas that help output does not carry. Confirm the installed version with pip show runpod before concluding a command or parameter is unavailable.

Which SDK is this project on?

Check before writing code. The two APIs do not mix.

You see	Project is on	Do
import runpod, runpod.App(...), @app.queue, rp flash ...	Runpod SDK 2.0	This file + reference/
from runpod_flash import Endpoint, @Endpoint(...), flash dev, runpod-flash in deps	Legacy runpod-flash	reference/runpod-flash-legacy.md to maintain it, reference/migrate-from-runpod-flash.md to move it
A new project	Runpod SDK 2.0	Start below

Write new code on the SDK. For an existing runpod-flash project, propose migrating and check its Known differences first; don't migrate silently.

If the SDK path fails on an SDK bug, runpod-flash is the fallback: a traceback inside runpod/apps/, a worker that crashes before your function runs, or the same code working on runpod-flash. Upgrade runpod first; the steps to check for a fix and fall back are in reference/runpod-flash-legacy.md.

Worked examples first for anything multi-step. Flash appears in end-to-end paths: 03 variant B (whisper endpoint), 07 (network-volume handoff) and 08 (fine-tune → serve); the full index is runpod/golden-paths/README.md. Runnable single-file apps live in examples/.

Load on demand:

Need	Read
Install, rp login, rp flash init, every rp command	reference/setup-and-cli.md
Decorator parameters, call styles, Queue/Api/Endpoint clients, volumes, Model, Secret, GPU & CPU names	reference/api.md
Worked patterns: choosing a model, warm model loading, CPU→GPU pipeline, fan-out, tasks, raw HTTP	reference/patterns.md
Moving a runpod-flash project to the SDK	reference/migrate-from-runpod-flash.md
Maintaining a runpod-flash project as it is	reference/runpod-flash-legacy.md

Quick start: uv add "runpod>=2.0.0" (or pip install "runpod>=2.0.0") → rp login (or export RUNPOD_API_KEY=...) → rp flash init my-app → cd my-app && rp flash dev main.py.

The app model

import runpod

app = runpod.App("my-app")                     # names the group of resources

@app.queue(gpu="4090", dependencies=["torch"]) # queue-based serverless endpoint
def infer(x: list[float]):
    import torch
    return {"sum": torch.tensor(x, device="cuda").sum().item()}

@runpod.local_entrypoint                       # runs on YOUR machine
def main():
    print(infer.remote([1.0, 2.0, 3.0]))       # runs on a Runpod worker

Decorator	Creates	Use for
@app.queue(...) on a function	Autoscaling queue-based endpoint	Request/response inference, batch jobs
@app.api(...) on a class	Load-balanced endpoint serving routes (@runpod.get/@runpod.post, setup in @runpod.init)	HTTP services, multi-route APIs, load-once model state
@app.task(...) on a function	One ephemeral Pod per call, terminated when it returns	Training, fine-tuning, long jobs

Call styles on a queue or task handle: fn.remote(...) (blocks), await fn.remote.aio(...), fn.spawn(...) (returns a Job, or a TaskJob for a task), fn.stream(...) (queue generator functions only), fn.local(...) (runs on your machine). API classes are called by route: MyApi.post("/path", body).

Dev vs deploy

• rp flash dev main.py — iterate. Creates temporary endpoints (dev-<app>-<resource>-<hash>), runs the @runpod.local_entrypoint on your machine, streams the workers' print output to your terminal, and re-runs after every file save. Ctrl-C deletes the endpoints. There is no local HTTP server; the entrypoint is how you exercise the code.
• rp flash dev main.py --once — run the entrypoint once, tear down, exit non-zero on failure. The smoke test for CI and for "does the migrated code work".
• rp flash deploy — ship. Builds an artifact (≤1,500 MB, torch excluded on built-in GPU images) and creates persistent endpoints in the default environment (--env NAME for others). Re-run to update. --build-only packages without deploying, a cheap offline check that discovery and dependency resolution work.

Dev and deploy run the same module source, so module-level imports, constants and helpers behave identically in both. Deploy packages the project directory, but a dev session ships only the module's source, so keep the app in one file when you use rp flash dev.

Autonomous dev loop

A dev session is long-running. Three rules:

• Run it in the background with stdin closed (< /dev/null). With no stdin it re-runs the entrypoint on every file save, so you never need to press Enter.
• Capture its output to a log file and read it. Worker prints, tracebacks, ✗ remote execution failed: ... and the per-run ✓ done in ... / ✗ failed in ... lines land there.
• Stop it with kill (SIGTERM) or SIGINT, so it deletes its endpoints, and confirm the log ends with cleaning up N deleted. A second signal stops waiting for a running entrypoint and goes straight to cleanup. kill -9 skips cleanup. When one run is enough, prefer --once; it cleans up on its own.

rp flash dev main.py < /dev/null > /tmp/rp-dev.log 2>&1 &
DEV=$!
# poll the log (bounded) for the run result: "✓ done in ..." or "✗ failed in ..."
for i in $(seq 1 120); do grep -qE "✓ done in|✗ failed in" /tmp/rp-dev.log && break; sleep 3; done
tail -40 /tmp/rp-dev.log
# edit main.py and save -> the session re-scans, refreshes endpoints, re-runs main()
# (each run appends a new result line; read the last one)
kill $DEV; wait $DEV; tail -3 /tmp/rp-dev.log   # expect "cleaning up N deleted"

For a single check, rp flash dev main.py --once is simpler and cleans up on its own. If a session died without cleanup, list leftovers with runpodctl serverless list and delete the dev-<app>-... endpoints this session created, by ID.

Choosing the resource shape

Default to plain Python on the Runpod runtime image: dependencies=[...] and system_dependencies=[...] install on the worker, so no Dockerfile is needed, and even large HuggingFace models stay in code (see reference/patterns.md).

• image="org/image:tag" on any decorator runs your function on that image; it needs Python 3, not a Serverless handler. Use it for system libraries or a specific CUDA stack. registry_auth= names saved credentials for a private image (rp registry add).
• A prebuilt inference server image (vLLM, worker-comfyui) that already implements a Serverless handler is not an SDK app. Deploy it with runpodctl or runpod-mcp and call it with runpod.Endpoint("ENDPOINT_ID").
• An existing endpoint by ID: runpod.Endpoint("ENDPOINT_ID").run_sync({...}).
• A resource in another deployed app: runpod.Queue(app="media", name="transcribe") / runpod.Api(app="media", name="inference").

Gotchas

1. Queue arguments and results must be JSON-serializable. Queue jobs are plain JSON in dev and deploy alike. A non-JSON argument fails in dev with arguments must be json-serializable and against a deployed endpoint with Python's own JSON TypeError. Pass URLs or base64, not objects. Whole-number floats can arrive as integers (1.0 → 1), so cast numeric inputs explicitly (torch.tensor(x, dtype=torch.float32)).
2. One file per app in dev. A dev session ships only the module's source, so a sibling local module imports in deploy but fails in dev. For dev sessions, keep helpers in the same file or in an installable package.
3. Keep blocking work out of module scope. The SDK imports your module to discover resources, locally and on every worker. Put heavy imports (torch) and work inside functions; guard local-only code with runpod.is_local().
4. gpu= and cpu= are exclusive (gpu and cpu are mutually exclusive). GPU strings can be short ("4090", "H100") or a list of acceptable GPUs.
5. Default workers=(0, 3) (runpod-flash was (0, 1)). Set it explicitly when cost or concurrency matters. idle_timeout is seconds (default 60).
6. Auto GPU switching needs workers max ≥ 5 when you pass a list of GPUs.
7. 10 MB payload limit on requests. Return large outputs via a volume or object storage URL.
8. Volumes outlive everything. Dev sessions and rp flash undeploy delete endpoints, not the NetworkVolumes they created, and those keep billing. Queue and API resources mount one volume at /runpod-volume; read vol.path inside the remote function, never at import time. Load model weights through VolumeCache, not straight off the mount (reference/patterns.md).
9. Tasks bill until they return. A .spawn()ed task that you never wait() or cancel() keeps its Pod running after your script exits.
10. Native CUDA libraries go in dependencies too, e.g. faster-whisper needs nvidia-cublas-cu12 + nvidia-cudnn-cu12 or it silently falls back to CPU.
11. Dev sessions and tasks need local Python 3.10–3.14. They ship code pickled by your interpreter, so the worker matches your local version. Deploy falls back to 3.12 with a warning on other versions; rp flash deploy --python-version sets it explicitly.
12. A set RUNPOD_API_KEY wins over rp login. A stale exported key causes 401s while the saved login is fine. Check with runpodctl user, then unset RUNPOD_API_KEY or export a good key.
13. The torch family comes from the GPU image; never pin it. Builds leave torch, torchvision, torchaudio and triton out of the artifact whether you list them or a dependency pulls them in (peft, accelerate, sentence-transformers), plus the wheels only torch needs. GPU workers use the image's build (torch 2.9.1+cu128), so a version pin on these packages is ignored. In an app that mixes CPU and GPU resources, a CPU resource whose dependencies need torch installs it at cold start; give it container_disk_gb room for that (the build warns when this applies).
14. Dev sessions run the runtime image's own runpod. Deploys vendor your local package, but rp flash dev workers import the runpod baked into the built-in image. An AttributeError on a runpod. name in a dev worker traceback means the image is behind your SDK; rp flash deploy runs your installed version.
15. Teardown by app and environment: rp flash undeploy --app APP --env default -y removes one environment's endpoints; rp flash app delete APP --yes removes every environment (without --yes it prompts, and aborts with no terminal). Then delete volumes you no longer need.
16. Queue and API names need 3+ characters to deploy. The name defaults to the function or class name, and dev sessions don't check it, so class SD passes rp flash dev and fails rp flash deploy. Rename it or pass name=.
17. Every endpoint gets your account API key. Deploy and dev put RUNPOD_API_KEY in each endpoint's template env, whether or not its code calls another resource, so anyone who can read the template can read the key.
18. Re-running the same app soon after a dev session can fail. Dev sessions leave their templates behind, and the platform holds them for up to about 90 minutes, so a new rp flash dev of the same app fails with "endpoint template names must be unique". Wait, or rename the runpod.App.

Resources

• Setup & CLI: reference/setup-and-cli.md · API: reference/api.md · Patterns: reference/patterns.md · Examples: examples/
• SDK source and upstream examples: https://github.com/runpod/runpod-python (examples/apps/)
• Docs: https://docs.runpod.io/sdk/overview · CLI: https://docs.runpod.io/sdk/cli · Migration: https://docs.runpod.io/sdk/migrate-from-flash