[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6BY2B8JFGYNP3JC62EW8ABS":3,"skill-same:01M6BY2B8JFGYNP3JC62EW8ABS":72},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":50,"commitSha":51,"skillDirectory":52,"forksCount":53,"repositoryHtmlUrl":54,"repositoryHomepageUrl":55,"ownerHtmlUrl":56,"ownerBlogUrl":12,"ownerDescription":57,"updatedAt":58,"install":59,"resources":71},"01M6BY2B8JFGYNP3JC62EW8ABS","nodetool-model-provider-config",null,"Configure NodeTool model providers and API keys (list_secrets, request_secret), pick a model with find_model, or set up local Ollama, llama.cpp and Hugging Face models.","nodetool-ai\u002Fnodetool","nodetool-ai","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F164873569?v=4","01M6BXPFDW36K9K1MR8ZH63HG2","nodetool.ai","01M6BXPFJ893XM219GQN60B0WK",635,"AGPL-3.0","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fpackages\u002Fsystem-skills\u002Fnodetool-model-provider-config\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"nodetool model provider config","config","model","provider","nodetool","hugging face","llama.cpp","list","request","local","up","api keys","Providers","secrets","ollama","api","configure","cpp","pick","SET","2026-11-10 10:12:30",false,"---\nname: nodetool-model-provider-config\ndescription: \"Configure NodeTool model providers and API keys (list_secrets, request_secret), pick a model with find_model, or set up local Ollama, llama.cpp and Hugging Face models.\"\nfeatured: true\n---\n\nYou help users configure AI model providers and select the right models for their tasks.\n\n# In the product\n\nThe agent has no shell. Configure keys and pick models with these tools:\n\n1. `list_secrets` shows which credentials are configured. It never returns a\n   value.\n2. `request_secret {key, reason, help_url}` opens a dialog where the user types\n   a missing key. The value is saved from the dialog and never passes through\n   the agent. Never ask the user to paste a key into chat. When the run has no\n   interactive client, name the missing key and point the user to\n   Settings > Secrets.\n3. `find_model {capability}` returns a real `{provider, model_id}` and a `ref`\n   to assign verbatim to a node's model property. Call it before adding any\n   generic AI node. Never write a model id from memory, including the examples\n   below.\n4. `list_provider_models {provider}` lists one provider's language models.\n\nInside a code action the same calls are `nodetool.secrets` \u002F\n`nodetool.settings` (load `api-settings`) and `nodetool.models` (load\n`api-models`).\n\nThe `nodetool` provider is NodeTool's managed service. Its models run on\nplatform keys and draw from the user's credit balance, so it needs no key of\nthe user's own. Every other provider bills the user's own account.\n\n# Provider Overview\n\n| Provider | Type | Key Env Var | Models |\n|----------|------|-------------|--------|\n| **OpenAI** | Cloud | `OPENAI_API_KEY` | GPT, GPT-Image, TTS, Whisper |\n| **Anthropic** | Cloud | `ANTHROPIC_API_KEY` | Claude Opus, Sonnet, Haiku |\n| **Gemini (Google)** | Cloud | `GEMINI_API_KEY` | Gemini, Veo, Nano Banana |\n| **xAI** | Cloud | `XAI_API_KEY` | Grok |\n| **Ollama** | Local | `OLLAMA_API_URL` | Qwen, Llama 3, Mistral, any GGUF. Separate install, must be running |\n| **HuggingFace** | Cloud | `HF_TOKEN` | Hosted Inference Providers |\n| **HuggingFace local** (`huggingface-local`) | Local | — | Diffusers\u002FTransformers models via the HuggingFace Python pack |\n| **NodeTool** (`nodetool`) | Cloud, managed | — (credits) | Curated models |\n| **FAL** (`fal_ai`) | Cloud | `FAL_API_KEY` | Fast image\u002Fvideo generation |\n| **KIE** (`kie`) | Cloud | `KIE_API_KEY` | Image, video and music models from many vendors |\n| **ElevenLabs** (`elevenlabs`) | Cloud | `ELEVENLABS_API_KEY` | TTS, voice, sound effects |\n| **Replicate** | Cloud | `REPLICATE_API_TOKEN` | Community models |\n| **Comfy Cloud** | Cloud | `COMFY_API_KEY` | No model list; runs a ComfyUI workflow on Comfy's GPUs via `lib.comfy.RunWorkflowOnCloud` |\n| **vLLM** | Local | `VLLM_BASE_URL` (required) | Self-hosted, OpenAI-compatible |\n| **LM Studio** | Local | `LMSTUDIO_API_URL` | Models loaded in LM Studio's local server |\n| **llama.cpp server** (`llama_cpp`) | Local | `LLAMA_CPP_URL` (required) | A `llama-server` you run |\n| **llama.cpp local** (`node_llama_cpp`) | Local | — | GGUF files in-process; runtime from Package Manager → Engines & tools |\n| **MLX** | Local | — | MLX Python pack, Apple Silicon only |\n\nOther registered chat providers (any of these is valid for the CLI's `-p\u002F--provider`):\n`groq`, `mistral`, `deepseek`, `moonshot`, `sarvam` (`SARVAM_API_KEY`, also\nBulbul TTS and Saaras ASR), `minimax`, `zai`, `cerebras`, `alibaba`\n(`DASHSCOPE_API_KEY`, Qwen via Alibaba Cloud Model Studio), `together`,\n`openrouter`, `requesty`, `opper`, `cheaperinference`, `api_route`, `runcrate`, `standardcompute`, `uberllm`, `codex`, `claude_agent_sdk`, `lmstudio`,\n`azure_openai` (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, models from\n`AZURE_OPENAI_DEPLOYMENTS`), `bedrock` (`BEDROCK_ACCESS_KEY_ID`,\n`BEDROCK_SECRET_ACCESS_KEY`, `BEDROCK_REGION`).\n\n# From a shell (self-host)\n\nThese run in the user's terminal, not in the product. `nodetool models\nproviders` shows configured providers, `nodetool models recommended` the curated\nlist, and `nodetool models by-provider \u003Cprovider>` one provider's ids.\n\n```bash\n# Via CLI (encrypted storage)\nnodetool secrets store OPENAI_API_KEY\nnodetool secrets store ANTHROPIC_API_KEY\nnodetool secrets store GEMINI_API_KEY\nnodetool secrets store HF_TOKEN\nnodetool secrets store FAL_API_KEY\nnodetool secrets store REPLICATE_API_TOKEN\nnodetool secrets store COMFY_API_KEY\n\n# Via environment variables\nexport OPENAI_API_KEY=sk-...\nexport ANTHROPIC_API_KEY=sk-ant-...\nexport GEMINI_API_KEY=AI...\nexport HF_TOKEN=hf_...\nexport FAL_API_KEY=...\nexport REPLICATE_API_TOKEN=r8_...\nexport COMFY_API_KEY=comfyui-...\nexport OLLAMA_API_URL=http:\u002F\u002Flocalhost:11434\n```\n\n# Model Selection by Task\n\nThe tables name model families as starting points. `find_model` ranks what is\nactually available on this install and returns the exact id.\n\n## whisper.cpp speech recognition\n\nBoth providers are local-only and unavailable on the cloud profile.\n`whisper_cpp` runs speech recognition in the backend through the optional\n`@fugood\u002Fwhisper.node@1.1.3` runtime package. Install **whisper.cpp** from the\nPackage Manager, then download a GGML model from Models. It serves\n`nodetool.text.AutomaticSpeechRecognition` and `whisper_cpp.LiveTranscription`.\nUse the absolute model path returned by discovery unchanged as the ASR model id.\n\nModels use the Hugging Face hub cache (see HuggingFace models below for how its\nlocation resolves). `WHISPER_CPP_MODELS_DIR` adds another directory.\n`WHISPER_CPP_GPU_BACKEND` accepts `auto`, `metal`, `cuda`, `vulkan`, or `cpu`.\n`auto` uses Metal on macOS. On Windows and Linux it tries the CUDA build, then\nthe Vulkan build, then the default CPU build. Restart the backend after\nchanging the backend setting. Live transcription accepts base64 PCM16\nmono chunks with an optional `content_metadata.sample_rate` and uses Silero\nVAD when installed, otherwise fixed windows.\n\n`whisper_cpp_server` calls a user-run `whisper-server`. Set\n`WHISPER_CPP_SERVER_URL` to its base URL. The model id is `default`, representing\nthe model loaded by that server. It requires no native runtime package.\n\n## Language \u002F Chat\n\n| Need | Family | Provider | Notes |\n|------|--------|----------|-------|\n| Best quality | GPT flagship, Claude Opus or Sonnet | OpenAI, Anthropic | Highest capability |\n| Good balance | GPT mini, Gemini Flash | OpenAI, Gemini | Fast + cheap |\n| Local\u002Fprivate | Llama, Qwen | Ollama | No data leaves machine |\n| Lightweight local | Small Llama, Mistral 7B | Ollama | Low memory |\n| Code | Claude Sonnet, GPT flagship | Anthropic, OpenAI | Best for coding |\n\n## Image Generation\n\n| Need | Model | Provider | Notes |\n|------|-------|----------|-------|\n| Best quality | FLUX.2 Dev | HuggingFace, FAL | State-of-art |\n| Fast | FLUX Schnell | HuggingFace | Quick iterations |\n| Versatile | SDXL | HuggingFace | Many LoRAs available |\n| API-based | GPT Image 2, Nano Banana | OpenAI, Gemini\u002FKIE | No local GPU needed |\n\n## Video Generation\n\n| Need | Model | Provider |\n|------|-------|----------|\n| Best quality | Sora 2 Pro | OpenAI (KIE) |\n| Fast | Wan 2.6 | KIE |\n| Image-to-video | Kling 2.6 | KIE |\n| Talking avatar | Kling AI Avatar | KIE |\n\n## Speech & Audio\n\n| Need | Model | Provider |\n|------|-------|----------|\n| TTS (quality) | ElevenLabs | ElevenLabs |\n| TTS (fast\u002Ffree) | Kokoro | HuggingFace pack, MLX pack, Transformers.js |\n| ASR (accuracy) | Whisper Large V3 | HuggingFace pack, whisper.cpp |\n| ASR (fast) | Whisper Turbo | HuggingFace pack, MLX pack, whisper.cpp |\n\n## Embeddings\n\n| Need | Model | Provider |\n|------|-------|----------|\n| General text | text-embedding-3-small | OpenAI |\n| Best quality | text-embedding-3-large | OpenAI |\n| Local\u002Ffree | sentence-transformers | HuggingFace |\n\n# Local Model Setup\n\n## Ollama (Easiest)\n\nOllama is a separate program. NodeTool does not ship or start it, so it must be\nrunning before its models appear. On macOS and Windows, download it from\nollama.com. On Linux:\n\n```bash\n# Install\ncurl -fsSL https:\u002F\u002Follama.com\u002Finstall.sh | sh\n\n# Pull models\nollama pull llama3\nollama pull mistral\nollama pull qwen2\n\n# Verify\nollama list\n\n# NodeTool finds Ollama at 127.0.0.1:11434\n# Override: OLLAMA_API_URL in Settings → Integrations → Local Model Servers,\n# or export OLLAMA_API_URL=http:\u002F\u002Fhost:11434\n```\n\n`OLLAMA_CONTEXT_LENGTH` sets the context window sent with each request.\nUnset, NodeTool uses the model's own value, capped at 32768.\n\n## Python packs (HuggingFace local, MLX, Wan2GP)\n\nLocal Diffusers\u002FTransformers models, MLX models and Wan2GP video run in the\nPython worker, and each needs its pack.\n\n- Desktop app: **Tools → Package Manager → Python nodes**. Installing a pack\n  sets up Python first if it is missing. The HuggingFace pack pulls a PyTorch\n  2.14 build matched to the GPU. MLX is offered only on Apple Silicon Macs.\n  On a Mac, both packs need Apple Silicon and macOS 14 or newer.\n- Without the desktop app: a Python 3.11+ env with\n  `pip install nodetool-core nodetool-huggingface` (plus `nodetool-mlx` on\n  Apple Silicon), then start the server with `NODETOOL_PYTHON` pointing at that\n  env's `python`. `curl -fsSL https:\u002F\u002Fraw.githubusercontent.com\u002Fnodetool-ai\u002Fnodetool\u002Fmain\u002Finstall.sh | bash -s -- -y --pack huggingface`\n  builds that env on Linux and macOS. `--pack` takes `huggingface`, `mlx`, or\n  `wan2gp`. On macOS, always set `NODETOOL_PYTHON` to the env it prints.\n- `NODETOOL_TORCH_DEVICE` (`cuda`, `cuda:1`, `mps`, `cpu`) picks the device.\n- Wan2GP nodes call a Wan2GP server the user runs. `WAN2GP_MCP_URL` (default\n  `http:\u002F\u002F127.0.0.1:7866\u002Fmcp`) or the node's `server_url` points at it.\n\n## HuggingFace models\n\nDownload local models in the Model Manager before running the workflow. A node\nthat downloads on first use can fail on a gated repo or stall a run.\n\nFor gated models:\n1. Accept the license on the model page at huggingface.co\n2. Store `HF_TOKEN` in Settings (or `nodetool secrets store HF_TOKEN`). Model\n   Manager downloads use it\n3. Download the model in the Model Manager\n\nModels go to the Hugging Face hub cache: `HF_HUB_CACHE`, else\n`HUGGINGFACE_HUB_CACHE`, else `$HF_HOME\u002Fhub`, else\n`$XDG_CACHE_HOME\u002Fhuggingface\u002Fhub`, else `~\u002F.cache\u002Fhuggingface\u002Fhub`. The\nserver and the Python worker (through `huggingface_hub`) apply this one order.\nThe desktop app keeps `~\u002F.cache\u002Fhuggingface` when `XDG_CACHE_HOME` is set (as\nin Flatpak) but that cache already holds models.\n\n## llama.cpp\n\nTwo providers. `node_llama_cpp` loads GGUF files in-process after the runtime\nis installed from Package Manager → Engines & tools. `llama_cpp` calls a\n`llama-server` the user runs, at `LLAMA_CPP_URL`. Best for CPU inference and\nquantized models.\n\n`MODEL_FOLDERS` (Settings → Folders) adds folders, such as a ComfyUI `models`\nfolder, that `node_llama_cpp` and `stable_diffusion_cpp` search with their\nsubfolders. llama.cpp lists the GGUF language models in them.\nstable-diffusion.cpp lists `.safetensors` checkpoints that bundle their VAE and\ntext encoders. Separate folders with `;`.\n\n## whisper.cpp and Transformers.js\n\nwhisper.cpp is speech recognition only (see above). Transformers.js runs small\nONNX models in-process after its runtime is installed from Package Manager →\nSoftware. Its cache is `TRANSFORMERS_JS_CACHE_DIR`.\n\n## MLX (Apple Silicon only)\n\nNeeds the MLX pack and macOS 14 or newer. Lower memory usage than standard\nPyTorch.\n\n# Local Inference Performance\n\n| Framework | Throughput | Memory | Hardware |\n|-----------|-----------|--------|----------|\n| **llama.cpp** | Medium | Excellent | CPU, GPU |\n| **MLX** | Good | Excellent | Apple Silicon |\n| **Transformers** | Medium | Good | Any |\n\n# Provider-Agnostic Nodes\n\nThese nodes work with any provider — just select the model:\n\n| Node | Purpose |\n|------|---------|\n| `nodetool.agents.Agent` | Any LLM for chat\u002Freasoning |\n| `nodetool.image.TextToImage` | Any image generation model |\n| `nodetool.image.ImageToImage` | Any image transformation model |\n| `nodetool.video.TextToVideo` | Any video generation model |\n| `nodetool.video.ImageToVideo` | Any image-to-video model |\n| `nodetool.audio.TextToSpeech` | Any TTS model |\n| `nodetool.text.AutomaticSpeechRecognition` | Any ASR model |\n\n# Provider Capabilities\n\n| Capability | OpenAI | Anthropic | Google | Ollama | HF |\n|-----------|--------|-----------|--------|--------|-----|\n| Chat\u002FText | yes | yes | yes | yes | yes |\n| Vision | yes | yes | yes | some | yes |\n| Image Gen | yes (GPT-Image) | no | no | no | yes |\n| Video Gen | no | no | yes (Veo) | no | some |\n| TTS | yes | no | no | no | yes |\n| ASR | yes (Whisper) | no | no | no | yes |\n| Embeddings | yes | no | yes | yes | yes |\n| Tool Calling | yes | yes | yes | some | no |\n\n# Common Pitfalls\n\n- **Wrong key env var name**: Each provider has a specific name (see table above)\n- **Ollama not running**: NodeTool does not start it. Start the Ollama app or `ollama serve`\n- **Gated HF models**: Must accept terms on huggingface.co first\n- **GPU memory**: Large models need 8-24GB VRAM; use quantized versions\n- **PyTorch on the CPU despite a GPU**: the desktop app installs CUDA 12.8 for Turing (RTX 20xx) and newer (driver 570+), CUDA 12.6 for Maxwell to Volta, and CPU for Kepler, pre-RDNA 2 AMD and AMD on Windows. It also falls back to CPU, with a warning, when no GPU build resolves. A manual `pip install torch` on Linux gets CUDA 13.0, which needs driver 580+ and Turing or newer, as does the Docker worker image\n- **No local provider in the model menu**: the Python pack is missing, or `NODETOOL_PYTHON_ON_DEMAND=true` and no workflow has started the worker yet\n- **Rate limits**: Cloud providers have rate limits; implement retries or use local\n- **Model ID mismatch**: Take ids from `find_model` or `list_provider_models`, never from memory. From a shell, `nodetool models by-provider \u003Cprovider>` lists them\n- **Key pasted into chat**: Use `request_secret`. A key in a chat message is stored in the thread\n\n# Repository only: custom provider\n\nThis is TypeScript for contributors working in a NodeTool checkout. The\nin-product agent cannot add a provider. Providers extend `BaseProvider` from `@nodetool-ai\u002Fruntime` (not `@nodetool-ai\u002Fcore`).\nBoth `generateMessage` and `generateMessages` take a single **args object**.\n\n```typescript\nimport {\n  BaseProvider,\n  type ProviderId,\n  type Message,\n  type ProviderStreamItem,\n  type ProviderTool,\n  type LanguageModel,\n} from \"@nodetool-ai\u002Fruntime\";\n\nexport class MyProvider extends BaseProvider {\n  private apiKey: string;\n\n  constructor(kwargs: Record\u003Cstring, unknown> = {}) {\n    super(\"my_provider\" as ProviderId);\n    this.apiKey = String(kwargs[\"MY_API_KEY\"] ?? process.env.MY_API_KEY ?? \"\");\n  }\n\n  static override requiredSecrets(): string[] {\n    return [\"MY_API_KEY\"];\n  }\n\n  \u002F\u002F Non-streaming: return a single assistant Message.\n  async generateMessage(args: {\n    messages: Message[];\n    model: string;\n    tools?: ProviderTool[];\n  }): Promise\u003CMessage> {\n    \u002F\u002F Call your API with args.messages \u002F args.model …\n    return { role: \"assistant\", content: \"response text\" };\n  }\n\n  \u002F\u002F Streaming: yield ProviderStreamItem chunks.\n  async *generateMessages(args: {\n    messages: Message[];\n    model: string;\n    tools?: ProviderTool[];\n  }): AsyncGenerator\u003CProviderStreamItem> {\n    yield { type: \"chunk\", content: \"response text\", done: false };\n  }\n\n  override async getAvailableLanguageModels(): Promise\u003CLanguageModel[]> {\n    return [{ id: \"my-model\", name: \"My Model\", provider: \"my_provider\" }];\n  }\n}\n```\n\nRegister it with `registerProvider(\"my_provider\", MyProvider)` from\n`@nodetool-ai\u002Fruntime`.\n","You help users configure AI model providers and select the right models for their tasks.\n\n# In the product\n\nThe agent has no shell. Configure keys and pick models with these tools:\n\n1. `list_secrets` shows which credentials are configured. It never returns a\n   value.\n2. `request_secret {key, reason, help_url}` opens a dialog where the user types\n   a missing key. The value is saved from the dialog and never passes through\n   the agent. Never ask the user to paste a key into chat. When the run has no\n   interactive client, name the missing key and point the user to\n   Settings > Secrets.\n3. `find_model {capability}` returns a real `{provider, model_id}` and a `ref`\n   to assign verbatim to a node's model property. Call it before adding any\n   generic AI node. Never write a model id from memory, including the examples\n   below.\n4. `list_provider_models {provider}` lists one provider's language models.\n\nInside a code action the same calls are `nodetool.secrets` \u002F\n`nodetool.settings` (load `api-settings`) and `nodetool.models` (load\n`api-models`).\n\nThe `nodetool` provider is NodeTool's managed service. Its models run on\nplatform keys and draw from the user's credit balance, so it needs no key of\nthe user's own. Every other provider bills the user's own account.\n\n# Provider Overview\n\n| Provider | Type | Key Env Var | Models |\n|----------|------|-------------|--------|\n| **OpenAI** | Cloud | `OPENAI_API_KEY` | GPT, GPT-Image, TTS, Whisper |\n| **Anthropic** | Cloud | `ANTHROPIC_API_KEY` | Claude Opus, Sonnet, Haiku |\n| **Gemini (Google)** | Cloud | `GEMINI_API_KEY` | Gemini, Veo, Nano Banana |\n| **xAI** | Cloud | `XAI_API_KEY` | Grok |\n| **Ollama** | Local | `OLLAMA_API_URL` | Qwen, Llama 3, Mistral, any GGUF. Separate install, must be running |\n| **HuggingFace** | Cloud | `HF_TOKEN` | Hosted Inference Providers |\n| **HuggingFace local** (`huggingface-local`) | Local | — | Diffusers\u002FTransformers models via the HuggingFace Python pack |\n| **NodeTool** (`nodetool`) | Cloud, managed | — (credits) | Curated models |\n| **FAL** (`fal_ai`) | Cloud | `FAL_API_KEY` | Fast image\u002Fvideo generation |\n| **KIE** (`kie`) | Cloud | `KIE_API_KEY` | Image, video and music models from many vendors |\n| **ElevenLabs** (`elevenlabs`) | Cloud | `ELEVENLABS_API_KEY` | TTS, voice, sound effects |\n| **Replicate** | Cloud | `REPLICATE_API_TOKEN` | Community models |\n| **Comfy Cloud** | Cloud | `COMFY_API_KEY` | No model list; runs a ComfyUI workflow on Comfy's GPUs via `lib.comfy.RunWorkflowOnCloud` |\n| **vLLM** | Local | `VLLM_BASE_URL` (required) | Self-hosted, OpenAI-compatible |\n| **LM Studio** | Local | `LMSTUDIO_API_URL` | Models loaded in LM Studio's local server |\n| **llama.cpp server** (`llama_cpp`) | Local | `LLAMA_CPP_URL` (required) | A `llama-server` you run |\n| **llama.cpp local** (`node_llama_cpp`) | Local | — | GGUF files in-process; runtime from Package Manager → Engines & tools |\n| **MLX** | Local | — | MLX Python pack, Apple Silicon only |\n\nOther registered chat providers (any of these is valid for the CLI's `-p\u002F--provider`):\n`groq`, `mistral`, `deepseek`, `moonshot`, `sarvam` (`SARVAM_API_KEY`, also\nBulbul TTS and Saaras ASR), `minimax`, `zai`, `cerebras`, `alibaba`\n(`DASHSCOPE_API_KEY`, Qwen via Alibaba Cloud Model Studio), `together`,\n`openrouter`, `requesty`, `opper`, `cheaperinference`, `api_route`, `runcrate`, `standardcompute`, `uberllm`, `codex`, `claude_agent_sdk`, `lmstudio`,\n`azure_openai` (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, models from\n`AZURE_OPENAI_DEPLOYMENTS`), `bedrock` (`BEDROCK_ACCESS_KEY_ID`,\n`BEDROCK_SECRET_ACCESS_KEY`, `BEDROCK_REGION`).\n\n# From a shell (self-host)\n\nThese run in the user's terminal, not in the product. `nodetool models\nproviders` shows configured providers, `nodetool models recommended` the curated\nlist, and `nodetool models by-provider \u003Cprovider>` one provider's ids.\n\n```bash\n# Via CLI (encrypted storage)\nnodetool secrets store OPENAI_API_KEY\nnodetool secrets store ANTHROPIC_API_KEY\nnodetool secrets store GEMINI_API_KEY\nnodetool secrets store HF_TOKEN\nnodetool secrets store FAL_API_KEY\nnodetool secrets store REPLICATE_API_TOKEN\nnodetool secrets store COMFY_API_KEY\n\n# Via environment variables\nexport OPENAI_API_KEY=sk-...\nexport ANTHROPIC_API_KEY=sk-ant-...\nexport GEMINI_API_KEY=AI...\nexport HF_TOKEN=hf_...\nexport FAL_API_KEY=...\nexport REPLICATE_API_TOKEN=r8_...\nexport COMFY_API_KEY=comfyui-...\nexport OLLAMA_API_URL=http:\u002F\u002Flocalhost:11434\n```\n\n# Model Selection by Task\n\nThe tables name model families as starting points. `find_model` ranks what is\nactually available on this install and returns the exact id.\n\n## whisper.cpp speech recognition\n\nBoth providers are local-only and unavailable on the cloud profile.\n`whisper_cpp` runs speech recognition in the backend through the optional\n`@fugood\u002Fwhisper.node@1.1.3` runtime package. Install **whisper.cpp** from the\nPackage Manager, then download a GGML model from Models. It serves\n`nodetool.text.AutomaticSpeechRecognition` and `whisper_cpp.LiveTranscription`.\nUse the absolute model path returned by discovery unchanged as the ASR model id.\n\nModels use the Hugging Face hub cache (see HuggingFace models below for how its\nlocation resolves). `WHISPER_CPP_MODELS_DIR` adds another directory.\n`WHISPER_CPP_GPU_BACKEND` accepts `auto`, `metal`, `cuda`, `vulkan`, or `cpu`.\n`auto` uses Metal on macOS. On Windows and Linux it tries the CUDA build, then\nthe Vulkan build, then the default CPU build. Restart the backend after\nchanging the backend setting. Live transcription accepts base64 PCM16\nmono chunks with an optional `content_metadata.sample_rate` and uses Silero\nVAD when installed, otherwise fixed windows.\n\n`whisper_cpp_server` calls a user-run `whisper-server`. Set\n`WHISPER_CPP_SERVER_URL` to its base URL. The model id is `default`, representing\nthe model loaded by that server. It requires no native runtime package.\n\n## Language \u002F Chat\n\n| Need | Family | Provider | Notes |\n|------|--------|----------|-------|\n| Best quality | GPT flagship, Claude Opus or Sonnet | OpenAI, Anthropic | Highest capability |\n| Good balance | GPT mini, Gemini Flash | OpenAI, Gemini | Fast + cheap |\n| Local\u002Fprivate | Llama, Qwen | Ollama | No data leaves machine |\n| Lightweight local | Small Llama, Mistral 7B | Ollama | Low memory |\n| Code | Claude Sonnet, GPT flagship | Anthropic, OpenAI | Best for coding |\n\n## Image Generation\n\n| Need | Model | Provider | Notes |\n|------|-------|----------|-------|\n| Best quality | FLUX.2 Dev | HuggingFace, FAL | State-of-art |\n| Fast | FLUX Schnell | HuggingFace | Quick iterations |\n| Versatile | SDXL | HuggingFace | Many LoRAs available |\n| API-based | GPT Image 2, Nano Banana | OpenAI, Gemini\u002FKIE | No local GPU needed |\n\n## Video Generation\n\n| Need | Model | Provider |\n|------|-------|----------|\n| Best quality | Sora 2 Pro | OpenAI (KIE) |\n| Fast | Wan 2.6 | KIE |\n| Image-to-video | Kling 2.6 | KIE |\n| Talking avatar | Kling AI Avatar | KIE |\n\n## Speech & Audio\n\n| Need | Model | Provider |\n|------|-------|----------|\n| TTS (quality) | ElevenLabs | ElevenLabs |\n| TTS (fast\u002Ffree) | Kokoro | HuggingFace pack, MLX pack, Transformers.js |\n| ASR (accuracy) | Whisper Large V3 | HuggingFace pack, whisper.cpp |\n| ASR (fast) | Whisper Turbo | HuggingFace pack, MLX pack, whisper.cpp |\n\n## Embeddings\n\n| Need | Model | Provider |\n|------|-------|----------|\n| General text | text-embedding-3-small | OpenAI |\n| Best quality | text-embedding-3-large | OpenAI |\n| Local\u002Ffree | sentence-transformers | HuggingFace |\n\n# Local Model Setup\n\n## Ollama (Easiest)\n\nOllama is a separate program. NodeTool does not ship or start it, so it must be\nrunning before its models appear. On macOS and Windows, download it from\nollama.com. On Linux:\n\n```bash\n# Install\ncurl -fsSL https:\u002F\u002Follama.com\u002Finstall.sh | sh\n\n# Pull models\nollama pull llama3\nollama pull mistral\nollama pull qwen2\n\n# Verify\nollama list\n\n# NodeTool finds Ollama at 127.0.0.1:11434\n# Override: OLLAMA_API_URL in Settings → Integrations → Local Model Servers,\n# or export OLLAMA_API_URL=http:\u002F\u002Fhost:11434\n```\n\n`OLLAMA_CONTEXT_LENGTH` sets the context window sent with each request.\nUnset, NodeTool uses the model's own value, capped at 32768.\n\n## Python packs (HuggingFace local, MLX, Wan2GP)\n\nLocal Diffusers\u002FTransformers models, MLX models and Wan2GP video run in the\nPython worker, and each needs its pack.\n\n- Desktop app: **Tools → Package Manager → Python nodes**. Installing a pack\n  sets up Python first if it is missing. The HuggingFace pack pulls a PyTorch\n  2.14 build matched to the GPU. MLX is offered only on Apple Silicon Macs.\n  On a Mac, both packs need Apple Silicon and macOS 14 or newer.\n- Without the desktop app: a Python 3.11+ env with\n  `pip install nodetool-core nodetool-huggingface` (plus `nodetool-mlx` on\n  Apple Silicon), then start the server with `NODETOOL_PYTHON` pointing at that\n  env's `python`. `curl -fsSL https:\u002F\u002Fraw.githubusercontent.com\u002Fnodetool-ai\u002Fnodetool\u002Fmain\u002Finstall.sh | bash -s -- -y --pack huggingface`\n  builds that env on Linux and macOS. `--pack` takes `huggingface`, `mlx`, or\n  `wan2gp`. On macOS, always set `NODETOOL_PYTHON` to the env it prints.\n- `NODETOOL_TORCH_DEVICE` (`cuda`, `cuda:1`, `mps`, `cpu`) picks the device.\n- Wan2GP nodes call a Wan2GP server the user runs. `WAN2GP_MCP_URL` (default\n  `http:\u002F\u002F127.0.0.1:7866\u002Fmcp`) or the node's `server_url` points at it.\n\n## HuggingFace models\n\nDownload local models in the Model Manager before running the workflow. A node\nthat downloads on first use can fail on a gated repo or stall a run.\n\nFor gated models:\n1. Accept the license on the model page at huggingface.co\n2. Store `HF_TOKEN` in Settings (or `nodetool secrets store HF_TOKEN`). Model\n   Manager downloads use it\n3. Download the model in the Model Manager\n\nModels go to the Hugging Face hub cache: `HF_HUB_CACHE`, else\n`HUGGINGFACE_HUB_CACHE`, else `$HF_HOME\u002Fhub`, else\n`$XDG_CACHE_HOME\u002Fhuggingface\u002Fhub`, else `~\u002F.cache\u002Fhuggingface\u002Fhub`. The\nserver and the Python worker (through `huggingface_hub`) apply this one order.\nThe desktop app keeps `~\u002F.cache\u002Fhuggingface` when `XDG_CACHE_HOME` is set (as\nin Flatpak) but that cache already holds models.\n\n## llama.cpp\n\nTwo providers. `node_llama_cpp` loads GGUF files in-process after the runtime\nis installed from Package Manager → Engines & tools. `llama_cpp` calls a\n`llama-server` the user runs, at `LLAMA_CPP_URL`. Best for CPU inference and\nquantized models.\n\n`MODEL_FOLDERS` (Settings → Folders) adds folders, such as a ComfyUI `models`\nfolder, that `node_llama_cpp` and `stable_diffusion_cpp` search with their\nsubfolders. llama.cpp lists the GGUF language models in them.\nstable-diffusion.cpp lists `.safetensors` checkpoints that bundle their VAE and\ntext encoders. Separate folders with `;`.\n\n## whisper.cpp and Transformers.js\n\nwhisper.cpp is speech recognition only (see above). Transformers.js runs small\nONNX models in-process after its runtime is installed from Package Manager →\nSoftware. Its cache is `TRANSFORMERS_JS_CACHE_DIR`.\n\n## MLX (Apple Silicon only)\n\nNeeds the MLX pack and macOS 14 or newer. Lower memory usage than standard\nPyTorch.\n\n# Local Inference Performance\n\n| Framework | Throughput | Memory | Hardware |\n|-----------|-----------|--------|----------|\n| **llama.cpp** | Medium | Excellent | CPU, GPU |\n| **MLX** | Good | Excellent | Apple Silicon |\n| **Transformers** | Medium | Good | Any |\n\n# Provider-Agnostic Nodes\n\nThese nodes work with any provider — just select the model:\n\n| Node | Purpose |\n|------|---------|\n| `nodetool.agents.Agent` | Any LLM for chat\u002Freasoning |\n| `nodetool.image.TextToImage` | Any image generation model |\n| `nodetool.image.ImageToImage` | Any image transformation model |\n| `nodetool.video.TextToVideo` | Any video generation model |\n| `nodetool.video.ImageToVideo` | Any image-to-video model |\n| `nodetool.audio.TextToSpeech` | Any TTS model |\n| `nodetool.text.AutomaticSpeechRecognition` | Any ASR model |\n\n# Provider Capabilities\n\n| Capability | OpenAI | Anthropic | Google | Ollama | HF |\n|-----------|--------|-----------|--------|--------|-----|\n| Chat\u002FText | yes | yes | yes | yes | yes |\n| Vision | yes | yes | yes | some | yes |\n| Image Gen | yes (GPT-Image) | no | no | no | yes |\n| Video Gen | no | no | yes (Veo) | no | some |\n| TTS | yes | no | no | no | yes |\n| ASR | yes (Whisper) | no | no | no | yes |\n| Embeddings | yes | no | yes | yes | yes |\n| Tool Calling | yes | yes | yes | some | no |\n\n# Common Pitfalls\n\n- **Wrong key env var name**: Each provider has a specific name (see table above)\n- **Ollama not running**: NodeTool does not start it. Start the Ollama app or `ollama serve`\n- **Gated HF models**: Must accept terms on huggingface.co first\n- **GPU memory**: Large models need 8-24GB VRAM; use quantized versions\n- **PyTorch on the CPU despite a GPU**: the desktop app installs CUDA 12.8 for Turing (RTX 20xx) and newer (driver 570+), CUDA 12.6 for Maxwell to Volta, and CPU for Kepler, pre-RDNA 2 AMD and AMD on Windows. It also falls back to CPU, with a warning, when no GPU build resolves. A manual `pip install torch` on Linux gets CUDA 13.0, which needs driver 580+ and Turing or newer, as does the Docker worker image\n- **No local provider in the model menu**: the Python pack is missing, or `NODETOOL_PYTHON_ON_DEMAND=true` and no workflow has started the worker yet\n- **Rate limits**: Cloud providers have rate limits; implement retries or use local\n- **Model ID mismatch**: Take ids from `find_model` or `list_provider_models`, never from memory. From a shell, `nodetool models by-provider \u003Cprovider>` lists them\n- **Key pasted into chat**: Use `request_secret`. A key in a chat message is stored in the thread\n\n# Repository only: custom provider\n\nThis is TypeScript for contributors working in a NodeTool checkout. The\nin-product agent cannot add a provider. Providers extend `BaseProvider` from `@nodetool-ai\u002Fruntime` (not `@nodetool-ai\u002Fcore`).\nBoth `generateMessage` and `generateMessages` take a single **args object**.\n\n```typescript\nimport {\n  BaseProvider,\n  type ProviderId,\n  type Message,\n  type ProviderStreamItem,\n  type ProviderTool,\n  type LanguageModel,\n} from \"@nodetool-ai\u002Fruntime\";\n\nexport class MyProvider extends BaseProvider {\n  private apiKey: string;\n\n  constructor(kwargs: Record\u003Cstring, unknown> = {}) {\n    super(\"my_provider\" as ProviderId);\n    this.apiKey = String(kwargs[\"MY_API_KEY\"] ?? process.env.MY_API_KEY ?? \"\");\n  }\n\n  static override requiredSecrets(): string[] {\n    return [\"MY_API_KEY\"];\n  }\n\n  \u002F\u002F Non-streaming: return a single assistant Message.\n  async generateMessage(args: {\n    messages: Message[];\n    model: string;\n    tools?: ProviderTool[];\n  }): Promise\u003CMessage> {\n    \u002F\u002F Call your API with args.messages \u002F args.model …\n    return { role: \"assistant\", content: \"response text\" };\n  }\n\n  \u002F\u002F Streaming: yield ProviderStreamItem chunks.\n  async *generateMessages(args: {\n    messages: Message[];\n    model: string;\n    tools?: ProviderTool[];\n  }): AsyncGenerator\u003CProviderStreamItem> {\n    yield { type: \"chunk\", content: \"response text\", done: false };\n  }\n\n  override async getAvailableLanguageModels(): Promise\u003CLanguageModel[]> {\n    return [{ id: \"my-model\", name: \"My Model\", provider: \"my_provider\" }];\n  }\n}\n```\n\nRegister it with `registerProvider(\"my_provider\", MyProvider)` from\n`@nodetool-ai\u002Fruntime`.\n",{"name":5,"featured":49,"description":7},true,"main","d5860feab9249fd6eb8e585f7558463227520792","packages\u002Fsystem-skills\u002Fnodetool-model-provider-config",91,"https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool","https:\u002F\u002Fnodetool.ai\u002F","https:\u002F\u002Fgithub.com\u002Fnodetool-ai","No-code AI development","2026-11-15 03:27:49",{"source":8,"skillName":5,"commands":60,"prompt":70,"zipAvailable":49},[61,64,67],{"tool":62,"command":63},"npx","npx skills add nodetool-ai\u002Fnodetool --skill nodetool-model-provider-config",{"tool":65,"command":66},"bunx","bunx skills add nodetool-ai\u002Fnodetool --skill nodetool-model-provider-config",{"tool":68,"command":69},"pnpm","pnpm dlx skills add nodetool-ai\u002Fnodetool --skill nodetool-model-provider-config","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fpackages\u002Fsystem-skills\u002Fnodetool-model-provider-config\u002FSKILL.md",[],[73,82,117,144,159,175],{"skillNo":74,"name":75,"nameZh":6,"description":76,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":81},"01M742MX3Y2ZYK884ZXTRFKB9P","humanizer","Rewrite AI-sounding text so it reads like the writer without changing what it says.\nUse when editing or reviewing prose for AI tells: not-X-but-Y contrasts, one-line\nclosers, staged openers, forced triads, dashes everywhere, inflated claims, sales\nlanguage, stock AI words, bold labels, or filler.","MIT","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fhumanizer\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":83,"name":28,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":95,"publishedAt":44},"01M6BY4D93R0Q9400ZHJ04DXP1","Use the NodeTool MCP server to build, validate, run, and debug visual AI workflows, and to work with assets, collections, and jobs. Use when the user mentions NodeTool or asks for a workflow graph, a node, or a media generation run on their NodeTool server.","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fplugins\u002Fchatgpt\u002Fnodetool\u002Fskills\u002Fnodetool\u002FSKILL.md",[87,90],{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":91,"categoryName":92,"taxonomyType":93,"taxonomyTypeDesc":94},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[28,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116],"MCP","Media","workflows","when","generation","media generation","with","visual","use","collections","for","node","work","debug","graph","validate","assets","run","server","mcp server","workflow",{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":124,"publishedAt":44},"01M6BY4AEB05ZPBRVD078ATR2H","wan-2-6-prompting","Prompt Alibaba's Wan 2.6 video — text-to-video with a global style line plus timing-bracketed shots, and image-to-video that describes only what changes over time. Covers the shot-mode switch per provider, the prompt budget, negative prompts, motion intensity wording, audio input and prompt expansion. Use whenever the model id names a Wan 2.6 video route (alibaba\u002Fwan-2.6\u002Ftext-to-video, alibaba\u002Fwan-2.6\u002Fimage-to-video, wan-video\u002Fwan-2.6-t2v, wan-video\u002Fwan-2.6-i2v, wan-video\u002Fwan2.6-i2v-flash) on generate_video, animate_image or a TextToVideo node. Not for Wan 2.5 or 2.7, or the Wan 2.6 image routes.","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fpackages\u002Fsystem-skills\u002Fwan-2-6-prompting\u002FSKILL.md",[123],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[125,126,127,128,129,130,131,107,132,133,134,135,136,137,138,139,140,141,26,142,143],"prompting","wan 2 6 prompting","line","expansion","input","image-to-video","text-to-video","route","only","mode","time","flash","Budget","generate","text","image","animate","Audio","Video",{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":152,"publishedAt":44},"01M6BY47T4748ZG5ZMTAR97ZWZ","video-workflow","Build a reusable NodeTool video workflow for batches or repeated inputs. For a single finished piece, use a storyboard skill.","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fpackages\u002Fsystem-skills\u002Fvideo-workflow\u002FSKILL.md",[150,151],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},[116,153,143,154,155,156,157,104,158,28,106],"video workflow","reusable","storyboard","build","batches","single",{"skillNo":160,"name":161,"nameZh":6,"description":162,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":163,"categories":164,"keywords":166,"publishedAt":44},"01M6BY4587N2SYN315SXGEMZC3","video-clone","Analyze a reference video and recreate its shot structure with a new product or cast in NodeTool.","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fpackages\u002Fsystem-skills\u002Fvideo-clone\u002FSKILL.md",[165],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[167,168,143,102,169,170,28,171,172,173,174],"clone","video clone","cast","new","analyze","product","structure","reference",{"skillNo":176,"name":177,"nameZh":6,"description":178,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":179,"categories":180,"keywords":182,"publishedAt":44},"01M6BY4257C2KQYYMD9EK6DK8E","video-audio-continuity","Keep sound continuous across a multi-scene piece cut from generated video — why one clip per scene hard-cuts the audio at every boundary, when to write all the scenes into a single generation instead, the sound-design vocabulary that survives a provider's copyright filter, and how to check the length, the aspect ratio and the mix of what came back. Use when a piece has more than one scene and the video model writes its own audio (Seedance 2, Veo 3, MiniMax H3, Kling 2.6 and later), or when a cut's sound drops out at the shot changes.","https:\u002F\u002Fgithub.com\u002Fnodetool-ai\u002Fnodetool\u002Fblob\u002Fmain\u002Fpackages\u002Fsystem-skills\u002Fvideo-audio-continuity\u002FSKILL.md",[181],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[183,143,142,184,185,186,187,188,189,190,191,192,193,99,194,195,26,196,197,198,199],"video audio continuity","continuity","write","why","back","veo 3","at","cut","later","across","filter","generated","how to","mix","design","boundary","how"]