nodetool-model-provider-config
Configure NodeTool model providers and API keys (list_secrets, request_secret), pick a model with find_model, or set up local Ollama, llama.cpp and Hugging Face models.
- License 许可
- AGPL-3.0
- 最近更新
- 2026-11-15
- Stars
- ★ 635
- Forks
- 91
01 前言 Front Matter
- name
- nodetool-model-provider-config
- featured
- true
- description
- Configure NodeTool model providers and API keys (list_secrets, request_secret), pick a model with find_model, or set up local Ollama, llama.cpp and Hugging Face models.
03 SKILL.md 正文
You help users configure AI model providers and select the right models for their tasks.
In the product
The agent has no shell. Configure keys and pick models with these tools:
list_secretsshows which credentials are configured. It never returns a value.request_secret {key, reason, help_url}opens a dialog where the user types a missing key. The value is saved from the dialog and never passes through the agent. Never ask the user to paste a key into chat. When the run has no interactive client, name the missing key and point the user to Settings > Secrets.find_model {capability}returns a real{provider, model_id}and arefto assign verbatim to a node's model property. Call it before adding any generic AI node. Never write a model id from memory, including the examples below.list_provider_models {provider}lists one provider's language models.
Inside a code action the same calls are nodetool.secrets /
nodetool.settings (load api-settings) and nodetool.models (load
api-models).
The nodetool provider is NodeTool's managed service. Its models run on
platform keys and draw from the user's credit balance, so it needs no key of
the user's own. Every other provider bills the user's own account.
Provider Overview
| Provider | Type | Key Env Var | Models |
|---|---|---|---|
| OpenAI | Cloud | OPENAI_API_KEY |
GPT, GPT-Image, TTS, Whisper |
| Anthropic | Cloud | ANTHROPIC_API_KEY |
Claude Opus, Sonnet, Haiku |
| Gemini (Google) | Cloud | GEMINI_API_KEY |
Gemini, Veo, Nano Banana |
| xAI | Cloud | XAI_API_KEY |
Grok |
| Ollama | Local | OLLAMA_API_URL |
Qwen, Llama 3, Mistral, any GGUF. Separate install, must be running |
| HuggingFace | Cloud | HF_TOKEN |
Hosted Inference Providers |
HuggingFace local (huggingface-local) |
Local | — | Diffusers/Transformers models via the HuggingFace Python pack |
NodeTool (nodetool) |
Cloud, managed | — (credits) | Curated models |
FAL (fal_ai) |
Cloud | FAL_API_KEY |
Fast image/video generation |
KIE (kie) |
Cloud | KIE_API_KEY |
Image, video and music models from many vendors |
ElevenLabs (elevenlabs) |
Cloud | ELEVENLABS_API_KEY |
TTS, voice, sound effects |
| Replicate | Cloud | REPLICATE_API_TOKEN |
Community models |
| Comfy Cloud | Cloud | COMFY_API_KEY |
No model list; runs a ComfyUI workflow on Comfy's GPUs via lib.comfy.RunWorkflowOnCloud |
| vLLM | Local | VLLM_BASE_URL (required) |
Self-hosted, OpenAI-compatible |
| LM Studio | Local | LMSTUDIO_API_URL |
Models loaded in LM Studio's local server |
llama.cpp server (llama_cpp) |
Local | LLAMA_CPP_URL (required) |
A llama-server you run |
llama.cpp local (node_llama_cpp) |
Local | — | GGUF files in-process; runtime from Package Manager → Engines & tools |
| MLX | Local | — | MLX Python pack, Apple Silicon only |
Other registered chat providers (any of these is valid for the CLI's -p/--provider):
groq, mistral, deepseek, moonshot, sarvam (SARVAM_API_KEY, also
Bulbul TTS and Saaras ASR), minimax, zai, cerebras, alibaba
(DASHSCOPE_API_KEY, Qwen via Alibaba Cloud Model Studio), together,
openrouter, requesty, opper, cheaperinference, api_route, runcrate, standardcompute, uberllm, codex, claude_agent_sdk, lmstudio,
azure_openai (AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, models from
AZURE_OPENAI_DEPLOYMENTS), bedrock (BEDROCK_ACCESS_KEY_ID,
BEDROCK_SECRET_ACCESS_KEY, BEDROCK_REGION).
From a shell (self-host)
These run in the user's terminal, not in the product. nodetool models providers shows configured providers, nodetool models recommended the curated
list, and nodetool models by-provider <provider> one provider's ids.
# Via CLI (encrypted storage)
nodetool secrets store OPENAI_API_KEY
nodetool secrets store ANTHROPIC_API_KEY
nodetool secrets store GEMINI_API_KEY
nodetool secrets store HF_TOKEN
nodetool secrets store FAL_API_KEY
nodetool secrets store REPLICATE_API_TOKEN
nodetool secrets store COMFY_API_KEY
# Via environment variables
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export GEMINI_API_KEY=AI...
export HF_TOKEN=hf_...
export FAL_API_KEY=...
export REPLICATE_API_TOKEN=r8_...
export COMFY_API_KEY=comfyui-...
export OLLAMA_API_URL=http://localhost:11434
Model Selection by Task
The tables name model families as starting points. find_model ranks what is
actually available on this install and returns the exact id.
whisper.cpp speech recognition
Both providers are local-only and unavailable on the cloud profile.
whisper_cpp runs speech recognition in the backend through the optional
@fugood/whisper.node@1.1.3 runtime package. Install whisper.cpp from the
Package Manager, then download a GGML model from Models. It serves
nodetool.text.AutomaticSpeechRecognition and whisper_cpp.LiveTranscription.
Use the absolute model path returned by discovery unchanged as the ASR model id.
Models use the Hugging Face hub cache (see HuggingFace models below for how its
location resolves). WHISPER_CPP_MODELS_DIR adds another directory.
WHISPER_CPP_GPU_BACKEND accepts auto, metal, cuda, vulkan, or cpu.
auto uses Metal on macOS. On Windows and Linux it tries the CUDA build, then
the Vulkan build, then the default CPU build. Restart the backend after
changing the backend setting. Live transcription accepts base64 PCM16
mono chunks with an optional content_metadata.sample_rate and uses Silero
VAD when installed, otherwise fixed windows.
whisper_cpp_server calls a user-run whisper-server. Set
WHISPER_CPP_SERVER_URL to its base URL. The model id is default, representing
the model loaded by that server. It requires no native runtime package.
Language / Chat
| Need | Family | Provider | Notes |
|---|---|---|---|
| Best quality | GPT flagship, Claude Opus or Sonnet | OpenAI, Anthropic | Highest capability |
| Good balance | GPT mini, Gemini Flash | OpenAI, Gemini | Fast + cheap |
| Local/private | Llama, Qwen | Ollama | No data leaves machine |
| Lightweight local | Small Llama, Mistral 7B | Ollama | Low memory |
| Code | Claude Sonnet, GPT flagship | Anthropic, OpenAI | Best for coding |
Image Generation
| Need | Model | Provider | Notes |
|---|---|---|---|
| Best quality | FLUX.2 Dev | HuggingFace, FAL | State-of-art |
| Fast | FLUX Schnell | HuggingFace | Quick iterations |
| Versatile | SDXL | HuggingFace | Many LoRAs available |
| API-based | GPT Image 2, Nano Banana | OpenAI, Gemini/KIE | No local GPU needed |
Video Generation
| Need | Model | Provider |
|---|---|---|
| Best quality | Sora 2 Pro | OpenAI (KIE) |
| Fast | Wan 2.6 | KIE |
| Image-to-video | Kling 2.6 | KIE |
| Talking avatar | Kling AI Avatar | KIE |
Speech & Audio
| Need | Model | Provider |
|---|---|---|
| TTS (quality) | ElevenLabs | ElevenLabs |
| TTS (fast/free) | Kokoro | HuggingFace pack, MLX pack, Transformers.js |
| ASR (accuracy) | Whisper Large V3 | HuggingFace pack, whisper.cpp |
| ASR (fast) | Whisper Turbo | HuggingFace pack, MLX pack, whisper.cpp |
Embeddings
| Need | Model | Provider |
|---|---|---|
| General text | text-embedding-3-small | OpenAI |
| Best quality | text-embedding-3-large | OpenAI |
| Local/free | sentence-transformers | HuggingFace |
Local Model Setup
Ollama (Easiest)
Ollama is a separate program. NodeTool does not ship or start it, so it must be running before its models appear. On macOS and Windows, download it from ollama.com. On Linux:
# Install
curl -fsSL https://ollama.com/install.sh | sh
# Pull models
ollama pull llama3
ollama pull mistral
ollama pull qwen2
# Verify
ollama list
# NodeTool finds Ollama at 127.0.0.1:11434
# Override: OLLAMA_API_URL in Settings → Integrations → Local Model Servers,
# or export OLLAMA_API_URL=http://host:11434
OLLAMA_CONTEXT_LENGTH sets the context window sent with each request.
Unset, NodeTool uses the model's own value, capped at 32768.
Python packs (HuggingFace local, MLX, Wan2GP)
Local Diffusers/Transformers models, MLX models and Wan2GP video run in the Python worker, and each needs its pack.
- Desktop app: Tools → Package Manager → Python nodes. Installing a pack sets up Python first if it is missing. The HuggingFace pack pulls a PyTorch 2.14 build matched to the GPU. MLX is offered only on Apple Silicon Macs. On a Mac, both packs need Apple Silicon and macOS 14 or newer.
- Without the desktop app: a Python 3.11+ env with
pip install nodetool-core nodetool-huggingface(plusnodetool-mlxon Apple Silicon), then start the server withNODETOOL_PYTHONpointing at that env'spython.curl -fsSL https://raw.githubusercontent.com/nodetool-ai/nodetool/main/install.sh | bash -s -- -y --pack huggingfacebuilds that env on Linux and macOS.--packtakeshuggingface,mlx, orwan2gp. On macOS, always setNODETOOL_PYTHONto the env it prints. NODETOOL_TORCH_DEVICE(cuda,cuda:1,mps,cpu) picks the device.- Wan2GP nodes call a Wan2GP server the user runs.
WAN2GP_MCP_URL(defaulthttp://127.0.0.1:7866/mcp) or the node'sserver_urlpoints at it.
HuggingFace models
Download local models in the Model Manager before running the workflow. A node that downloads on first use can fail on a gated repo or stall a run.
For gated models:
- Accept the license on the model page at huggingface.co
- Store
HF_TOKENin Settings (ornodetool secrets store HF_TOKEN). Model Manager downloads use it - Download the model in the Model Manager
Models go to the Hugging Face hub cache: HF_HUB_CACHE, else
HUGGINGFACE_HUB_CACHE, else $HF_HOME/hub, else
$XDG_CACHE_HOME/huggingface/hub, else ~/.cache/huggingface/hub. The
server and the Python worker (through huggingface_hub) apply this one order.
The desktop app keeps ~/.cache/huggingface when XDG_CACHE_HOME is set (as
in Flatpak) but that cache already holds models.
llama.cpp
Two providers. node_llama_cpp loads GGUF files in-process after the runtime
is installed from Package Manager → Engines & tools. llama_cpp calls a
llama-server the user runs, at LLAMA_CPP_URL. Best for CPU inference and
quantized models.
MODEL_FOLDERS (Settings → Folders) adds folders, such as a ComfyUI models
folder, that node_llama_cpp and stable_diffusion_cpp search with their
subfolders. llama.cpp lists the GGUF language models in them.
stable-diffusion.cpp lists .safetensors checkpoints that bundle their VAE and
text encoders. Separate folders with ;.
whisper.cpp and Transformers.js
whisper.cpp is speech recognition only (see above). Transformers.js runs small
ONNX models in-process after its runtime is installed from Package Manager →
Software. Its cache is TRANSFORMERS_JS_CACHE_DIR.
MLX (Apple Silicon only)
Needs the MLX pack and macOS 14 or newer. Lower memory usage than standard PyTorch.
Local Inference Performance
| Framework | Throughput | Memory | Hardware |
|---|---|---|---|
| llama.cpp | Medium | Excellent | CPU, GPU |
| MLX | Good | Excellent | Apple Silicon |
| Transformers | Medium | Good | Any |
Provider-Agnostic Nodes
These nodes work with any provider — just select the model:
| Node | Purpose |
|---|---|
nodetool.agents.Agent |
Any LLM for chat/reasoning |
nodetool.image.TextToImage |
Any image generation model |
nodetool.image.ImageToImage |
Any image transformation model |
nodetool.video.TextToVideo |
Any video generation model |
nodetool.video.ImageToVideo |
Any image-to-video model |
nodetool.audio.TextToSpeech |
Any TTS model |
nodetool.text.AutomaticSpeechRecognition |
Any ASR model |
Provider Capabilities
| Capability | OpenAI | Anthropic | Ollama | HF | |
|---|---|---|---|---|---|
| Chat/Text | yes | yes | yes | yes | yes |
| Vision | yes | yes | yes | some | yes |
| Image Gen | yes (GPT-Image) | no | no | no | yes |
| Video Gen | no | no | yes (Veo) | no | some |
| TTS | yes | no | no | no | yes |
| ASR | yes (Whisper) | no | no | no | yes |
| Embeddings | yes | no | yes | yes | yes |
| Tool Calling | yes | yes | yes | some | no |
Common Pitfalls
- Wrong key env var name: Each provider has a specific name (see table above)
- Ollama not running: NodeTool does not start it. Start the Ollama app or
ollama serve - Gated HF models: Must accept terms on huggingface.co first
- GPU memory: Large models need 8-24GB VRAM; use quantized versions
- PyTorch on the CPU despite a GPU: the desktop app installs CUDA 12.8 for Turing (RTX 20xx) and newer (driver 570+), CUDA 12.6 for Maxwell to Volta, and CPU for Kepler, pre-RDNA 2 AMD and AMD on Windows. It also falls back to CPU, with a warning, when no GPU build resolves. A manual
pip install torchon Linux gets CUDA 13.0, which needs driver 580+ and Turing or newer, as does the Docker worker image - No local provider in the model menu: the Python pack is missing, or
NODETOOL_PYTHON_ON_DEMAND=trueand no workflow has started the worker yet - Rate limits: Cloud providers have rate limits; implement retries or use local
- Model ID mismatch: Take ids from
find_modelorlist_provider_models, never from memory. From a shell,nodetool models by-provider <provider>lists them - Key pasted into chat: Use
request_secret. A key in a chat message is stored in the thread
Repository only: custom provider
This is TypeScript for contributors working in a NodeTool checkout. The
in-product agent cannot add a provider. Providers extend BaseProvider from @nodetool-ai/runtime (not @nodetool-ai/core).
Both generateMessage and generateMessages take a single args object.
import {
BaseProvider,
type ProviderId,
type Message,
type ProviderStreamItem,
type ProviderTool,
type LanguageModel,
} from "@nodetool-ai/runtime";
export class MyProvider extends BaseProvider {
private apiKey: string;
constructor(kwargs: Record<string, unknown> = {}) {
super("my_provider" as ProviderId);
this.apiKey = String(kwargs["MY_API_KEY"] ?? process.env.MY_API_KEY ?? "");
}
static override requiredSecrets(): string[] {
return ["MY_API_KEY"];
}
// Non-streaming: return a single assistant Message.
async generateMessage(args: {
messages: Message[];
model: string;
tools?: ProviderTool[];
}): Promise<Message> {
// Call your API with args.messages / args.model …
return { role: "assistant", content: "response text" };
}
// Streaming: yield ProviderStreamItem chunks.
async *generateMessages(args: {
messages: Message[];
model: string;
tools?: ProviderTool[];
}): AsyncGenerator<ProviderStreamItem> {
yield { type: "chunk", content: "response text", done: false };
}
override async getAvailableLanguageModels(): Promise<LanguageModel[]> {
return [{ id: "my-model", name: "My Model", provider: "my_provider" }];
}
}
Register it with registerProvider("my_provider", MyProvider) from
@nodetool-ai/runtime.
04 同仓库更多 Skills
humanizer
Rewrite AI-sounding text so it reads like the writer without changing what it says. Use when editing or reviewing prose for AI tells: not-X-but-Y contrasts, one-line closers, staged openers, forced triads, dashes everywhere, inflated claims, sales language, stock AI words, bold labels, or filler.
nodetool
Use the NodeTool MCP server to build, validate, run, and debug visual AI workflows, and to work with assets, collections, and jobs. Use when the user mentions NodeTool or asks for a workflow graph, a node, or a media generation run on their NodeTool server.
wan-2-6-prompting
Prompt Alibaba's Wan 2.6 video — text-to-video with a global style line plus timing-bracketed shots, and image-to-video that describes only what changes over time. Covers the shot-mode switch per provider, the prompt budget, negative prompts, motion intensity wording, audio input and prompt expansion. Use whenever the model id names a Wan 2.6 video route (alibaba/wan-2.6/text-to-video, alibaba/wan-2.6/image-to-video, wan-video/wan-2.6-t2v, wan-video/wan-2.6-i2v, wan-video/wan2.6-i2v-flash) on generate_video, animate_image or a TextToVideo node. Not for Wan 2.5 or 2.7, or the Wan 2.6 image routes.
video-workflow
Build a reusable NodeTool video workflow for batches or repeated inputs. For a single finished piece, use a storyboard skill.
video-clone
Analyze a reference video and recreate its shot structure with a new product or cast in NodeTool.
video-audio-continuity
Keep sound continuous across a multi-scene piece cut from generated video — why one clip per scene hard-cuts the audio at every boundary, when to write all the scenes into a single generation instead, the sound-design vocabulary that survives a provider's copyright filter, and how to check the length, the aspect ratio and the mix of what came back. Use when a piece has more than one scene and the video model writes its own audio (Seedance 2, Veo 3, MiniMax H3, Kling 2.6 and later), or when a cut's sound drops out at the shot changes.