[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XFM16RV432VJS9A9D08V29":3,"skill-same:01M6XFM16RV432VJS9A9D08V29":74},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":54,"commitSha":55,"skillDirectory":56,"forksCount":14,"repositoryHtmlUrl":57,"repositoryHomepageUrl":6,"ownerHtmlUrl":58,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":59,"install":60,"resources":73},"01M6XFM16RV432VJS9A9D08V29","fireworks-ai-inference",null,"Fast inference and fine-tuning platform with serverless and on-demand GPU deployments. OpenAI-compatible API for chat completions, embeddings, function calling, vision, and structured output. Supports SFT, DPO, and RL fine-tuning. SOC2 + HIPAA compliant.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fcloud-compute\u002Ffireworks-ai\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"fireworks ai inference","fireworks ai","inference","deployments","On-Demand GPU","for","openai-compatible","synthetic","structured output","platform","calling","vision","function calling","api","soc2","tuning","Fast Inference","Fine-Tuning","embeddings","openai","2026-11-10 10:12:30",false,"---\nname: fireworks-ai-inference\ndescription: Fast inference and fine-tuning platform with serverless and on-demand GPU deployments. OpenAI-compatible API for chat completions, embeddings, function calling, vision, and structured output. Supports SFT, DPO, and RL fine-tuning. SOC2 + HIPAA compliant.\ncategory: cloud-compute\nversion: 1.0.0\nauthor: Synthetic Sciences\nlicense: MIT\ntags: [Inference, Fireworks AI, Fine-Tuning, Serverless, On-Demand GPU, OpenAI-Compatible]\ndependencies: [fireworks-ai, openai]\n---\n\n# Fireworks AI -- Fast Inference & Fine-Tuning\n\nFastest open-model inference platform with serverless and on-demand GPU deployments, OpenAI-compatible API, and built-in fine-tuning (SFT, DPO, RL).\n\n## When to Use Fireworks AI\n\n**Use Fireworks AI when:**\n- Need fast serverless inference for open-source models (Llama, Qwen, DeepSeek, Mixtral)\n- Want OpenAI SDK drop-in replacement with open models\n- Need fine-tuning without managing infrastructure (SFT, DPO, RL)\n- Require structured output \u002F JSON mode \u002F function calling with open models\n- Need dedicated GPU deployments with predictable latency\n- Require SOC2 or HIPAA compliance\n- Want prompt caching and batch inference for cost savings\n\n**Use alternatives instead:**\n\n| Need | Use Instead |\n|------|-------------|\n| Self-hosted inference (full control) | **vLLM**, **TensorRT-LLM** |\n| Cheapest serverless inference | **Groq** (free tier), **Together AI** |\n| Managed LoRA fine-tuning (no infra) | **Tinker** |\n| Closed-model APIs (GPT-4, Claude) | **OpenAI**, **Anthropic** direct |\n| GPU instances with SSH access | **Lambda Labs**, **RunPod** |\n| Multi-cloud orchestration | **SkyPilot** |\n\n\n## Credential Setup\n\nCredentials are auto-injected by openscience when connected via the dashboard.\n\n```bash\n# Verify credentials\n[ -n \"$FIREWORKS_API_KEY\" ] && echo \"FIREWORKS_API_KEY set\" || echo \"NOT SET\"\n```\n\nIf not set: connect Fireworks AI at https:\u002F\u002Fapp.syntheticsciences.ai -> Services, then restart openscience.\n\n## Quick Start\n\n### Install\n\n```python\npip install fireworks-ai openai\n```\n\n### Set API key\n\n```python\nimport os\nos.environ[\"FIREWORKS_API_KEY\"] = \"fw_...\"  # from https:\u002F\u002Ffireworks.ai\u002Fapi-keys\n```\n\n### Basic chat completion\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\",\n    api_key=os.environ[\"FIREWORKS_API_KEY\"],\n)\n\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain gradient descent in two sentences.\"},\n    ],\n    max_tokens=256,\n    temperature=0.7,\n)\nprint(response.choices[0].message.content)\n```\n\n## Inference\n\n### Chat completions\n\n**Endpoint:** `POST https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\u002Fchat\u002Fcompletions`\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\",\n    api_key=os.environ[\"FIREWORKS_API_KEY\"],\n)\n\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[\n        {\"role\": \"user\", \"content\": \"Write a Python quicksort function.\"},\n    ],\n    max_tokens=512,\n    temperature=0.0,\n)\nprint(response.choices[0].message.content)\n```\n\n### Streaming\n\n```python\nstream = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain transformers.\"}],\n    stream=True,\n    max_tokens=512,\n)\nfor chunk in stream:\n    delta = chunk.choices[0].delta.content\n    if delta:\n        print(delta, end=\"\", flush=True)\n```\n\n### Function calling \u002F tool use\n\n```python\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_weather\",\n            \"description\": \"Get current weather for a location\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"location\": {\"type\": \"string\", \"description\": \"City name\"},\n                    \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n                },\n                \"required\": [\"location\"],\n            },\n        },\n    }\n]\n\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"What is the weather in San Francisco?\"}],\n    tools=tools,\n    tool_choice=\"auto\",\n)\ntool_call = response.choices[0].message.tool_calls[0]\nprint(tool_call.function.name, tool_call.function.arguments)\n```\n\n### Structured output \u002F JSON mode\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"List 3 planets with mass and diameter.\"}],\n    response_format={\"type\": \"json_object\"},\n    max_tokens=512,\n)\nimport json\ndata = json.loads(response.choices[0].message.content)\n```\n\nFor strict schema enforcement, use `response_format` with a JSON schema:\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Extract name and age from: John is 30.\"}],\n    response_format={\n        \"type\": \"json_object\",\n        \"schema\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"name\": {\"type\": \"string\"},\n                \"age\": {\"type\": \"integer\"},\n            },\n            \"required\": [\"name\", \"age\"],\n        },\n    },\n)\n```\n\n### Vision (multimodal)\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p2-11b-vision-instruct\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\"url\": \"https:\u002F\u002Fexample.com\u002Fphoto.jpg\"},\n                },\n            ],\n        }\n    ],\n    max_tokens=256,\n)\nprint(response.choices[0].message.content)\n```\n\n## Fine-Tuning\n\nFireworks supports three fine-tuning methods: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Fine-Tuning (RL). All use LoRA adapters by default.\n\n### SFT -- Supervised Fine-Tuning\n\n**Data format (JSONL):** Each line is a conversation with `messages` array:\n\n```json\n{\"messages\": [{\"role\": \"system\", \"content\": \"You are a coding assistant.\"}, {\"role\": \"user\", \"content\": \"Write a Python hello world.\"}, {\"role\": \"assistant\", \"content\": \"print('Hello, world!')\"}]}\n{\"messages\": [{\"role\": \"user\", \"content\": \"What is 2+2?\"}, {\"role\": \"assistant\", \"content\": \"4\"}]}\n```\n\n**Create SFT job via API:**\n\n```python\nimport requests\n\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002FsupervisedFineTuningJobs\"\nheaders = {\n    \"Authorization\": f\"Bearer {os.environ['FIREWORKS_API_KEY']}\",\n    \"Content-Type\": \"application\u002Fjson\",\n}\npayload = {\n    \"displayName\": \"my-sft-job\",\n    \"model\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"dataset\": \"accounts\u002F{account_id}\u002Fdatasets\u002F{dataset_id}\",\n    \"hyperparameters\": {\n        \"epochs\": 3,\n        \"learning_rate\": 1e-4,\n        \"batch_size\": 8,\n        \"lora_rank\": 16,\n    },\n}\nresponse = requests.post(url, headers=headers, json=payload)\njob = response.json()\nprint(f\"Job ID: {job['name']}\")\n```\n\n**Monitor job:**\n\n```python\njob_url = f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002F{job['name']}\"\nstatus = requests.get(job_url, headers=headers).json()\nprint(f\"State: {status['state']}, Progress: {status.get('progress', 'N\u002FA')}\")\n```\n\n### DPO -- Direct Preference Optimization\n\n**Data format (JSONL):** Each line has `chosen` and `rejected` conversations:\n\n```json\n{\"chosen\": [{\"role\": \"user\", \"content\": \"Explain ML\"}, {\"role\": \"assistant\", \"content\": \"Machine learning is...\"}], \"rejected\": [{\"role\": \"user\", \"content\": \"Explain ML\"}, {\"role\": \"assistant\", \"content\": \"ML is complicated...\"}]}\n```\n\n**Create DPO job:**\n\n```python\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002FdpoFineTuningJobs\"\npayload = {\n    \"displayName\": \"my-dpo-job\",\n    \"model\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"dataset\": \"accounts\u002F{account_id}\u002Fdatasets\u002F{dataset_id}\",\n    \"hyperparameters\": {\n        \"epochs\": 2,\n        \"learning_rate\": 5e-5,\n        \"beta\": 0.1,\n    },\n}\nresponse = requests.post(url, headers=headers, json=payload)\n```\n\n### RL -- Reinforcement Fine-Tuning\n\nReinforcement fine-tuning uses a reward model or reward function to optimize the base model. Jobs run on on-demand GPUs and are billed at GPU-hour rates.\n\n**Create RL job:**\n\n```python\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002FreinforcementFineTuningJobs\"\npayload = {\n    \"displayName\": \"my-rl-job\",\n    \"baseModel\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"rewardModel\": \"accounts\u002F{account_id}\u002Fmodels\u002F{reward_model_id}\",\n    \"dataset\": \"accounts\u002F{account_id}\u002Fdatasets\u002F{dataset_id}\",\n}\nresponse = requests.post(url, headers=headers, json=payload)\n```\n\n### Fine-tuning pricing\n\n| Model Size | SFT ($\u002Fhr) | DPO ($\u002Fhr) |\n|------------|------------|------------|\n| Up to 16B  | $0.50      | $1.00      |\n| 16B - 80B  | $3.00      | $6.00      |\n| 80B - 300B | $6.00      | $12.00     |\n| 300B+      | $10.00     | $20.00     |\n\nRL fine-tuning is billed at on-demand GPU rates.\n\n## On-Demand GPU Deployments\n\nDedicated GPU deployments provide predictable latency, no rate limits, and support for custom\u002Ffine-tuned models. Billed per GPU-second.\n\n### Create deployment\n\n```python\nimport requests\n\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002Fdeployments\"\nheaders = {\n    \"Authorization\": f\"Bearer {os.environ['FIREWORKS_API_KEY']}\",\n    \"Content-Type\": \"application\u002Fjson\",\n}\npayload = {\n    \"displayName\": \"my-llama-deployment\",\n    \"model\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"deploymentShape\": \"fast\",  # Options: fast, throughput, minimal\n    \"minReplicaCount\": 1,\n    \"maxReplicaCount\": 4,\n}\nresponse = requests.post(url, headers=headers, json=payload)\ndeployment = response.json()\n```\n\n### Deployment shapes\n\n| Shape | Optimized For | Use Case |\n|-------|---------------|----------|\n| `fast` | Lowest latency | Real-time chat, interactive apps |\n| `throughput` | Maximum tokens\u002Fsec | Batch processing, high volume |\n| `minimal` | Lowest cost | Development, testing |\n\n### GPU options and pricing\n\n| GPU | VRAM | Price\u002FGPU\u002Fhr |\n|-----|------|--------------|\n| A100 80GB | 80 GB | $2.90 |\n| H100 80GB | 80 GB | $4.00 |\n| H200 141GB | 141 GB | $6.00 |\n| B200 180GB | 180 GB | $9.00 |\n\n### Manage deployments\n\n```python\n# List deployments\ndeployments = requests.get(\n    f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{{account_id}}\u002Fdeployments\",\n    headers=headers,\n).json()\n\n# Scale deployment\nrequests.patch(\n    f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002F{deployment['name']}\",\n    headers=headers,\n    json={\"minReplicaCount\": 2, \"maxReplicaCount\": 8},\n)\n\n# Delete deployment\nrequests.delete(\n    f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002F{deployment['name']}\",\n    headers=headers,\n)\n```\n\n### Query your deployment\n\nOnce deployed, query using the same OpenAI-compatible API but with your deployment's model ID:\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002F{account_id}\u002Fdeployments\u002F{deployment_id}\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\n```\n\n## Embeddings\n\n**Endpoint:** `POST https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\u002Fembeddings`\n\n### Supported embedding models\n\n| Model ID | Dimensions |\n|----------|------------|\n| `nomic-ai\u002Fnomic-embed-text-v1.5` | 768 |\n| `nomic-ai\u002Fnomic-embed-text-v1` | 768 |\n| `thenlper\u002Fgte-large` | 1024 |\n| `WhereIsAI\u002FUAE-Large-V1` | 1024 |\n\n### Generate embeddings\n\n```python\nresponse = client.embeddings.create(\n    model=\"nomic-ai\u002Fnomic-embed-text-v1.5\",\n    input=[\"Machine learning is a subset of AI.\", \"Deep learning uses neural networks.\"],\n)\nfor i, emb in enumerate(response.data):\n    print(f\"Embedding {i}: {len(emb.embedding)} dimensions\")\n```\n\n## Model Selection\n\n### Popular serverless models\n\n| Model | Model ID | Params | Context | Tier |\n|-------|----------|--------|---------|------|\n| Llama 3.3 70B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct` | 70B | 131K | >16B |\n| Llama 3.2 11B Vision | `accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p2-11b-vision-instruct` | 11B | 128K | 4-16B |\n| Llama 3.2 3B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p2-3b-instruct` | 3B | 128K | \u003C4B |\n| Qwen 2.5 72B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fqwen2p5-72b-instruct` | 72B | 32K | >16B |\n| Qwen3 Coder 480B A35B | `accounts\u002Ffireworks\u002Fmodels\u002Fqwen3-coder-480b-a35b-instruct` | 480B (35B active) | 262K | MoE |\n| DeepSeek V3 | `accounts\u002Ffireworks\u002Fmodels\u002Fdeepseek-v3-0324` | 671B (37B active) | 164K | MoE |\n| Mixtral 8x7B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fmixtral-8x7b-instruct` | 46B (12B active) | 32K | MoE 0-56B |\n| Mixtral 8x22B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fmixtral-8x22b-instruct` | 141B (39B active) | 65K | MoE 56-176B |\n\n### Serverless pricing by tier\n\n| Tier | Price per 1M tokens |\n|------|---------------------|\n| \u003C 4B params | $0.10 |\n| 4B - 16B params | $0.20 |\n| > 16B params | $0.90 |\n| MoE 0 - 56B params | $0.50 |\n| MoE 56B - 176B params | $1.20 |\n\n### Model selection guide\n\n| Use Case | Recommended Model |\n|----------|-------------------|\n| General chat \u002F instruction following | `llama-v3p3-70b-instruct` |\n| Code generation | `qwen3-coder-480b-a35b-instruct` |\n| Vision \u002F multimodal | `llama-v3p2-11b-vision-instruct` |\n| Cost-sensitive workloads | `llama-v3p2-3b-instruct` |\n| Reasoning \u002F complex tasks | `deepseek-v3-0324` |\n| Fast MoE inference | `mixtral-8x7b-instruct` |\n\n## CLI (firectl)\n\n### Installation\n\n```bash\n# macOS \u002F Linux (Homebrew)\nbrew tap fw-ai\u002Ffirectl && brew install firectl\n\n# Install script\ncurl -sSL https:\u002F\u002Fcli.fireworks.ai\u002Finstall.sh | bash\n\n# Windows (Chocolatey)\nchoco install firectl\n\n# Verify\nfirectl version\n\n# Upgrade\nfirectl upgrade\n```\n\n### Authentication\n\n```bash\nfirectl signin      # Interactive login\nfirectl whoami      # Show current account\n```\n\n### Model management\n\n```bash\n# Upload a custom model\nfirectl model create my-model \u002Fpath\u002Fto\u002Fmodel\u002Fweights\n\n# List models\nfirectl model list\n\n# Get model details\nfirectl model get accounts\u002F{account_id}\u002Fmodels\u002Fmy-model\n\n# Delete model\nfirectl model delete accounts\u002F{account_id}\u002Fmodels\u002Fmy-model\n```\n\n### Deployment management\n\n```bash\n# Create on-demand deployment\nfirectl deployment create accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct \\\n    --display-name \"prod-llama\"\n\n# List deployments\nfirectl deployment list\n\n# Scale deployment\nfirectl deployment scale {deployment_id} \\\n    --min-replica-count 2 --max-replica-count 8\n\n# Delete deployment\nfirectl deployment delete {deployment_id}\n```\n\n### Fine-tuning via CLI\n\n```bash\n# Create SFT job\nfirectl supervised-fine-tuning-job create my-sft-job \\\n    --model accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct \\\n    --dataset accounts\u002F{account_id}\u002Fdatasets\u002Fmy-dataset\n\n# Create RL fine-tuning job\nfirectl reinforcement-fine-tuning-job create my-rl-job \\\n    --base-model accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct \\\n    --reward-model accounts\u002F{account_id}\u002Fmodels\u002Fmy-reward-model\n\n# Monitor jobs\nfirectl fine-tuning-job list\nfirectl fine-tuning-job get my-sft-job\n\n# Stop \u002F resume\nfirectl fine-tuning-job stop my-sft-job\nfirectl fine-tuning-job resume my-sft-job\n```\n\n## OpenAI Compatibility\n\nFireworks AI is a drop-in replacement for the OpenAI Python SDK. Change `base_url` and `api_key` -- all existing code works unchanged.\n\n### Using the OpenAI SDK\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\",\n    api_key=os.environ[\"FIREWORKS_API_KEY\"],\n)\n\n# Chat completions -- same API as OpenAI\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\n\n# Streaming -- same API\nstream = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n    stream=True,\n)\n\n# Embeddings -- same API\nembeddings = client.embeddings.create(\n    model=\"nomic-ai\u002Fnomic-embed-text-v1.5\",\n    input=[\"text to embed\"],\n)\n```\n\n### Environment variable approach\n\n```bash\nexport OPENAI_API_BASE=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\"\nexport OPENAI_API_KEY=\"fw_...\"\n```\n\nThen use the OpenAI SDK without any code changes.\n\n### Fireworks-specific parameters\n\nFireworks adds `context_length_exceeded_behavior` to control what happens when `prompt + max_tokens` exceeds the model's context window:\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"...\"}],\n    max_tokens=512,\n    extra_body={\"context_length_exceeded_behavior\": \"truncate\"},  # or \"error\"\n)\n```\n\n### Using the native Fireworks SDK\n\n```python\nimport fireworks.client\n\nfireworks.client.api_key = os.environ[\"FIREWORKS_API_KEY\"]\n\nresponse = fireworks.client.ChatCompletion.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n## Cost Optimization\n\n### Prompt caching\n\nFireworks automatically caches repeated prompt prefixes. No configuration needed -- identical prefixes across requests reuse cached KV states, reducing both latency and cost.\n\nBest practices for prompt caching:\n- Place static system prompts at the beginning of messages\n- Keep dynamic content at the end\n- Use consistent system prompts across requests\n\n### Batch inference\n\nBatch API provides up to 50% cost savings for non-latency-sensitive workloads:\n\n```python\n# Prepare batch file (JSONL)\n# Each line: {\"custom_id\": \"req-1\", \"method\": \"POST\", \"url\": \"\u002Fv1\u002Fchat\u002Fcompletions\", \"body\": {...}}\n\n# Upload batch file\nbatch_file = client.files.create(\n    file=open(\"batch_requests.jsonl\", \"rb\"),\n    purpose=\"batch\",\n)\n\n# Create batch job\nbatch = client.batches.create(\n    input_file_id=batch_file.id,\n    endpoint=\"\u002Fv1\u002Fchat\u002Fcompletions\",\n    completion_window=\"24h\",\n)\nprint(f\"Batch ID: {batch.id}, Status: {batch.status}\")\n\n# Check status\nbatch_status = client.batches.retrieve(batch.id)\nprint(f\"Status: {batch_status.status}\")\n```\n\n### Cost reduction strategies\n\n| Strategy | Savings | How |\n|----------|---------|-----|\n| Use smaller models | 50-90% | `llama-v3p2-3b-instruct` at $0.10\u002FM tokens vs 70B at $0.90\u002FM |\n| Batch API | ~50% | Async processing for non-real-time workloads |\n| Prompt caching | 20-40% | Consistent system prompts, static prefixes |\n| MoE models | 30-50% | Mixtral\u002FDeepSeek: large capacity, smaller active params |\n| On-demand deployments | Variable | Predictable pricing at scale, no per-token markup |\n| Reduce max_tokens | 10-30% | Set realistic output limits |\n\n## Common Issues\n\n| Problem | Solution |\n|---------|----------|\n| `401 Unauthorized` | Check `FIREWORKS_API_KEY` is set and valid. Get key from https:\u002F\u002Ffireworks.ai\u002Fapi-keys |\n| `Model not found` | Use full model ID: `accounts\u002Ffireworks\u002Fmodels\u002F{model_name}` |\n| `Context length exceeded` | Reduce input or set `context_length_exceeded_behavior: \"truncate\"` |\n| Rate limited (serverless) | Switch to on-demand deployment for no rate limits |\n| Slow cold start on deployment | Set `minReplicaCount >= 1` to keep replicas warm |\n| Fine-tuning job stuck | Check dataset format matches expected JSONL schema. Use `firectl fine-tuning-job get` |\n| Tool calls not working | Use models that support function calling (Llama 3.3, Qwen, DeepSeek V3) |\n| JSON mode returns invalid JSON | Use `response_format` with explicit schema for strict enforcement |\n| Streaming usage stats missing | Upgrade `openai` SDK to >= 1.6.1. Usage is in the final stream chunk |\n| Deployment not scaling | Check `maxReplicaCount` is set high enough. Review deployment shape |\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fdocs.fireworks.ai\n- **API Reference**: https:\u002F\u002Fdocs.fireworks.ai\u002Fapi-reference\u002Fintroduction\n- **Dashboard**: https:\u002F\u002Ffireworks.ai\u002Fdashboard\n- **Model Catalog**: https:\u002F\u002Ffireworks.ai\u002Fmodels\n- **Pricing**: https:\u002F\u002Ffireworks.ai\u002Fpricing\n- **firectl CLI**: https:\u002F\u002Fdocs.fireworks.ai\u002Ftools-sdks\u002Ffirectl\u002Ffirectl\n- **OpenAI Compatibility**: https:\u002F\u002Fdocs.fireworks.ai\u002Ftools-sdks\u002Fopenai-compatibility\n- **Fine-Tuning Guide**: https:\u002F\u002Fdocs.fireworks.ai\u002Ffine-tuning\u002Ffine-tuning-models\n- **Cookbook (GitHub)**: https:\u002F\u002Fgithub.com\u002Ffw-ai\u002Fcookbook\n- **Status Page**: https:\u002F\u002Fstatus.fireworks.ai\n","# Fireworks AI -- Fast Inference & Fine-Tuning\n\nFastest open-model inference platform with serverless and on-demand GPU deployments, OpenAI-compatible API, and built-in fine-tuning (SFT, DPO, RL).\n\n## When to Use Fireworks AI\n\n**Use Fireworks AI when:**\n- Need fast serverless inference for open-source models (Llama, Qwen, DeepSeek, Mixtral)\n- Want OpenAI SDK drop-in replacement with open models\n- Need fine-tuning without managing infrastructure (SFT, DPO, RL)\n- Require structured output \u002F JSON mode \u002F function calling with open models\n- Need dedicated GPU deployments with predictable latency\n- Require SOC2 or HIPAA compliance\n- Want prompt caching and batch inference for cost savings\n\n**Use alternatives instead:**\n\n| Need | Use Instead |\n|------|-------------|\n| Self-hosted inference (full control) | **vLLM**, **TensorRT-LLM** |\n| Cheapest serverless inference | **Groq** (free tier), **Together AI** |\n| Managed LoRA fine-tuning (no infra) | **Tinker** |\n| Closed-model APIs (GPT-4, Claude) | **OpenAI**, **Anthropic** direct |\n| GPU instances with SSH access | **Lambda Labs**, **RunPod** |\n| Multi-cloud orchestration | **SkyPilot** |\n\n\n## Credential Setup\n\nCredentials are auto-injected by openscience when connected via the dashboard.\n\n```bash\n# Verify credentials\n[ -n \"$FIREWORKS_API_KEY\" ] && echo \"FIREWORKS_API_KEY set\" || echo \"NOT SET\"\n```\n\nIf not set: connect Fireworks AI at https:\u002F\u002Fapp.syntheticsciences.ai -> Services, then restart openscience.\n\n## Quick Start\n\n### Install\n\n```python\npip install fireworks-ai openai\n```\n\n### Set API key\n\n```python\nimport os\nos.environ[\"FIREWORKS_API_KEY\"] = \"fw_...\"  # from https:\u002F\u002Ffireworks.ai\u002Fapi-keys\n```\n\n### Basic chat completion\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\",\n    api_key=os.environ[\"FIREWORKS_API_KEY\"],\n)\n\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain gradient descent in two sentences.\"},\n    ],\n    max_tokens=256,\n    temperature=0.7,\n)\nprint(response.choices[0].message.content)\n```\n\n## Inference\n\n### Chat completions\n\n**Endpoint:** `POST https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\u002Fchat\u002Fcompletions`\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\",\n    api_key=os.environ[\"FIREWORKS_API_KEY\"],\n)\n\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[\n        {\"role\": \"user\", \"content\": \"Write a Python quicksort function.\"},\n    ],\n    max_tokens=512,\n    temperature=0.0,\n)\nprint(response.choices[0].message.content)\n```\n\n### Streaming\n\n```python\nstream = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain transformers.\"}],\n    stream=True,\n    max_tokens=512,\n)\nfor chunk in stream:\n    delta = chunk.choices[0].delta.content\n    if delta:\n        print(delta, end=\"\", flush=True)\n```\n\n### Function calling \u002F tool use\n\n```python\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_weather\",\n            \"description\": \"Get current weather for a location\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"location\": {\"type\": \"string\", \"description\": \"City name\"},\n                    \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n                },\n                \"required\": [\"location\"],\n            },\n        },\n    }\n]\n\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"What is the weather in San Francisco?\"}],\n    tools=tools,\n    tool_choice=\"auto\",\n)\ntool_call = response.choices[0].message.tool_calls[0]\nprint(tool_call.function.name, tool_call.function.arguments)\n```\n\n### Structured output \u002F JSON mode\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"List 3 planets with mass and diameter.\"}],\n    response_format={\"type\": \"json_object\"},\n    max_tokens=512,\n)\nimport json\ndata = json.loads(response.choices[0].message.content)\n```\n\nFor strict schema enforcement, use `response_format` with a JSON schema:\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Extract name and age from: John is 30.\"}],\n    response_format={\n        \"type\": \"json_object\",\n        \"schema\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"name\": {\"type\": \"string\"},\n                \"age\": {\"type\": \"integer\"},\n            },\n            \"required\": [\"name\", \"age\"],\n        },\n    },\n)\n```\n\n### Vision (multimodal)\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p2-11b-vision-instruct\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\"url\": \"https:\u002F\u002Fexample.com\u002Fphoto.jpg\"},\n                },\n            ],\n        }\n    ],\n    max_tokens=256,\n)\nprint(response.choices[0].message.content)\n```\n\n## Fine-Tuning\n\nFireworks supports three fine-tuning methods: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Fine-Tuning (RL). All use LoRA adapters by default.\n\n### SFT -- Supervised Fine-Tuning\n\n**Data format (JSONL):** Each line is a conversation with `messages` array:\n\n```json\n{\"messages\": [{\"role\": \"system\", \"content\": \"You are a coding assistant.\"}, {\"role\": \"user\", \"content\": \"Write a Python hello world.\"}, {\"role\": \"assistant\", \"content\": \"print('Hello, world!')\"}]}\n{\"messages\": [{\"role\": \"user\", \"content\": \"What is 2+2?\"}, {\"role\": \"assistant\", \"content\": \"4\"}]}\n```\n\n**Create SFT job via API:**\n\n```python\nimport requests\n\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002FsupervisedFineTuningJobs\"\nheaders = {\n    \"Authorization\": f\"Bearer {os.environ['FIREWORKS_API_KEY']}\",\n    \"Content-Type\": \"application\u002Fjson\",\n}\npayload = {\n    \"displayName\": \"my-sft-job\",\n    \"model\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"dataset\": \"accounts\u002F{account_id}\u002Fdatasets\u002F{dataset_id}\",\n    \"hyperparameters\": {\n        \"epochs\": 3,\n        \"learning_rate\": 1e-4,\n        \"batch_size\": 8,\n        \"lora_rank\": 16,\n    },\n}\nresponse = requests.post(url, headers=headers, json=payload)\njob = response.json()\nprint(f\"Job ID: {job['name']}\")\n```\n\n**Monitor job:**\n\n```python\njob_url = f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002F{job['name']}\"\nstatus = requests.get(job_url, headers=headers).json()\nprint(f\"State: {status['state']}, Progress: {status.get('progress', 'N\u002FA')}\")\n```\n\n### DPO -- Direct Preference Optimization\n\n**Data format (JSONL):** Each line has `chosen` and `rejected` conversations:\n\n```json\n{\"chosen\": [{\"role\": \"user\", \"content\": \"Explain ML\"}, {\"role\": \"assistant\", \"content\": \"Machine learning is...\"}], \"rejected\": [{\"role\": \"user\", \"content\": \"Explain ML\"}, {\"role\": \"assistant\", \"content\": \"ML is complicated...\"}]}\n```\n\n**Create DPO job:**\n\n```python\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002FdpoFineTuningJobs\"\npayload = {\n    \"displayName\": \"my-dpo-job\",\n    \"model\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"dataset\": \"accounts\u002F{account_id}\u002Fdatasets\u002F{dataset_id}\",\n    \"hyperparameters\": {\n        \"epochs\": 2,\n        \"learning_rate\": 5e-5,\n        \"beta\": 0.1,\n    },\n}\nresponse = requests.post(url, headers=headers, json=payload)\n```\n\n### RL -- Reinforcement Fine-Tuning\n\nReinforcement fine-tuning uses a reward model or reward function to optimize the base model. Jobs run on on-demand GPUs and are billed at GPU-hour rates.\n\n**Create RL job:**\n\n```python\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002FreinforcementFineTuningJobs\"\npayload = {\n    \"displayName\": \"my-rl-job\",\n    \"baseModel\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"rewardModel\": \"accounts\u002F{account_id}\u002Fmodels\u002F{reward_model_id}\",\n    \"dataset\": \"accounts\u002F{account_id}\u002Fdatasets\u002F{dataset_id}\",\n}\nresponse = requests.post(url, headers=headers, json=payload)\n```\n\n### Fine-tuning pricing\n\n| Model Size | SFT ($\u002Fhr) | DPO ($\u002Fhr) |\n|------------|------------|------------|\n| Up to 16B  | $0.50      | $1.00      |\n| 16B - 80B  | $3.00      | $6.00      |\n| 80B - 300B | $6.00      | $12.00     |\n| 300B+      | $10.00     | $20.00     |\n\nRL fine-tuning is billed at on-demand GPU rates.\n\n## On-Demand GPU Deployments\n\nDedicated GPU deployments provide predictable latency, no rate limits, and support for custom\u002Ffine-tuned models. Billed per GPU-second.\n\n### Create deployment\n\n```python\nimport requests\n\nurl = \"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{account_id}\u002Fdeployments\"\nheaders = {\n    \"Authorization\": f\"Bearer {os.environ['FIREWORKS_API_KEY']}\",\n    \"Content-Type\": \"application\u002Fjson\",\n}\npayload = {\n    \"displayName\": \"my-llama-deployment\",\n    \"model\": \"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    \"deploymentShape\": \"fast\",  # Options: fast, throughput, minimal\n    \"minReplicaCount\": 1,\n    \"maxReplicaCount\": 4,\n}\nresponse = requests.post(url, headers=headers, json=payload)\ndeployment = response.json()\n```\n\n### Deployment shapes\n\n| Shape | Optimized For | Use Case |\n|-------|---------------|----------|\n| `fast` | Lowest latency | Real-time chat, interactive apps |\n| `throughput` | Maximum tokens\u002Fsec | Batch processing, high volume |\n| `minimal` | Lowest cost | Development, testing |\n\n### GPU options and pricing\n\n| GPU | VRAM | Price\u002FGPU\u002Fhr |\n|-----|------|--------------|\n| A100 80GB | 80 GB | $2.90 |\n| H100 80GB | 80 GB | $4.00 |\n| H200 141GB | 141 GB | $6.00 |\n| B200 180GB | 180 GB | $9.00 |\n\n### Manage deployments\n\n```python\n# List deployments\ndeployments = requests.get(\n    f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002Faccounts\u002F{{account_id}}\u002Fdeployments\",\n    headers=headers,\n).json()\n\n# Scale deployment\nrequests.patch(\n    f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002F{deployment['name']}\",\n    headers=headers,\n    json={\"minReplicaCount\": 2, \"maxReplicaCount\": 8},\n)\n\n# Delete deployment\nrequests.delete(\n    f\"https:\u002F\u002Fapi.fireworks.ai\u002Fv1\u002F{deployment['name']}\",\n    headers=headers,\n)\n```\n\n### Query your deployment\n\nOnce deployed, query using the same OpenAI-compatible API but with your deployment's model ID:\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002F{account_id}\u002Fdeployments\u002F{deployment_id}\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\n```\n\n## Embeddings\n\n**Endpoint:** `POST https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\u002Fembeddings`\n\n### Supported embedding models\n\n| Model ID | Dimensions |\n|----------|------------|\n| `nomic-ai\u002Fnomic-embed-text-v1.5` | 768 |\n| `nomic-ai\u002Fnomic-embed-text-v1` | 768 |\n| `thenlper\u002Fgte-large` | 1024 |\n| `WhereIsAI\u002FUAE-Large-V1` | 1024 |\n\n### Generate embeddings\n\n```python\nresponse = client.embeddings.create(\n    model=\"nomic-ai\u002Fnomic-embed-text-v1.5\",\n    input=[\"Machine learning is a subset of AI.\", \"Deep learning uses neural networks.\"],\n)\nfor i, emb in enumerate(response.data):\n    print(f\"Embedding {i}: {len(emb.embedding)} dimensions\")\n```\n\n## Model Selection\n\n### Popular serverless models\n\n| Model | Model ID | Params | Context | Tier |\n|-------|----------|--------|---------|------|\n| Llama 3.3 70B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct` | 70B | 131K | >16B |\n| Llama 3.2 11B Vision | `accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p2-11b-vision-instruct` | 11B | 128K | 4-16B |\n| Llama 3.2 3B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p2-3b-instruct` | 3B | 128K | \u003C4B |\n| Qwen 2.5 72B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fqwen2p5-72b-instruct` | 72B | 32K | >16B |\n| Qwen3 Coder 480B A35B | `accounts\u002Ffireworks\u002Fmodels\u002Fqwen3-coder-480b-a35b-instruct` | 480B (35B active) | 262K | MoE |\n| DeepSeek V3 | `accounts\u002Ffireworks\u002Fmodels\u002Fdeepseek-v3-0324` | 671B (37B active) | 164K | MoE |\n| Mixtral 8x7B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fmixtral-8x7b-instruct` | 46B (12B active) | 32K | MoE 0-56B |\n| Mixtral 8x22B Instruct | `accounts\u002Ffireworks\u002Fmodels\u002Fmixtral-8x22b-instruct` | 141B (39B active) | 65K | MoE 56-176B |\n\n### Serverless pricing by tier\n\n| Tier | Price per 1M tokens |\n|------|---------------------|\n| \u003C 4B params | $0.10 |\n| 4B - 16B params | $0.20 |\n| > 16B params | $0.90 |\n| MoE 0 - 56B params | $0.50 |\n| MoE 56B - 176B params | $1.20 |\n\n### Model selection guide\n\n| Use Case | Recommended Model |\n|----------|-------------------|\n| General chat \u002F instruction following | `llama-v3p3-70b-instruct` |\n| Code generation | `qwen3-coder-480b-a35b-instruct` |\n| Vision \u002F multimodal | `llama-v3p2-11b-vision-instruct` |\n| Cost-sensitive workloads | `llama-v3p2-3b-instruct` |\n| Reasoning \u002F complex tasks | `deepseek-v3-0324` |\n| Fast MoE inference | `mixtral-8x7b-instruct` |\n\n## CLI (firectl)\n\n### Installation\n\n```bash\n# macOS \u002F Linux (Homebrew)\nbrew tap fw-ai\u002Ffirectl && brew install firectl\n\n# Install script\ncurl -sSL https:\u002F\u002Fcli.fireworks.ai\u002Finstall.sh | bash\n\n# Windows (Chocolatey)\nchoco install firectl\n\n# Verify\nfirectl version\n\n# Upgrade\nfirectl upgrade\n```\n\n### Authentication\n\n```bash\nfirectl signin      # Interactive login\nfirectl whoami      # Show current account\n```\n\n### Model management\n\n```bash\n# Upload a custom model\nfirectl model create my-model \u002Fpath\u002Fto\u002Fmodel\u002Fweights\n\n# List models\nfirectl model list\n\n# Get model details\nfirectl model get accounts\u002F{account_id}\u002Fmodels\u002Fmy-model\n\n# Delete model\nfirectl model delete accounts\u002F{account_id}\u002Fmodels\u002Fmy-model\n```\n\n### Deployment management\n\n```bash\n# Create on-demand deployment\nfirectl deployment create accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct \\\n    --display-name \"prod-llama\"\n\n# List deployments\nfirectl deployment list\n\n# Scale deployment\nfirectl deployment scale {deployment_id} \\\n    --min-replica-count 2 --max-replica-count 8\n\n# Delete deployment\nfirectl deployment delete {deployment_id}\n```\n\n### Fine-tuning via CLI\n\n```bash\n# Create SFT job\nfirectl supervised-fine-tuning-job create my-sft-job \\\n    --model accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct \\\n    --dataset accounts\u002F{account_id}\u002Fdatasets\u002Fmy-dataset\n\n# Create RL fine-tuning job\nfirectl reinforcement-fine-tuning-job create my-rl-job \\\n    --base-model accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct \\\n    --reward-model accounts\u002F{account_id}\u002Fmodels\u002Fmy-reward-model\n\n# Monitor jobs\nfirectl fine-tuning-job list\nfirectl fine-tuning-job get my-sft-job\n\n# Stop \u002F resume\nfirectl fine-tuning-job stop my-sft-job\nfirectl fine-tuning-job resume my-sft-job\n```\n\n## OpenAI Compatibility\n\nFireworks AI is a drop-in replacement for the OpenAI Python SDK. Change `base_url` and `api_key` -- all existing code works unchanged.\n\n### Using the OpenAI SDK\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\",\n    api_key=os.environ[\"FIREWORKS_API_KEY\"],\n)\n\n# Chat completions -- same API as OpenAI\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\n\n# Streaming -- same API\nstream = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n    stream=True,\n)\n\n# Embeddings -- same API\nembeddings = client.embeddings.create(\n    model=\"nomic-ai\u002Fnomic-embed-text-v1.5\",\n    input=[\"text to embed\"],\n)\n```\n\n### Environment variable approach\n\n```bash\nexport OPENAI_API_BASE=\"https:\u002F\u002Fapi.fireworks.ai\u002Finference\u002Fv1\"\nexport OPENAI_API_KEY=\"fw_...\"\n```\n\nThen use the OpenAI SDK without any code changes.\n\n### Fireworks-specific parameters\n\nFireworks adds `context_length_exceeded_behavior` to control what happens when `prompt + max_tokens` exceeds the model's context window:\n\n```python\nresponse = client.chat.completions.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"...\"}],\n    max_tokens=512,\n    extra_body={\"context_length_exceeded_behavior\": \"truncate\"},  # or \"error\"\n)\n```\n\n### Using the native Fireworks SDK\n\n```python\nimport fireworks.client\n\nfireworks.client.api_key = os.environ[\"FIREWORKS_API_KEY\"]\n\nresponse = fireworks.client.ChatCompletion.create(\n    model=\"accounts\u002Ffireworks\u002Fmodels\u002Fllama-v3p3-70b-instruct\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n## Cost Optimization\n\n### Prompt caching\n\nFireworks automatically caches repeated prompt prefixes. No configuration needed -- identical prefixes across requests reuse cached KV states, reducing both latency and cost.\n\nBest practices for prompt caching:\n- Place static system prompts at the beginning of messages\n- Keep dynamic content at the end\n- Use consistent system prompts across requests\n\n### Batch inference\n\nBatch API provides up to 50% cost savings for non-latency-sensitive workloads:\n\n```python\n# Prepare batch file (JSONL)\n# Each line: {\"custom_id\": \"req-1\", \"method\": \"POST\", \"url\": \"\u002Fv1\u002Fchat\u002Fcompletions\", \"body\": {...}}\n\n# Upload batch file\nbatch_file = client.files.create(\n    file=open(\"batch_requests.jsonl\", \"rb\"),\n    purpose=\"batch\",\n)\n\n# Create batch job\nbatch = client.batches.create(\n    input_file_id=batch_file.id,\n    endpoint=\"\u002Fv1\u002Fchat\u002Fcompletions\",\n    completion_window=\"24h\",\n)\nprint(f\"Batch ID: {batch.id}, Status: {batch.status}\")\n\n# Check status\nbatch_status = client.batches.retrieve(batch.id)\nprint(f\"Status: {batch_status.status}\")\n```\n\n### Cost reduction strategies\n\n| Strategy | Savings | How |\n|----------|---------|-----|\n| Use smaller models | 50-90% | `llama-v3p2-3b-instruct` at $0.10\u002FM tokens vs 70B at $0.90\u002FM |\n| Batch API | ~50% | Async processing for non-real-time workloads |\n| Prompt caching | 20-40% | Consistent system prompts, static prefixes |\n| MoE models | 30-50% | Mixtral\u002FDeepSeek: large capacity, smaller active params |\n| On-demand deployments | Variable | Predictable pricing at scale, no per-token markup |\n| Reduce max_tokens | 10-30% | Set realistic output limits |\n\n## Common Issues\n\n| Problem | Solution |\n|---------|----------|\n| `401 Unauthorized` | Check `FIREWORKS_API_KEY` is set and valid. Get key from https:\u002F\u002Ffireworks.ai\u002Fapi-keys |\n| `Model not found` | Use full model ID: `accounts\u002Ffireworks\u002Fmodels\u002F{model_name}` |\n| `Context length exceeded` | Reduce input or set `context_length_exceeded_behavior: \"truncate\"` |\n| Rate limited (serverless) | Switch to on-demand deployment for no rate limits |\n| Slow cold start on deployment | Set `minReplicaCount >= 1` to keep replicas warm |\n| Fine-tuning job stuck | Check dataset format matches expected JSONL schema. Use `firectl fine-tuning-job get` |\n| Tool calls not working | Use models that support function calling (Llama 3.3, Qwen, DeepSeek V3) |\n| JSON mode returns invalid JSON | Use `response_format` with explicit schema for strict enforcement |\n| Streaming usage stats missing | Upgrade `openai` SDK to >= 1.6.1. Usage is in the final stream chunk |\n| Deployment not scaling | Check `maxReplicaCount` is set high enough. Review deployment shape |\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fdocs.fireworks.ai\n- **API Reference**: https:\u002F\u002Fdocs.fireworks.ai\u002Fapi-reference\u002Fintroduction\n- **Dashboard**: https:\u002F\u002Ffireworks.ai\u002Fdashboard\n- **Model Catalog**: https:\u002F\u002Ffireworks.ai\u002Fmodels\n- **Pricing**: https:\u002F\u002Ffireworks.ai\u002Fpricing\n- **firectl CLI**: https:\u002F\u002Fdocs.fireworks.ai\u002Ftools-sdks\u002Ffirectl\u002Ffirectl\n- **OpenAI Compatibility**: https:\u002F\u002Fdocs.fireworks.ai\u002Ftools-sdks\u002Fopenai-compatibility\n- **Fine-Tuning Guide**: https:\u002F\u002Fdocs.fireworks.ai\u002Ffine-tuning\u002Ffine-tuning-models\n- **Cookbook (GitHub)**: https:\u002F\u002Fgithub.com\u002Ffw-ai\u002Fcookbook\n- **Status Page**: https:\u002F\u002Fstatus.fireworks.ai\n",{"name":5,"tags":44,"author":49,"license":15,"version":50,"category":51,"description":7,"dependencies":52},[45,46,36,47,23,48],"Inference","Fireworks AI","Serverless","OpenAI-Compatible","Synthetic Sciences","1.0.0","cloud-compute",[53,38],"fireworks-ai","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fcloud-compute\u002Ffireworks-ai","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":61,"prompt":71,"zipAvailable":72},[62,65,68],{"tool":63,"command":64},"npx","npx skills add fadrienne\u002FMyGitHub --skill fireworks-ai-inference",{"tool":66,"command":67},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill fireworks-ai-inference",{"tool":69,"command":70},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill fireworks-ai-inference","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fcloud-compute\u002Ffireworks-ai\u002FSKILL.md",true,[],[75,107,133,166,193,220],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":86,"publishedAt":39},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[81],{"categoryNo":82,"categoryName":83,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106],"power","statistical power","effect","scipy","regression","choose","design","uv","when","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":108,"name":109,"nameZh":6,"description":110,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":111,"categories":112,"keywords":116,"publishedAt":39},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[113],{"categoryNo":114,"categoryName":115,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000008","测试与质量",[98,117,87,118,119,120,95,121,122,123,24,124,125,126,127,128,129,130,131,132],"statistical analysis","academic research","implementing","selection","you","reporting","data","tests","use","with","test","research","results","statsmodels","models","help",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":148,"publishedAt":39},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[139,142,143],{"categoryNo":140,"categoryName":141,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":82,"categoryName":83,"taxonomyType":84,"taxonomyTypeDesc":85},{"categoryNo":144,"categoryName":145,"taxonomyType":146,"taxonomyTypeDesc":147},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[149,150,151,152,153,154,155,156,157,158,96,32,159,160,161,162,163,164,122,165],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":167,"name":168,"nameZh":6,"description":169,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":170,"categories":171,"keywords":176,"publishedAt":39},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[172,175],{"categoryNo":173,"categoryName":174,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":82,"categoryName":83,"taxonomyType":84,"taxonomyTypeDesc":85},[177,150,178,179,180,181,182,183,184,157,95,185,186,187,188,189,190,191,24,192],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":194,"name":195,"nameZh":6,"description":196,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":197,"categories":198,"keywords":205,"publishedAt":39},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[199,202],{"categoryNo":200,"categoryName":201,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":203,"categoryName":204,"taxonomyType":146,"taxonomyTypeDesc":147},"01JBSEEDCAT000000000000030","开发工程师",[206,207,128,208,209,210,211,94,212,95,96,98,213,214,215,216,217,218,164,219],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":221,"name":222,"nameZh":6,"description":223,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":224,"licenseFileUrl":6,"sourceHtmlUrl":225,"categories":226,"keywords":233,"publishedAt":39},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[227,230,231,232],{"categoryNo":228,"categoryName":229,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":140,"categoryName":141,"taxonomyType":84,"taxonomyTypeDesc":85},{"categoryNo":82,"categoryName":83,"taxonomyType":84,"taxonomyTypeDesc":85},{"categoryNo":144,"categoryName":145,"taxonomyType":146,"taxonomyTypeDesc":147},[149,234,235,236,237,238,177,239,240,95,96,98,241,242,243,244,245,246,150,164],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]