[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XG6XB6A0C6TBYDVXERH6WD":3,"skill-same:01M6XG6XB6A0C6TBYDVXERH6WD":78},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":58,"commitSha":59,"skillDirectory":60,"forksCount":14,"repositoryHtmlUrl":61,"repositoryHomepageUrl":6,"ownerHtmlUrl":62,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":63,"install":64,"resources":77},"01M6XG6XB6A0C6TBYDVXERH6WD","groq-inference",null,"Ultra-fast LLM inference on custom LPU hardware. OpenAI-compatible API at api.groq.com. Lowest latency in the industry (500-1000+ tok\u002Fs). Supports chat completions, vision, audio (Whisper STT + TTS), tool calling, JSON mode, and streaming. Free tier available. Inference only — no training.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fgroq\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"groq inference","groq","inference","free-tier","Low-Latency","LPU","ultra","at","mode","synthetic","free","stt","vision","whisper","only","Audio","api","streaming","openai","training","2026-11-10 10:12:30",false,"---\nname: groq-inference\ndescription: Ultra-fast LLM inference on custom LPU hardware. OpenAI-compatible API at api.groq.com. Lowest latency in the industry (500-1000+ tok\u002Fs). Supports chat completions, vision, audio (Whisper STT + TTS), tool calling, JSON mode, and streaming. Free tier available. Inference only — no training.\ncategory: ml-inference\nversion: 1.0.0\nauthor: Synthetic Sciences\nlicense: MIT\ntags: [Inference, Groq, LPU, Low-Latency, OpenAI-Compatible, Free-Tier]\ndependencies: [groq, openai]\n---\n\n# Groq — Ultra-Fast LLM Inference\n\nGroq runs open-weight LLMs on custom LPU (Language Processing Unit) hardware, delivering the lowest inference latency in the industry — up to 1000+ tokens\u002Fsec. OpenAI-compatible API, free tier, pay-per-token pricing. Inference only, no training.\n\n## When to Use Groq\n\n**Use when:**\n- You need the fastest possible inference latency (real-time chat, agents, interactive apps)\n- You want open-weight models (Llama, Qwen, DeepSeek) without managing GPUs\n- You need an OpenAI-compatible drop-in replacement with open models\n- You want free-tier access for prototyping\n- You need fast Whisper transcription or vision with Llama 4\n\n**Don't use when:**\n- You need to fine-tune or train models (inference-only)\n- You need proprietary models (GPT-4o, Claude, Gemini)\n- You need embeddings or image generation APIs (not offered)\n\n| Feature | Groq | Together AI | Fireworks | Replicate |\n|---------|------|-------------|-----------|-----------|\n| Hardware | Custom LPU | NVIDIA GPUs | NVIDIA GPUs | NVIDIA GPUs |\n| Speed (Llama 70B) | ~280 tok\u002Fs | ~100 tok\u002Fs | ~100 tok\u002Fs | ~50 tok\u002Fs |\n| Free tier | Yes | Yes | Yes | No |\n| Training | No | Yes | Yes | Yes |\n\n\n## Credential Setup\n\nCredentials are auto-injected by openscience when connected via the dashboard.\n\n```bash\n# Verify credentials\n[ -n \"$GROQ_API_KEY\" ] && echo \"GROQ_API_KEY set\" || echo \"NOT SET\"\n```\n\nIf not set: connect Groq at https:\u002F\u002Fapp.syntheticsciences.ai -> Services, then restart openscience.\n\n## Quick Start\n\n```bash\npip install groq\nexport GROQ_API_KEY=\"gsk_...\"   # https:\u002F\u002Fconsole.groq.com\u002Fkeys\n```\n\n```python\nfrom groq import Groq\n\nclient = Groq()\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain quantum computing in one paragraph.\"}\n    ],\n    temperature=0.7,\n    max_completion_tokens=512,\n)\nprint(response.choices[0].message.content)\n```\n\n## Chat Completions\n\n### Streaming\n\n```python\nfrom groq import Groq\n\nclient = Groq()\nstream = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain fast inference\"}],\n    stream=True,\n)\nfor chunk in stream:\n    content = chunk.choices[0].delta.content\n    if content:\n        print(content, end=\"\")\n```\n\n### JSON Mode\n\n```python\nfrom groq import Groq\nimport json\n\nclient = Groq()\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"Output valid JSON only.\"},\n        {\"role\": \"user\", \"content\": \"List 3 programming languages with their paradigms.\"}\n    ],\n    response_format={\"type\": \"json_object\"},\n    temperature=0,\n)\ndata = json.loads(response.choices[0].message.content)\n```\n\n### Tool \u002F Function Calling\n\n```python\nfrom groq import Groq\nimport json\n\nclient = Groq()\ntools = [{\n    \"type\": \"function\",\n    \"function\": {\n        \"name\": \"get_weather\",\n        \"description\": \"Get weather for a location\",\n        \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"location\": {\"type\": \"string\", \"description\": \"City name\"},\n                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]}\n            },\n            \"required\": [\"location\"],\n            \"additionalProperties\": False\n        }\n    }\n}]\n\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[{\"role\": \"user\", \"content\": \"Weather in San Francisco?\"}],\n    tools=tools,\n    tool_choice=\"auto\",\n)\n\nmessage = response.choices[0].message\nif message.tool_calls:\n    call = message.tool_calls[0]\n    args = json.loads(call.function.arguments)\n    # Execute function, then send result back as role=\"tool\" message\n    result = {\"temperature\": 62, \"condition\": \"foggy\"}\n    followup = client.chat.completions.create(\n        model=\"llama-3.3-70b-versatile\",\n        messages=[\n            {\"role\": \"user\", \"content\": \"Weather in San Francisco?\"},\n            message,\n            {\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": json.dumps(result)}\n        ],\n    )\n    print(followup.choices[0].message.content)\n```\n\n## Vision\n\nLlama 4 models support image input via URL or base64. Max image size: 20 MB.\n\n```python\nfrom groq import Groq\n\nclient = Groq()\nresponse = client.chat.completions.create(\n    model=\"meta-llama\u002Fllama-4-scout-17b-16e-instruct\",\n    messages=[{\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"text\", \"text\": \"Describe this image.\"},\n            {\"type\": \"image_url\", \"image_url\": {\"url\": \"https:\u002F\u002Fexample.com\u002Fphoto.jpg\"}}\n        ]\n    }],\n    max_completion_tokens=1024,\n)\nprint(response.choices[0].message.content)\n```\n\nVision models: `meta-llama\u002Fllama-4-scout-17b-16e-instruct`, `meta-llama\u002Fllama-4-maverick-17b-128e-instruct`.\n\n## Audio \u002F Speech-to-Text\n\nWhisper on LPU for near-instant transcription. Formats: flac, mp3, mp4, mpeg, mpga, m4a, ogg, wav, webm. Max: 25 MB free \u002F 100 MB developer tier.\n\n```python\nfrom groq import Groq\n\nclient = Groq()\n\n# Transcription\nwith open(\"audio.mp3\", \"rb\") as f:\n    result = client.audio.transcriptions.create(\n        file=(\"audio.mp3\", f),\n        model=\"whisper-large-v3-turbo\",\n        language=\"en\",\n        response_format=\"verbose_json\",\n    )\nprint(result.text)\n\n# Translation (any language -> English)\nwith open(\"french.mp3\", \"rb\") as f:\n    result = client.audio.translations.create(\n        file=(\"french.mp3\", f), model=\"whisper-large-v3\",\n    )\nprint(result.text)\n```\n\n### Text-to-Speech\n\n```python\nfrom groq import Groq\nfrom pathlib import Path\n\nclient = Groq()\nspeech = client.audio.speech.create(\n    model=\"playai-tts\",\n    voice=\"Arista-PlayAI\",\n    input=\"Hello from Groq!\",\n)\nPath(\"output.mp3\").write_bytes(speech.content)\n```\n\n## Model Selection\n\n### Production Models\n\n| Model | Model ID | Speed | Context | Max Output | Input $\u002F1M | Output $\u002F1M |\n|-------|----------|-------|---------|------------|------------|-------------|\n| Llama 3.1 8B | `llama-3.1-8b-instant` | 560 t\u002Fs | 131,072 | 131,072 | $0.05 | $0.08 |\n| Llama 3.3 70B | `llama-3.3-70b-versatile` | 280 t\u002Fs | 131,072 | 32,768 | $0.59 | $0.79 |\n| GPT-OSS 120B | `openai\u002Fgpt-oss-120b` | 500 t\u002Fs | 131,072 | 65,536 | $0.15 | $0.60 |\n| GPT-OSS 20B | `openai\u002Fgpt-oss-20b` | 1000 t\u002Fs | 131,072 | 65,536 | $0.075 | $0.30 |\n| Whisper V3 | `whisper-large-v3` | - | - | - | $0.111\u002Fhr | - |\n| Whisper V3 Turbo | `whisper-large-v3-turbo` | - | - | - | $0.04\u002Fhr | - |\n\n### Preview Models\n\n| Model | Model ID | Speed | Context | Input $\u002F1M | Output $\u002F1M |\n|-------|----------|-------|---------|------------|-------------|\n| Llama 4 Scout | `meta-llama\u002Fllama-4-scout-17b-16e-instruct` | 750 t\u002Fs | 131,072 | $0.11 | $0.34 |\n| Llama 4 Maverick | `meta-llama\u002Fllama-4-maverick-17b-128e-instruct` | 600 t\u002Fs | 131,072 | $0.20 | $0.60 |\n| Qwen3 32B | `qwen\u002Fqwen3-32b` | 400 t\u002Fs | 131,072 | $0.29 | $0.59 |\n| Kimi K2 | `moonshotai\u002Fkimi-k2-instruct-0905` | 200 t\u002Fs | 262,144 | $1.00 | $3.00 |\n| Safety GPT-OSS 20B | `openai\u002Fgpt-oss-safeguard-20b` | 1000 t\u002Fs | 131,072 | $0.075 | $0.30 |\n| Llama Guard 4 12B | `meta-llama\u002Fllama-guard-4-12b` | - | 131,072 | $0.20 | $0.20 |\n\n### Systems & TTS\n\n| Model | Model ID | Notes |\n|-------|----------|-------|\n| Compound | `groq\u002Fcompound` | Agentic: web search + code exec |\n| Compound Mini | `groq\u002Fcompound-mini` | Lighter agentic system |\n| PlayAI TTS | `playai-tts` | $22\u002F1M chars |\n| Orpheus English | `canopylabs\u002Forpheus-v1-english` | $22\u002F1M chars |\n\n### Selection Guide\n\n- **Fastest + cheapest**: `llama-3.1-8b-instant` — classification, extraction, simple tasks\n- **Best quality**: `llama-3.3-70b-versatile` — complex reasoning, coding, chat\n- **Best value**: `openai\u002Fgpt-oss-120b` — strong quality, 500 t\u002Fs, $0.15 input\n- **Vision**: `meta-llama\u002Fllama-4-scout-17b-16e-instruct` — 750 t\u002Fs, cheapest multimodal\n- **Long context**: `moonshotai\u002Fkimi-k2-instruct-0905` — 262K context window\n- **Safety**: `meta-llama\u002Fllama-guard-4-12b` or `openai\u002Fgpt-oss-safeguard-20b`\n\n## OpenAI Compatibility\n\nDrop-in replacement — change `base_url` and `api_key`:\n\n```python\nfrom openai import OpenAI\nimport os\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.groq.com\u002Fopenai\u002Fv1\",\n    api_key=os.environ.get(\"GROQ_API_KEY\"),\n)\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello from OpenAI SDK!\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n| Endpoint | Supported | Notes |\n|----------|-----------|-------|\n| `POST \u002Fchat\u002Fcompletions` | Yes | Streaming, tools, JSON mode |\n| `POST \u002Faudio\u002Ftranscriptions` | Yes | Whisper |\n| `POST \u002Faudio\u002Ftranslations` | Yes | Whisper |\n| `POST \u002Faudio\u002Fspeech` | Yes | TTS |\n| `GET \u002Fmodels` | Yes | List models |\n| `POST \u002Fembeddings` | No | Not available |\n| `POST \u002Fimages\u002Fgenerations` | No | Not available |\n\n**Unsupported fields** (400 error): `logprobs`, `logit_bias`, `top_logprobs`, `messages[].name`, `n` (must be 1).\n\n## Rate Limits\n\n| Tier | Price | RPM | TPM | TPD |\n|------|-------|-----|-----|-----|\n| Free | $0 | 30 | 6,000 | 500,000 |\n| Developer | Pay-per-token | Up to 1,000 | Up to 300,000 | Unlimited |\n\nRate limit headers returned with every response: `x-ratelimit-limit-requests`, `x-ratelimit-remaining-requests`, `x-ratelimit-limit-tokens`, `x-ratelimit-remaining-tokens`.\n\n```python\nimport time\nfrom groq import Groq, RateLimitError\n\nclient = Groq()\n\ndef call_with_retry(messages, model=\"llama-3.3-70b-versatile\", max_retries=3):\n    for attempt in range(max_retries):\n        try:\n            return client.chat.completions.create(model=model, messages=messages)\n        except RateLimitError:\n            if attempt == max_retries - 1:\n                raise\n            time.sleep(2 ** attempt)\n```\n\n## Cost Optimization\n\n1. **Use the smallest model that works.** `llama-3.1-8b-instant` is 10x cheaper than the 70B for simple tasks.\n2. **Cap output with `max_completion_tokens`.** Output tokens cost more than input.\n3. **Try `openai\u002Fgpt-oss-20b`** — best price\u002Fperformance at $0.075 input and 1000 t\u002Fs.\n4. **Cache at `temperature=0`.** Deterministic output means identical prompts yield identical results.\n5. **Use Whisper Turbo.** `whisper-large-v3-turbo` is 64% cheaper than v3 with minimal quality loss.\n6. **Preprocess audio** to 16KHz mono WAV to reduce upload size.\n\n## Common Issues\n\n| Problem | Solution |\n|---------|----------|\n| `401 Unauthorized` | Check `GROQ_API_KEY`. Get key at https:\u002F\u002Fconsole.groq.com\u002Fkeys |\n| `400 Bad Request` | Remove unsupported fields: `logprobs`, `logit_bias`, `top_logprobs` |\n| `429 Rate Limited` | Exponential backoff. Upgrade to developer tier for 10x limits |\n| Empty stream chunks | Check `chunk.choices[0].delta.content` for `None` |\n| Invalid JSON output | Add \"Output valid JSON only\" to system prompt, set `temperature=0` |\n| Tool calls missing | Not all models support tools. Use Llama 3.3 70B or GPT-OSS |\n| Audio too large | Free: 25 MB max. Use `url` param or upgrade to dev tier (100 MB) |\n| Vision fails | Only Llama 4 models support vision. Max 20 MB |\n| `n` param error | Groq only supports `n=1` |\n| `max_tokens` warning | Use `max_completion_tokens` instead |\n\n## Resources\n\n- **API Docs**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Foverview\n- **Models**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fmodels\n- **API Keys**: https:\u002F\u002Fconsole.groq.com\u002Fkeys\n- **Pricing**: https:\u002F\u002Fgroq.com\u002Fpricing\n- **Rate Limits**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Frate-limits\n- **OpenAI Compat**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fopenai\n- **Tool Use**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Ftool-use\u002Flocal-tool-calling\n- **Vision**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fvision\n- **Speech-to-Text**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fspeech-to-text\n- **Structured Outputs**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fstructured-outputs\n- **Python SDK**: https:\u002F\u002Fgithub.com\u002Fgroq\u002Fgroq-python (`pip install groq`)\n- **Status**: https:\u002F\u002Fstatus.groq.com\n- **LPU Architecture**: https:\u002F\u002Fgroq.com\u002Fblog\u002Finside-the-lpu-deconstructing-groq-speed\n","# Groq — Ultra-Fast LLM Inference\n\nGroq runs open-weight LLMs on custom LPU (Language Processing Unit) hardware, delivering the lowest inference latency in the industry — up to 1000+ tokens\u002Fsec. OpenAI-compatible API, free tier, pay-per-token pricing. Inference only, no training.\n\n## When to Use Groq\n\n**Use when:**\n- You need the fastest possible inference latency (real-time chat, agents, interactive apps)\n- You want open-weight models (Llama, Qwen, DeepSeek) without managing GPUs\n- You need an OpenAI-compatible drop-in replacement with open models\n- You want free-tier access for prototyping\n- You need fast Whisper transcription or vision with Llama 4\n\n**Don't use when:**\n- You need to fine-tune or train models (inference-only)\n- You need proprietary models (GPT-4o, Claude, Gemini)\n- You need embeddings or image generation APIs (not offered)\n\n| Feature | Groq | Together AI | Fireworks | Replicate |\n|---------|------|-------------|-----------|-----------|\n| Hardware | Custom LPU | NVIDIA GPUs | NVIDIA GPUs | NVIDIA GPUs |\n| Speed (Llama 70B) | ~280 tok\u002Fs | ~100 tok\u002Fs | ~100 tok\u002Fs | ~50 tok\u002Fs |\n| Free tier | Yes | Yes | Yes | No |\n| Training | No | Yes | Yes | Yes |\n\n\n## Credential Setup\n\nCredentials are auto-injected by openscience when connected via the dashboard.\n\n```bash\n# Verify credentials\n[ -n \"$GROQ_API_KEY\" ] && echo \"GROQ_API_KEY set\" || echo \"NOT SET\"\n```\n\nIf not set: connect Groq at https:\u002F\u002Fapp.syntheticsciences.ai -> Services, then restart openscience.\n\n## Quick Start\n\n```bash\npip install groq\nexport GROQ_API_KEY=\"gsk_...\"   # https:\u002F\u002Fconsole.groq.com\u002Fkeys\n```\n\n```python\nfrom groq import Groq\n\nclient = Groq()\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain quantum computing in one paragraph.\"}\n    ],\n    temperature=0.7,\n    max_completion_tokens=512,\n)\nprint(response.choices[0].message.content)\n```\n\n## Chat Completions\n\n### Streaming\n\n```python\nfrom groq import Groq\n\nclient = Groq()\nstream = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain fast inference\"}],\n    stream=True,\n)\nfor chunk in stream:\n    content = chunk.choices[0].delta.content\n    if content:\n        print(content, end=\"\")\n```\n\n### JSON Mode\n\n```python\nfrom groq import Groq\nimport json\n\nclient = Groq()\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"Output valid JSON only.\"},\n        {\"role\": \"user\", \"content\": \"List 3 programming languages with their paradigms.\"}\n    ],\n    response_format={\"type\": \"json_object\"},\n    temperature=0,\n)\ndata = json.loads(response.choices[0].message.content)\n```\n\n### Tool \u002F Function Calling\n\n```python\nfrom groq import Groq\nimport json\n\nclient = Groq()\ntools = [{\n    \"type\": \"function\",\n    \"function\": {\n        \"name\": \"get_weather\",\n        \"description\": \"Get weather for a location\",\n        \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n                \"location\": {\"type\": \"string\", \"description\": \"City name\"},\n                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]}\n            },\n            \"required\": [\"location\"],\n            \"additionalProperties\": False\n        }\n    }\n}]\n\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[{\"role\": \"user\", \"content\": \"Weather in San Francisco?\"}],\n    tools=tools,\n    tool_choice=\"auto\",\n)\n\nmessage = response.choices[0].message\nif message.tool_calls:\n    call = message.tool_calls[0]\n    args = json.loads(call.function.arguments)\n    # Execute function, then send result back as role=\"tool\" message\n    result = {\"temperature\": 62, \"condition\": \"foggy\"}\n    followup = client.chat.completions.create(\n        model=\"llama-3.3-70b-versatile\",\n        messages=[\n            {\"role\": \"user\", \"content\": \"Weather in San Francisco?\"},\n            message,\n            {\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": json.dumps(result)}\n        ],\n    )\n    print(followup.choices[0].message.content)\n```\n\n## Vision\n\nLlama 4 models support image input via URL or base64. Max image size: 20 MB.\n\n```python\nfrom groq import Groq\n\nclient = Groq()\nresponse = client.chat.completions.create(\n    model=\"meta-llama\u002Fllama-4-scout-17b-16e-instruct\",\n    messages=[{\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"text\", \"text\": \"Describe this image.\"},\n            {\"type\": \"image_url\", \"image_url\": {\"url\": \"https:\u002F\u002Fexample.com\u002Fphoto.jpg\"}}\n        ]\n    }],\n    max_completion_tokens=1024,\n)\nprint(response.choices[0].message.content)\n```\n\nVision models: `meta-llama\u002Fllama-4-scout-17b-16e-instruct`, `meta-llama\u002Fllama-4-maverick-17b-128e-instruct`.\n\n## Audio \u002F Speech-to-Text\n\nWhisper on LPU for near-instant transcription. Formats: flac, mp3, mp4, mpeg, mpga, m4a, ogg, wav, webm. Max: 25 MB free \u002F 100 MB developer tier.\n\n```python\nfrom groq import Groq\n\nclient = Groq()\n\n# Transcription\nwith open(\"audio.mp3\", \"rb\") as f:\n    result = client.audio.transcriptions.create(\n        file=(\"audio.mp3\", f),\n        model=\"whisper-large-v3-turbo\",\n        language=\"en\",\n        response_format=\"verbose_json\",\n    )\nprint(result.text)\n\n# Translation (any language -> English)\nwith open(\"french.mp3\", \"rb\") as f:\n    result = client.audio.translations.create(\n        file=(\"french.mp3\", f), model=\"whisper-large-v3\",\n    )\nprint(result.text)\n```\n\n### Text-to-Speech\n\n```python\nfrom groq import Groq\nfrom pathlib import Path\n\nclient = Groq()\nspeech = client.audio.speech.create(\n    model=\"playai-tts\",\n    voice=\"Arista-PlayAI\",\n    input=\"Hello from Groq!\",\n)\nPath(\"output.mp3\").write_bytes(speech.content)\n```\n\n## Model Selection\n\n### Production Models\n\n| Model | Model ID | Speed | Context | Max Output | Input $\u002F1M | Output $\u002F1M |\n|-------|----------|-------|---------|------------|------------|-------------|\n| Llama 3.1 8B | `llama-3.1-8b-instant` | 560 t\u002Fs | 131,072 | 131,072 | $0.05 | $0.08 |\n| Llama 3.3 70B | `llama-3.3-70b-versatile` | 280 t\u002Fs | 131,072 | 32,768 | $0.59 | $0.79 |\n| GPT-OSS 120B | `openai\u002Fgpt-oss-120b` | 500 t\u002Fs | 131,072 | 65,536 | $0.15 | $0.60 |\n| GPT-OSS 20B | `openai\u002Fgpt-oss-20b` | 1000 t\u002Fs | 131,072 | 65,536 | $0.075 | $0.30 |\n| Whisper V3 | `whisper-large-v3` | - | - | - | $0.111\u002Fhr | - |\n| Whisper V3 Turbo | `whisper-large-v3-turbo` | - | - | - | $0.04\u002Fhr | - |\n\n### Preview Models\n\n| Model | Model ID | Speed | Context | Input $\u002F1M | Output $\u002F1M |\n|-------|----------|-------|---------|------------|-------------|\n| Llama 4 Scout | `meta-llama\u002Fllama-4-scout-17b-16e-instruct` | 750 t\u002Fs | 131,072 | $0.11 | $0.34 |\n| Llama 4 Maverick | `meta-llama\u002Fllama-4-maverick-17b-128e-instruct` | 600 t\u002Fs | 131,072 | $0.20 | $0.60 |\n| Qwen3 32B | `qwen\u002Fqwen3-32b` | 400 t\u002Fs | 131,072 | $0.29 | $0.59 |\n| Kimi K2 | `moonshotai\u002Fkimi-k2-instruct-0905` | 200 t\u002Fs | 262,144 | $1.00 | $3.00 |\n| Safety GPT-OSS 20B | `openai\u002Fgpt-oss-safeguard-20b` | 1000 t\u002Fs | 131,072 | $0.075 | $0.30 |\n| Llama Guard 4 12B | `meta-llama\u002Fllama-guard-4-12b` | - | 131,072 | $0.20 | $0.20 |\n\n### Systems & TTS\n\n| Model | Model ID | Notes |\n|-------|----------|-------|\n| Compound | `groq\u002Fcompound` | Agentic: web search + code exec |\n| Compound Mini | `groq\u002Fcompound-mini` | Lighter agentic system |\n| PlayAI TTS | `playai-tts` | $22\u002F1M chars |\n| Orpheus English | `canopylabs\u002Forpheus-v1-english` | $22\u002F1M chars |\n\n### Selection Guide\n\n- **Fastest + cheapest**: `llama-3.1-8b-instant` — classification, extraction, simple tasks\n- **Best quality**: `llama-3.3-70b-versatile` — complex reasoning, coding, chat\n- **Best value**: `openai\u002Fgpt-oss-120b` — strong quality, 500 t\u002Fs, $0.15 input\n- **Vision**: `meta-llama\u002Fllama-4-scout-17b-16e-instruct` — 750 t\u002Fs, cheapest multimodal\n- **Long context**: `moonshotai\u002Fkimi-k2-instruct-0905` — 262K context window\n- **Safety**: `meta-llama\u002Fllama-guard-4-12b` or `openai\u002Fgpt-oss-safeguard-20b`\n\n## OpenAI Compatibility\n\nDrop-in replacement — change `base_url` and `api_key`:\n\n```python\nfrom openai import OpenAI\nimport os\n\nclient = OpenAI(\n    base_url=\"https:\u002F\u002Fapi.groq.com\u002Fopenai\u002Fv1\",\n    api_key=os.environ.get(\"GROQ_API_KEY\"),\n)\nresponse = client.chat.completions.create(\n    model=\"llama-3.3-70b-versatile\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello from OpenAI SDK!\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n| Endpoint | Supported | Notes |\n|----------|-----------|-------|\n| `POST \u002Fchat\u002Fcompletions` | Yes | Streaming, tools, JSON mode |\n| `POST \u002Faudio\u002Ftranscriptions` | Yes | Whisper |\n| `POST \u002Faudio\u002Ftranslations` | Yes | Whisper |\n| `POST \u002Faudio\u002Fspeech` | Yes | TTS |\n| `GET \u002Fmodels` | Yes | List models |\n| `POST \u002Fembeddings` | No | Not available |\n| `POST \u002Fimages\u002Fgenerations` | No | Not available |\n\n**Unsupported fields** (400 error): `logprobs`, `logit_bias`, `top_logprobs`, `messages[].name`, `n` (must be 1).\n\n## Rate Limits\n\n| Tier | Price | RPM | TPM | TPD |\n|------|-------|-----|-----|-----|\n| Free | $0 | 30 | 6,000 | 500,000 |\n| Developer | Pay-per-token | Up to 1,000 | Up to 300,000 | Unlimited |\n\nRate limit headers returned with every response: `x-ratelimit-limit-requests`, `x-ratelimit-remaining-requests`, `x-ratelimit-limit-tokens`, `x-ratelimit-remaining-tokens`.\n\n```python\nimport time\nfrom groq import Groq, RateLimitError\n\nclient = Groq()\n\ndef call_with_retry(messages, model=\"llama-3.3-70b-versatile\", max_retries=3):\n    for attempt in range(max_retries):\n        try:\n            return client.chat.completions.create(model=model, messages=messages)\n        except RateLimitError:\n            if attempt == max_retries - 1:\n                raise\n            time.sleep(2 ** attempt)\n```\n\n## Cost Optimization\n\n1. **Use the smallest model that works.** `llama-3.1-8b-instant` is 10x cheaper than the 70B for simple tasks.\n2. **Cap output with `max_completion_tokens`.** Output tokens cost more than input.\n3. **Try `openai\u002Fgpt-oss-20b`** — best price\u002Fperformance at $0.075 input and 1000 t\u002Fs.\n4. **Cache at `temperature=0`.** Deterministic output means identical prompts yield identical results.\n5. **Use Whisper Turbo.** `whisper-large-v3-turbo` is 64% cheaper than v3 with minimal quality loss.\n6. **Preprocess audio** to 16KHz mono WAV to reduce upload size.\n\n## Common Issues\n\n| Problem | Solution |\n|---------|----------|\n| `401 Unauthorized` | Check `GROQ_API_KEY`. Get key at https:\u002F\u002Fconsole.groq.com\u002Fkeys |\n| `400 Bad Request` | Remove unsupported fields: `logprobs`, `logit_bias`, `top_logprobs` |\n| `429 Rate Limited` | Exponential backoff. Upgrade to developer tier for 10x limits |\n| Empty stream chunks | Check `chunk.choices[0].delta.content` for `None` |\n| Invalid JSON output | Add \"Output valid JSON only\" to system prompt, set `temperature=0` |\n| Tool calls missing | Not all models support tools. Use Llama 3.3 70B or GPT-OSS |\n| Audio too large | Free: 25 MB max. Use `url` param or upgrade to dev tier (100 MB) |\n| Vision fails | Only Llama 4 models support vision. Max 20 MB |\n| `n` param error | Groq only supports `n=1` |\n| `max_tokens` warning | Use `max_completion_tokens` instead |\n\n## Resources\n\n- **API Docs**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Foverview\n- **Models**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fmodels\n- **API Keys**: https:\u002F\u002Fconsole.groq.com\u002Fkeys\n- **Pricing**: https:\u002F\u002Fgroq.com\u002Fpricing\n- **Rate Limits**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Frate-limits\n- **OpenAI Compat**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fopenai\n- **Tool Use**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Ftool-use\u002Flocal-tool-calling\n- **Vision**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fvision\n- **Speech-to-Text**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fspeech-to-text\n- **Structured Outputs**: https:\u002F\u002Fconsole.groq.com\u002Fdocs\u002Fstructured-outputs\n- **Python SDK**: https:\u002F\u002Fgithub.com\u002Fgroq\u002Fgroq-python (`pip install groq`)\n- **Status**: https:\u002F\u002Fstatus.groq.com\n- **LPU Architecture**: https:\u002F\u002Fgroq.com\u002Fblog\u002Finside-the-lpu-deconstructing-groq-speed\n",{"name":5,"tags":49,"author":54,"license":15,"version":55,"category":56,"description":7,"dependencies":57},[50,51,29,28,52,53],"Inference","Groq","OpenAI-Compatible","Free-Tier","Synthetic Sciences","1.0.0","ml-inference",[25,42],"main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fgroq","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add fadrienne\u002FMyGitHub --skill groq-inference",{"tool":70,"command":71},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill groq-inference",{"tool":73,"command":74},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill groq-inference","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fgroq\u002FSKILL.md",true,[],[79,109,136,169,196,223],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":88,"publishedAt":44},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[85],{"categoryNo":86,"categoryName":87,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108],"power","statistical power","effect","scipy","regression","choose","design","uv","when","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":110,"name":111,"nameZh":6,"description":112,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":113,"categories":114,"keywords":118,"publishedAt":44},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[115],{"categoryNo":116,"categoryName":117,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[100,119,89,120,121,122,97,123,124,125,126,127,128,129,130,131,132,133,134,135],"statistical analysis","academic research","implementing","selection","you","reporting","data","for","tests","use","with","test","research","results","statsmodels","models","help",{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":151,"publishedAt":44},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[142,145,146],{"categoryNo":143,"categoryName":144,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":86,"categoryName":87,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":147,"categoryName":148,"taxonomyType":149,"taxonomyTypeDesc":150},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[152,153,154,155,156,157,158,159,160,161,98,40,162,163,164,165,166,167,124,168],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":170,"name":171,"nameZh":6,"description":172,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":173,"categories":174,"keywords":179,"publishedAt":44},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[175,178],{"categoryNo":176,"categoryName":177,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":86,"categoryName":87,"taxonomyType":21,"taxonomyTypeDesc":22},[180,153,181,182,183,184,185,186,187,160,97,188,189,190,191,192,193,194,126,195],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":197,"name":198,"nameZh":6,"description":199,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":200,"categories":201,"keywords":208,"publishedAt":44},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[202,205],{"categoryNo":203,"categoryName":204,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":206,"categoryName":207,"taxonomyType":149,"taxonomyTypeDesc":150},"01JBSEEDCAT000000000000030","开发工程师",[209,210,131,211,212,213,214,96,215,97,98,100,216,217,218,219,220,221,167,222],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":224,"name":225,"nameZh":6,"description":226,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":227,"licenseFileUrl":6,"sourceHtmlUrl":228,"categories":229,"keywords":236,"publishedAt":44},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[230,233,234,235],{"categoryNo":231,"categoryName":232,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":143,"categoryName":144,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":86,"categoryName":87,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":147,"categoryName":148,"taxonomyType":149,"taxonomyTypeDesc":150},[152,237,238,239,240,241,180,242,243,97,98,100,244,245,246,247,248,249,153,167],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]