[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XG83CDY3A1Q57YSJAZF4TA":3,"skill-same:01M6XG83CDY3A1Q57YSJAZF4TA":102},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":64,"commitSha":65,"skillDirectory":66,"forksCount":14,"repositoryHtmlUrl":67,"repositoryHomepageUrl":6,"ownerHtmlUrl":68,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":69,"install":70,"resources":83},"01M6XG83CDY3A1Q57YSJAZF4TA","serving-llms-vllm",null,"Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency\u002Fthroughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ\u002FAWQ\u002FFP8), and tensor parallelism.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"llms","serving","vLLM","serving llms vllm","when","batching","openai-compatible","throughput","GPTQ","AWQ","synthetic","latency","use","api","Transformers","Inference Serving","FP8","openai api","openai","memory","2026-11-10 10:12:30",false,"---\nname: serving-llms-vllm\ndescription: Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency\u002Fthroughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ\u002FAWQ\u002FFP8), and tensor parallelism.\ncategory: ml-inference\nversion: 1.0.0\nauthor: Synthetic Sciences\nlicense: MIT\ntags: [vLLM, Inference Serving, PagedAttention, Continuous Batching, High Throughput, Production, OpenAI API, Quantization, Tensor Parallelism]\ndependencies: [vllm, torch, transformers]\n---\n\n# vLLM - High-Performance LLM Serving\n\n## Quick start\n\nvLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill\u002Fdecode requests).\n\n**Installation**:\n```bash\npip install vllm\n```\n\n**Basic offline inference**:\n```python\nfrom vllm import LLM, SamplingParams\n\nllm = LLM(model=\"meta-llama\u002FLlama-3-8B-Instruct\")\nsampling = SamplingParams(temperature=0.7, max_tokens=256)\n\noutputs = llm.generate([\"Explain quantum computing\"], sampling)\nprint(outputs[0].outputs[0].text)\n```\n\n**OpenAI-compatible server**:\n```bash\nvllm serve meta-llama\u002FLlama-3-8B-Instruct\n\n# Query with OpenAI SDK\npython -c \"\nfrom openai import OpenAI\nclient = OpenAI(base_url='http:\u002F\u002Flocalhost:8000\u002Fv1', api_key='EMPTY')\nprint(client.chat.completions.create(\n    model='meta-llama\u002FLlama-3-8B-Instruct',\n    messages=[{'role': 'user', 'content': 'Hello!'}]\n).choices[0].message.content)\n\"\n```\n\n## Common workflows\n\n### Workflow 1: Production API deployment\n\nCopy this checklist and track progress:\n\n```\nDeployment Progress:\n- [ ] Step 1: Configure server settings\n- [ ] Step 2: Test with limited traffic\n- [ ] Step 3: Enable monitoring\n- [ ] Step 4: Deploy to production\n- [ ] Step 5: Verify performance metrics\n```\n\n**Step 1: Configure server settings**\n\nChoose configuration based on your model size:\n\n```bash\n# For 7B-13B models on single GPU\nvllm serve meta-llama\u002FLlama-3-8B-Instruct \\\n  --gpu-memory-utilization 0.9 \\\n  --max-model-len 8192 \\\n  --port 8000\n\n# For 30B-70B models with tensor parallelism\nvllm serve meta-llama\u002FLlama-2-70b-hf \\\n  --tensor-parallel-size 4 \\\n  --gpu-memory-utilization 0.9 \\\n  --quantization awq \\\n  --port 8000\n\n# For production with caching and metrics\nvllm serve meta-llama\u002FLlama-3-8B-Instruct \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-prefix-caching \\\n  --enable-metrics \\\n  --metrics-port 9090 \\\n  --port 8000 \\\n  --host 0.0.0.0\n```\n\n**Step 2: Test with limited traffic**\n\nRun load test before production:\n\n```bash\n# Install load testing tool\npip install locust\n\n# Create test_load.py with sample requests\n# Run: locust -f test_load.py --host http:\u002F\u002Flocalhost:8000\n```\n\nVerify TTFT (time to first token) \u003C 500ms and throughput > 100 req\u002Fsec.\n\n**Step 3: Enable monitoring**\n\nvLLM exposes Prometheus metrics on port 9090:\n\n```bash\ncurl http:\u002F\u002Flocalhost:9090\u002Fmetrics | grep vllm\n```\n\nKey metrics to monitor:\n- `vllm:time_to_first_token_seconds` - Latency\n- `vllm:num_requests_running` - Active requests\n- `vllm:gpu_cache_usage_perc` - KV cache utilization\n\n**Step 4: Deploy to production**\n\nUse Docker for consistent deployment:\n\n```bash\n# Run vLLM in Docker\ndocker run --gpus all -p 8000:8000 \\\n  vllm\u002Fvllm-openai:latest \\\n  --model meta-llama\u002FLlama-3-8B-Instruct \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-prefix-caching\n```\n\n**Step 5: Verify performance metrics**\n\nCheck that deployment meets targets:\n- TTFT \u003C 500ms (for short prompts)\n- Throughput > target req\u002Fsec\n- GPU utilization > 80%\n- No OOM errors in logs\n\n### Workflow 2: Offline batch inference\n\nFor processing large datasets without server overhead.\n\nCopy this checklist:\n\n```\nBatch Processing:\n- [ ] Step 1: Prepare input data\n- [ ] Step 2: Configure LLM engine\n- [ ] Step 3: Run batch inference\n- [ ] Step 4: Process results\n```\n\n**Step 1: Prepare input data**\n\n```python\n# Load prompts from file\nprompts = []\nwith open(\"prompts.txt\") as f:\n    prompts = [line.strip() for line in f]\n\nprint(f\"Loaded {len(prompts)} prompts\")\n```\n\n**Step 2: Configure LLM engine**\n\n```python\nfrom vllm import LLM, SamplingParams\n\nllm = LLM(\n    model=\"meta-llama\u002FLlama-3-8B-Instruct\",\n    tensor_parallel_size=2,  # Use 2 GPUs\n    gpu_memory_utilization=0.9,\n    max_model_len=4096\n)\n\nsampling = SamplingParams(\n    temperature=0.7,\n    top_p=0.95,\n    max_tokens=512,\n    stop=[\"\u003C\u002Fs>\", \"\\n\\n\"]\n)\n```\n\n**Step 3: Run batch inference**\n\nvLLM automatically batches requests for efficiency:\n\n```python\n# Process all prompts in one call\noutputs = llm.generate(prompts, sampling)\n\n# vLLM handles batching internally\n# No need to manually chunk prompts\n```\n\n**Step 4: Process results**\n\n```python\n# Extract generated text\nresults = []\nfor output in outputs:\n    prompt = output.prompt\n    generated = output.outputs[0].text\n    results.append({\n        \"prompt\": prompt,\n        \"generated\": generated,\n        \"tokens\": len(output.outputs[0].token_ids)\n    })\n\n# Save to file\nimport json\nwith open(\"results.jsonl\", \"w\") as f:\n    for result in results:\n        f.write(json.dumps(result) + \"\\n\")\n\nprint(f\"Processed {len(results)} prompts\")\n```\n\n### Workflow 3: Quantized model serving\n\nFit large models in limited GPU memory.\n\n```\nQuantization Setup:\n- [ ] Step 1: Choose quantization method\n- [ ] Step 2: Find or create quantized model\n- [ ] Step 3: Launch with quantization flag\n- [ ] Step 4: Verify accuracy\n```\n\n**Step 1: Choose quantization method**\n\n- **AWQ**: Best for 70B models, minimal accuracy loss\n- **GPTQ**: Wide model support, good compression\n- **FP8**: Fastest on H100 GPUs\n\n**Step 2: Find or create quantized model**\n\nUse pre-quantized models from HuggingFace:\n\n```bash\n# Search for AWQ models\n# Example: TheBloke\u002FLlama-2-70B-AWQ\n```\n\n**Step 3: Launch with quantization flag**\n\n```bash\n# Using pre-quantized model\nvllm serve TheBloke\u002FLlama-2-70B-AWQ \\\n  --quantization awq \\\n  --tensor-parallel-size 1 \\\n  --gpu-memory-utilization 0.95\n\n# Results: 70B model in ~40GB VRAM\n```\n\n**Step 4: Verify accuracy**\n\nTest outputs match expected quality:\n\n```python\n# Compare quantized vs non-quantized responses\n# Verify task-specific performance unchanged\n```\n\n## When to use vs alternatives\n\n**Use vLLM when:**\n- Deploying production LLM APIs (100+ req\u002Fsec)\n- Serving OpenAI-compatible endpoints\n- Limited GPU memory but need large models\n- Multi-user applications (chatbots, assistants)\n- Need low latency with high throughput\n\n**Use alternatives instead:**\n- **llama.cpp**: CPU\u002Fedge inference, single-user\n- **HuggingFace transformers**: Research, prototyping, one-off generation\n- **TensorRT-LLM**: NVIDIA-only, need absolute maximum performance\n- **Text-Generation-Inference**: Already in HuggingFace ecosystem\n\n## Common issues\n\n**Issue: Out of memory during model loading**\n\nReduce memory usage:\n```bash\nvllm serve MODEL \\\n  --gpu-memory-utilization 0.7 \\\n  --max-model-len 4096\n```\n\nOr use quantization:\n```bash\nvllm serve MODEL --quantization awq\n```\n\n**Issue: Slow first token (TTFT > 1 second)**\n\nEnable prefix caching for repeated prompts:\n```bash\nvllm serve MODEL --enable-prefix-caching\n```\n\nFor long prompts, enable chunked prefill:\n```bash\nvllm serve MODEL --enable-chunked-prefill\n```\n\n**Issue: Model not found error**\n\nUse `--trust-remote-code` for custom models:\n```bash\nvllm serve MODEL --trust-remote-code\n```\n\n**Issue: Low throughput (\u003C50 req\u002Fsec)**\n\nIncrease concurrent sequences:\n```bash\nvllm serve MODEL --max-num-seqs 512\n```\n\nCheck GPU utilization with `nvidia-smi` - should be >80%.\n\n**Issue: Inference slower than expected**\n\nVerify tensor parallelism uses power of 2 GPUs:\n```bash\nvllm serve MODEL --tensor-parallel-size 4  # Not 3\n```\n\nEnable speculative decoding for faster generation:\n```bash\nvllm serve MODEL --speculative-model DRAFT_MODEL\n```\n\n## Advanced topics\n\n**Server deployment patterns**: See [references\u002Fserver-deployment.md](references\u002Fserver-deployment.md) for Docker, Kubernetes, and load balancing configurations.\n\n**Performance optimization**: See [references\u002Foptimization.md](references\u002Foptimization.md) for PagedAttention tuning, continuous batching details, and benchmark results.\n\n**Quantization guide**: See [references\u002Fquantization.md](references\u002Fquantization.md) for AWQ\u002FGPTQ\u002FFP8 setup, model preparation, and accuracy comparisons.\n\n**Troubleshooting**: See [references\u002Ftroubleshooting.md](references\u002Ftroubleshooting.md) for detailed error messages, debugging steps, and performance diagnostics.\n\n## Hardware requirements\n\n- **Small models (7B-13B)**: 1x A10 (24GB) or A100 (40GB)\n- **Medium models (30B-40B)**: 2x A100 (40GB) with tensor parallelism\n- **Large models (70B+)**: 4x A100 (40GB) or 2x A100 (80GB), use AWQ\u002FGPTQ\n\nSupported platforms: NVIDIA (primary), AMD ROCm, Intel GPUs, TPUs\n\n\n## Known Conflicts\n\n- **Do not install alongside `sglang`** in the same environment. Both bind to full GPU memory and cannot coexist in the same process. Use separate environments or containers.\n\n## Resources\n\n- Official docs: https:\u002F\u002Fdocs.vllm.ai\n- GitHub: https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\n- Paper: \"Efficient Memory Management for Large Language Model Serving with PagedAttention\" (SOSP 2023)\n- Community: https:\u002F\u002Fdiscuss.vllm.ai\n\n\n\n","# vLLM - High-Performance LLM Serving\n\n## Quick start\n\nvLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill\u002Fdecode requests).\n\n**Installation**:\n```bash\npip install vllm\n```\n\n**Basic offline inference**:\n```python\nfrom vllm import LLM, SamplingParams\n\nllm = LLM(model=\"meta-llama\u002FLlama-3-8B-Instruct\")\nsampling = SamplingParams(temperature=0.7, max_tokens=256)\n\noutputs = llm.generate([\"Explain quantum computing\"], sampling)\nprint(outputs[0].outputs[0].text)\n```\n\n**OpenAI-compatible server**:\n```bash\nvllm serve meta-llama\u002FLlama-3-8B-Instruct\n\n# Query with OpenAI SDK\npython -c \"\nfrom openai import OpenAI\nclient = OpenAI(base_url='http:\u002F\u002Flocalhost:8000\u002Fv1', api_key='EMPTY')\nprint(client.chat.completions.create(\n    model='meta-llama\u002FLlama-3-8B-Instruct',\n    messages=[{'role': 'user', 'content': 'Hello!'}]\n).choices[0].message.content)\n\"\n```\n\n## Common workflows\n\n### Workflow 1: Production API deployment\n\nCopy this checklist and track progress:\n\n```\nDeployment Progress:\n- [ ] Step 1: Configure server settings\n- [ ] Step 2: Test with limited traffic\n- [ ] Step 3: Enable monitoring\n- [ ] Step 4: Deploy to production\n- [ ] Step 5: Verify performance metrics\n```\n\n**Step 1: Configure server settings**\n\nChoose configuration based on your model size:\n\n```bash\n# For 7B-13B models on single GPU\nvllm serve meta-llama\u002FLlama-3-8B-Instruct \\\n  --gpu-memory-utilization 0.9 \\\n  --max-model-len 8192 \\\n  --port 8000\n\n# For 30B-70B models with tensor parallelism\nvllm serve meta-llama\u002FLlama-2-70b-hf \\\n  --tensor-parallel-size 4 \\\n  --gpu-memory-utilization 0.9 \\\n  --quantization awq \\\n  --port 8000\n\n# For production with caching and metrics\nvllm serve meta-llama\u002FLlama-3-8B-Instruct \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-prefix-caching \\\n  --enable-metrics \\\n  --metrics-port 9090 \\\n  --port 8000 \\\n  --host 0.0.0.0\n```\n\n**Step 2: Test with limited traffic**\n\nRun load test before production:\n\n```bash\n# Install load testing tool\npip install locust\n\n# Create test_load.py with sample requests\n# Run: locust -f test_load.py --host http:\u002F\u002Flocalhost:8000\n```\n\nVerify TTFT (time to first token) \u003C 500ms and throughput > 100 req\u002Fsec.\n\n**Step 3: Enable monitoring**\n\nvLLM exposes Prometheus metrics on port 9090:\n\n```bash\ncurl http:\u002F\u002Flocalhost:9090\u002Fmetrics | grep vllm\n```\n\nKey metrics to monitor:\n- `vllm:time_to_first_token_seconds` - Latency\n- `vllm:num_requests_running` - Active requests\n- `vllm:gpu_cache_usage_perc` - KV cache utilization\n\n**Step 4: Deploy to production**\n\nUse Docker for consistent deployment:\n\n```bash\n# Run vLLM in Docker\ndocker run --gpus all -p 8000:8000 \\\n  vllm\u002Fvllm-openai:latest \\\n  --model meta-llama\u002FLlama-3-8B-Instruct \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-prefix-caching\n```\n\n**Step 5: Verify performance metrics**\n\nCheck that deployment meets targets:\n- TTFT \u003C 500ms (for short prompts)\n- Throughput > target req\u002Fsec\n- GPU utilization > 80%\n- No OOM errors in logs\n\n### Workflow 2: Offline batch inference\n\nFor processing large datasets without server overhead.\n\nCopy this checklist:\n\n```\nBatch Processing:\n- [ ] Step 1: Prepare input data\n- [ ] Step 2: Configure LLM engine\n- [ ] Step 3: Run batch inference\n- [ ] Step 4: Process results\n```\n\n**Step 1: Prepare input data**\n\n```python\n# Load prompts from file\nprompts = []\nwith open(\"prompts.txt\") as f:\n    prompts = [line.strip() for line in f]\n\nprint(f\"Loaded {len(prompts)} prompts\")\n```\n\n**Step 2: Configure LLM engine**\n\n```python\nfrom vllm import LLM, SamplingParams\n\nllm = LLM(\n    model=\"meta-llama\u002FLlama-3-8B-Instruct\",\n    tensor_parallel_size=2,  # Use 2 GPUs\n    gpu_memory_utilization=0.9,\n    max_model_len=4096\n)\n\nsampling = SamplingParams(\n    temperature=0.7,\n    top_p=0.95,\n    max_tokens=512,\n    stop=[\"\u003C\u002Fs>\", \"\\n\\n\"]\n)\n```\n\n**Step 3: Run batch inference**\n\nvLLM automatically batches requests for efficiency:\n\n```python\n# Process all prompts in one call\noutputs = llm.generate(prompts, sampling)\n\n# vLLM handles batching internally\n# No need to manually chunk prompts\n```\n\n**Step 4: Process results**\n\n```python\n# Extract generated text\nresults = []\nfor output in outputs:\n    prompt = output.prompt\n    generated = output.outputs[0].text\n    results.append({\n        \"prompt\": prompt,\n        \"generated\": generated,\n        \"tokens\": len(output.outputs[0].token_ids)\n    })\n\n# Save to file\nimport json\nwith open(\"results.jsonl\", \"w\") as f:\n    for result in results:\n        f.write(json.dumps(result) + \"\\n\")\n\nprint(f\"Processed {len(results)} prompts\")\n```\n\n### Workflow 3: Quantized model serving\n\nFit large models in limited GPU memory.\n\n```\nQuantization Setup:\n- [ ] Step 1: Choose quantization method\n- [ ] Step 2: Find or create quantized model\n- [ ] Step 3: Launch with quantization flag\n- [ ] Step 4: Verify accuracy\n```\n\n**Step 1: Choose quantization method**\n\n- **AWQ**: Best for 70B models, minimal accuracy loss\n- **GPTQ**: Wide model support, good compression\n- **FP8**: Fastest on H100 GPUs\n\n**Step 2: Find or create quantized model**\n\nUse pre-quantized models from HuggingFace:\n\n```bash\n# Search for AWQ models\n# Example: TheBloke\u002FLlama-2-70B-AWQ\n```\n\n**Step 3: Launch with quantization flag**\n\n```bash\n# Using pre-quantized model\nvllm serve TheBloke\u002FLlama-2-70B-AWQ \\\n  --quantization awq \\\n  --tensor-parallel-size 1 \\\n  --gpu-memory-utilization 0.95\n\n# Results: 70B model in ~40GB VRAM\n```\n\n**Step 4: Verify accuracy**\n\nTest outputs match expected quality:\n\n```python\n# Compare quantized vs non-quantized responses\n# Verify task-specific performance unchanged\n```\n\n## When to use vs alternatives\n\n**Use vLLM when:**\n- Deploying production LLM APIs (100+ req\u002Fsec)\n- Serving OpenAI-compatible endpoints\n- Limited GPU memory but need large models\n- Multi-user applications (chatbots, assistants)\n- Need low latency with high throughput\n\n**Use alternatives instead:**\n- **llama.cpp**: CPU\u002Fedge inference, single-user\n- **HuggingFace transformers**: Research, prototyping, one-off generation\n- **TensorRT-LLM**: NVIDIA-only, need absolute maximum performance\n- **Text-Generation-Inference**: Already in HuggingFace ecosystem\n\n## Common issues\n\n**Issue: Out of memory during model loading**\n\nReduce memory usage:\n```bash\nvllm serve MODEL \\\n  --gpu-memory-utilization 0.7 \\\n  --max-model-len 4096\n```\n\nOr use quantization:\n```bash\nvllm serve MODEL --quantization awq\n```\n\n**Issue: Slow first token (TTFT > 1 second)**\n\nEnable prefix caching for repeated prompts:\n```bash\nvllm serve MODEL --enable-prefix-caching\n```\n\nFor long prompts, enable chunked prefill:\n```bash\nvllm serve MODEL --enable-chunked-prefill\n```\n\n**Issue: Model not found error**\n\nUse `--trust-remote-code` for custom models:\n```bash\nvllm serve MODEL --trust-remote-code\n```\n\n**Issue: Low throughput (\u003C50 req\u002Fsec)**\n\nIncrease concurrent sequences:\n```bash\nvllm serve MODEL --max-num-seqs 512\n```\n\nCheck GPU utilization with `nvidia-smi` - should be >80%.\n\n**Issue: Inference slower than expected**\n\nVerify tensor parallelism uses power of 2 GPUs:\n```bash\nvllm serve MODEL --tensor-parallel-size 4  # Not 3\n```\n\nEnable speculative decoding for faster generation:\n```bash\nvllm serve MODEL --speculative-model DRAFT_MODEL\n```\n\n## Advanced topics\n\n**Server deployment patterns**: See [references\u002Fserver-deployment.md](references\u002Fserver-deployment.md) for Docker, Kubernetes, and load balancing configurations.\n\n**Performance optimization**: See [references\u002Foptimization.md](references\u002Foptimization.md) for PagedAttention tuning, continuous batching details, and benchmark results.\n\n**Quantization guide**: See [references\u002Fquantization.md](references\u002Fquantization.md) for AWQ\u002FGPTQ\u002FFP8 setup, model preparation, and accuracy comparisons.\n\n**Troubleshooting**: See [references\u002Ftroubleshooting.md](references\u002Ftroubleshooting.md) for detailed error messages, debugging steps, and performance diagnostics.\n\n## Hardware requirements\n\n- **Small models (7B-13B)**: 1x A10 (24GB) or A100 (40GB)\n- **Medium models (30B-40B)**: 2x A100 (40GB) with tensor parallelism\n- **Large models (70B+)**: 4x A100 (40GB) or 2x A100 (80GB), use AWQ\u002FGPTQ\n\nSupported platforms: NVIDIA (primary), AMD ROCm, Intel GPUs, TPUs\n\n\n## Known Conflicts\n\n- **Do not install alongside `sglang`** in the same environment. Both bind to full GPU memory and cannot coexist in the same process. Use separate environments or containers.\n\n## Resources\n\n- Official docs: https:\u002F\u002Fdocs.vllm.ai\n- GitHub: https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm\n- Paper: \"Efficient Memory Management for Large Language Model Serving with PagedAttention\" (SOSP 2023)\n- Community: https:\u002F\u002Fdiscuss.vllm.ai\n\n\n\n",{"name":5,"tags":49,"author":57,"license":15,"version":58,"category":59,"description":7,"dependencies":60},[26,39,50,51,52,53,54,55,56],"PagedAttention","Continuous Batching","High Throughput","Production","OpenAI API","Quantization","Tensor Parallelism","Synthetic Sciences","1.0.0","ml-inference",[61,62,63],"vllm","torch","transformers","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":71,"prompt":81,"zipAvailable":82},[72,75,78],{"tool":73,"command":74},"npx","npx skills add fadrienne\u002FMyGitHub --skill serving-llms-vllm",{"tool":76,"command":77},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill serving-llms-vllm",{"tool":79,"command":80},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill serving-llms-vllm","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm\u002FSKILL.md",true,[84,90,94,98],{"path":85,"resourceType":86,"resourceTypeDesc":87,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002Foptimization.md","REFERENCE","参考文档",5769,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm\u002Freferences\u002Foptimization.md",{"path":91,"resourceType":86,"resourceTypeDesc":87,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Fquantization.md",6746,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm\u002Freferences\u002Fquantization.md",{"path":95,"resourceType":86,"resourceTypeDesc":87,"sizeBytes":96,"sourceHtmlUrl":97},"references\u002Fserver-deployment.md",5252,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm\u002Freferences\u002Fserver-deployment.md",{"path":99,"resourceType":86,"resourceTypeDesc":87,"sizeBytes":100,"sourceHtmlUrl":101},"references\u002Ftroubleshooting.md",9037,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-inference\u002Fvllm\u002Freferences\u002Ftroubleshooting.md",[103,132,158,191,218,244],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":112,"publishedAt":44},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[109],{"categoryNo":110,"categoryName":111,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[113,114,115,116,117,118,119,120,28,121,122,123,124,125,126,127,128,129,130,131],"power","statistical power","effect","scipy","regression","choose","design","uv","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":133,"name":134,"nameZh":6,"description":135,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":136,"categories":137,"keywords":141,"publishedAt":44},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[138],{"categoryNo":139,"categoryName":140,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[123,142,113,143,144,145,28,146,147,148,149,150,36,151,152,153,154,155,156,157],"statistical analysis","academic research","implementing","selection","you","reporting","data","for","tests","with","test","research","results","statsmodels","models","help",{"skillNo":159,"name":160,"nameZh":6,"description":161,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":162,"categories":163,"keywords":173,"publishedAt":44},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[164,167,168],{"categoryNo":165,"categoryName":166,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":110,"categoryName":111,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":169,"categoryName":170,"taxonomyType":171,"taxonomyTypeDesc":172},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[174,175,176,177,178,179,180,181,182,183,121,37,184,185,186,187,188,189,147,190],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":192,"name":193,"nameZh":6,"description":194,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":195,"categories":196,"keywords":201,"publishedAt":44},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[197,200],{"categoryNo":198,"categoryName":199,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":110,"categoryName":111,"taxonomyType":21,"taxonomyTypeDesc":22},[202,175,203,204,205,206,207,208,209,182,28,210,211,212,213,214,215,216,149,217],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":219,"name":220,"nameZh":6,"description":221,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":222,"categories":223,"keywords":230,"publishedAt":44},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[224,227],{"categoryNo":225,"categoryName":226,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":228,"categoryName":229,"taxonomyType":171,"taxonomyTypeDesc":172},"01JBSEEDCAT000000000000030","开发工程师",[231,232,153,233,43,234,235,120,236,28,121,123,237,238,239,240,241,242,189,243],"code","research code","which","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":245,"name":246,"nameZh":6,"description":247,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":248,"licenseFileUrl":6,"sourceHtmlUrl":249,"categories":250,"keywords":257,"publishedAt":44},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[251,254,255,256],{"categoryNo":252,"categoryName":253,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":165,"categoryName":166,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":110,"categoryName":111,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":169,"categoryName":170,"taxonomyType":171,"taxonomyTypeDesc":172},[174,258,259,260,261,262,202,263,264,28,121,123,265,266,267,268,269,270,175,189],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]