[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73A8BPC4EV6F8XG0H58H5H9":3,"skill-same:01M73A8BPC4EV6F8XG0H58H5H9":83},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":46,"repositoryHtmlUrl":47,"repositoryHomepageUrl":6,"ownerHtmlUrl":48,"ownerBlogUrl":49,"ownerDescription":6,"updatedAt":50,"install":51,"resources":64},"01M73A8BPC4EV6F8XG0H58H5H9","implementing-llms-litgpt",null,"Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA\u002FQLoRA. Single-file implementations, no abstraction layers.","qhjqhj00\u002Fresearch-skills-pool","qhjqhj00","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F29684092?v=4","01M72AE0ZGXVM5C57N512DGQG8","Tommy Chien","01M72AE107ESJMHT7QC5SXX144",3,"MIT","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: implementing-llms-litgpt\ndescription: Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA\u002FQLoRA. Single-file implementations, no abstraction layers.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, LitGPT, Lightning AI, LLM Implementation, LoRA, QLoRA, Fine-Tuning, Llama, Gemma, Phi, Mistral, Educational]\ndependencies: [litgpt, torch, transformers]\n---\n\n# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft\u002Fphi-2\")\n\n# Generate text\nresult = llm.generate(\n    \"What is the capital of France?\",\n    max_new_tokens=50,\n    temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft\u002Fphi-2\n\n# Download Gemma 2B\nlitgpt download google\u002Fgemma-2b\n```\n\nModels are saved to `checkpoints\u002F` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n  {\n    \"instruction\": \"What is the capital of France?\",\n    \"input\": \"\",\n    \"output\": \"The capital of France is Paris.\"\n  },\n  {\n    \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n    \"input\": \"\",\n    \"output\": \"Hola, ¿cómo estás?\"\n  }\n]\n```\n\nSave as `data\u002Fmy_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n  meta-llama\u002FMeta-Llama-3-8B \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 2e-5 \\\n  --train.micro_batch_size 1 \\\n  --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --lora_alpha 32 \\\n  --lora_dropout 0.05 \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out\u002Ffinetune\u002F` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out\u002Ffinetune\u002Flogs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out\u002Ffinetune\u002Flightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality\u002Fsize tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\          # LoRA rank (8-64, higher=more capacity)\n  --lora_alpha 32 \\      # LoRA scaling (typically 2×r)\n  --lora_dropout 0.05 \\  # Prevent overfitting\n  --lora_query true \\    # Apply LoRA to query projection\n  --lora_key false \\     # Usually not needed\n  --lora_value true \\    # Apply LoRA to value projection\n  --lora_projection true \\  # Apply LoRA to output projection\n  --lora_mlp false \\     # Usually not needed\n  --lora_head false      # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --train.epochs 3 \\\n  --train.learning_rate 1e-4 \\\n  --train.micro_batch_size 4 \\\n  --train.global_batch_size 32 \\\n  --out_dir out\u002Fphi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --out_dir out\u002Fphi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out\u002Fphi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts\u002Fprepare_dataset.py \\\n  --source_path data\u002Fmy_corpus.txt \\\n  --checkpoint_dir checkpoints\u002Ftokenizer \\\n  --destination_path data\u002Fpretrain \\\n  --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config\u002Fpythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n  --config config\u002Fpythia-160m.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --devices 8 \\\n  --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n  pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir \u002Fshared\u002Fdata\u002Fpretrain \\\n  --devices 8 \\\n  --num_nodes 8 \\\n  --train.global_batch_size 512 \\\n  --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n    print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --dtype bfloat16 \\\n  --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --quantize bnb.nf4-dq  # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts\u002Fconvert_lit_checkpoint.py \\\n  --checkpoint_path out\u002Fphi2-lora\u002Ffinal \\\n  --output_path models\u002Fphi2.gguf \\\n  --model_name microsoft\u002Fphi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n@app.post(\"\u002Fgenerate\")\ndef generate(prompt: str, max_tokens: int = 100):\n    result = llm.generate(\n        prompt,\n        max_new_tokens=max_tokens,\n        temperature=0.7\n    )\n    return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl\u002FTRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora  # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n  ... \\\n  --train.gradient_accumulation_iters 4  # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30\u002F40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32  # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints\u002F\n# Should see: meta-llama\u002FMeta-Llama-3-8B\u002F\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8  # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\  # Disable this\n--lora_mlp false  # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references\u002Fsupported-models.md](references\u002Fsupported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references\u002Ftraining-recipes.md](references\u002Ftraining-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references\u002Fdistributed-training.md](references\u002Fdistributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references\u002Fcustom-models.md](references\u002Fcustom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n  - Inference (Phi-2): 6GB\n  - LoRA fine-tuning (7B): 16GB\n  - Full fine-tuning (7B): 40GB+\n  - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitgpt\n- Docs: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\n- Tutorials: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\u002Ftutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n","# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft\u002Fphi-2\")\n\n# Generate text\nresult = llm.generate(\n    \"What is the capital of France?\",\n    max_new_tokens=50,\n    temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft\u002Fphi-2\n\n# Download Gemma 2B\nlitgpt download google\u002Fgemma-2b\n```\n\nModels are saved to `checkpoints\u002F` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n  {\n    \"instruction\": \"What is the capital of France?\",\n    \"input\": \"\",\n    \"output\": \"The capital of France is Paris.\"\n  },\n  {\n    \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n    \"input\": \"\",\n    \"output\": \"Hola, ¿cómo estás?\"\n  }\n]\n```\n\nSave as `data\u002Fmy_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n  meta-llama\u002FMeta-Llama-3-8B \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 2e-5 \\\n  --train.micro_batch_size 1 \\\n  --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --lora_alpha 32 \\\n  --lora_dropout 0.05 \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out\u002Ffinetune\u002F` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out\u002Ffinetune\u002Flogs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out\u002Ffinetune\u002Flightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality\u002Fsize tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\          # LoRA rank (8-64, higher=more capacity)\n  --lora_alpha 32 \\      # LoRA scaling (typically 2×r)\n  --lora_dropout 0.05 \\  # Prevent overfitting\n  --lora_query true \\    # Apply LoRA to query projection\n  --lora_key false \\     # Usually not needed\n  --lora_value true \\    # Apply LoRA to value projection\n  --lora_projection true \\  # Apply LoRA to output projection\n  --lora_mlp false \\     # Usually not needed\n  --lora_head false      # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --train.epochs 3 \\\n  --train.learning_rate 1e-4 \\\n  --train.micro_batch_size 4 \\\n  --train.global_batch_size 32 \\\n  --out_dir out\u002Fphi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --out_dir out\u002Fphi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out\u002Fphi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts\u002Fprepare_dataset.py \\\n  --source_path data\u002Fmy_corpus.txt \\\n  --checkpoint_dir checkpoints\u002Ftokenizer \\\n  --destination_path data\u002Fpretrain \\\n  --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config\u002Fpythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n  --config config\u002Fpythia-160m.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --devices 8 \\\n  --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n  pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir \u002Fshared\u002Fdata\u002Fpretrain \\\n  --devices 8 \\\n  --num_nodes 8 \\\n  --train.global_batch_size 512 \\\n  --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n    print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --dtype bfloat16 \\\n  --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --quantize bnb.nf4-dq  # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts\u002Fconvert_lit_checkpoint.py \\\n  --checkpoint_path out\u002Fphi2-lora\u002Ffinal \\\n  --output_path models\u002Fphi2.gguf \\\n  --model_name microsoft\u002Fphi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n@app.post(\"\u002Fgenerate\")\ndef generate(prompt: str, max_tokens: int = 100):\n    result = llm.generate(\n        prompt,\n        max_new_tokens=max_tokens,\n        temperature=0.7\n    )\n    return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl\u002FTRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora  # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n  ... \\\n  --train.gradient_accumulation_iters 4  # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30\u002F40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32  # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints\u002F\n# Should see: meta-llama\u002FMeta-Llama-3-8B\u002F\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8  # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\  # Disable this\n--lora_mlp false  # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references\u002Fsupported-models.md](references\u002Fsupported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references\u002Ftraining-recipes.md](references\u002Ftraining-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references\u002Fdistributed-training.md](references\u002Fdistributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references\u002Fcustom-models.md](references\u002Fcustom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n  - Inference (Phi-2): 6GB\n  - LoRA fine-tuning (7B): 16GB\n  - Full fine-tuning (7B): 40GB+\n  - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitgpt\n- Docs: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\n- Tutorials: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\u002Ftutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n",{"name":5,"tags":24,"author":37,"license":15,"version":38,"description":7,"dependencies":39},[25,26,27,28,29,30,31,32,33,34,35,36],"Model Architecture","LitGPT","Lightning AI","LLM Implementation","LoRA","QLoRA","Fine-Tuning","Llama","Gemma","Phi","Mistral","Educational","Orchestra Research","1.0.0",[40,41,42],"litgpt","torch","transformers","master","c247302c836de6438d746b45670990eb71691166","skills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt",1,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool","https:\u002F\u002Fgithub.com\u002Fqhjqhj00","chien.io","2026-05-11 15:07:59",{"source":8,"skillName":5,"commands":52,"prompt":62,"zipAvailable":63},[53,56,59],{"tool":54,"command":55},"npx","npx skills add qhjqhj00\u002Fresearch-skills-pool --skill implementing-llms-litgpt",{"tool":57,"command":58},"bunx","bunx skills add qhjqhj00\u002Fresearch-skills-pool --skill implementing-llms-litgpt",{"tool":60,"command":61},"pnpm","pnpm dlx skills add qhjqhj00\u002Fresearch-skills-pool --skill implementing-llms-litgpt","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt\u002FSKILL.md",true,[65,71,75,79],{"path":66,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":69,"sourceHtmlUrl":70},"references\u002Fcustom-models.md","REFERENCE","参考文档",15655,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt\u002Freferences\u002Fcustom-models.md",{"path":72,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":73,"sourceHtmlUrl":74},"references\u002Fdistributed-training.md",11099,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt\u002Freferences\u002Fdistributed-training.md",{"path":76,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":77,"sourceHtmlUrl":78},"references\u002Fsupported-models.md",7935,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt\u002Freferences\u002Fsupported-models.md",{"path":80,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":81,"sourceHtmlUrl":82},"references\u002Ftraining-recipes.md",10978,"https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Fskills_collected\u002FOrchestra-Research--AI-research-SKILLs--01-model-architecture__litgpt\u002Freferences\u002Ftraining-recipes.md",[84,91,98,105,112,119],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M73JJB6EQPFPPVQK2Y9N67Q0","transformer-lens-interpretability","Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F130-transformer-lens\u002FSKILL.md",[],[],{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":19},"01M73JHTANRRWEE9HP98TPEHAY","wiki-vitepress","Packages generated wiki Markdown into a VitePress static site with dark theme, dark-mode Mermaid diagrams with click-to-zoom, and production build output. Use when the user wants to create a browsable website from generated wiki pages.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F126-wiki-vitepress\u002FSKILL.md",[],[],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M73JHJBMVFW46F1Y8K3SBF43","phoenix-observability","Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F124-phoenix\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M73JHFPA3A2806PMP42ZN86Q","frontend-ui-dark-ts","Build dark-themed React applications using Tailwind CSS with custom theming, glassmorphism effects, and Framer Motion animations. Use when creating dashboards, admin panels, or data-rich interfaces with a refined dark aesthetic.","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F123-frontend-ui-dark-ts\u002FSKILL.md",[],[],{"skillNo":113,"name":114,"nameZh":6,"description":115,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":116,"categories":117,"keywords":118,"publishedAt":19},"01M73JFVBVK57NRJS49TT2VDCT","tensorboard","Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F113-tensorboard\u002FSKILL.md",[],[],{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":123,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":19},"01M73JFF6PK6ZCN41C89J7HS2B","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fqhjqhj00\u002Fresearch-skills-pool\u002Fblob\u002Fmaster\u002Ftop100-runs\u002F110-markdown-mermaid-writing\u002FSKILL.md",[],[]]