[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MB5X8Q72JF3PFJSQMV1SY8":3,"skill-same:01M6MB5X8Q72JF3PFJSQMV1SY8":88},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":50,"commitSha":51,"skillDirectory":52,"forksCount":53,"repositoryHtmlUrl":54,"repositoryHomepageUrl":55,"ownerHtmlUrl":56,"ownerBlogUrl":57,"ownerDescription":58,"updatedAt":59,"install":60,"resources":73},"01M6MB5X8Q72JF3PFJSQMV1SY8","skypilot-multi-cloud-orchestration",null,"Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1575,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fskypilot-multi-cloud-orchestration\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"multi","cloud","multi cloud","skypilot multi cloud orchestration","orchestration","SkyPilot","cost","automatic","for","you","leverage","instances","auto","use","multi-cloud","batch","run","training","infrastructure","cost optimization","2026-11-10 10:12:30",false,"---\nname: skypilot-multi-cloud-orchestration\ndescription: Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: ml-inference-and-ops\n  maintainer: Kalaris Labs\n  tags: Infrastructure, Multi-Cloud, Orchestration, GPU, Cost Optimization, SkyPilot\n  dependencies: skypilot>=0.7.0\n---\n\n# SkyPilot Multi-Cloud Orchestration\n\nComprehensive guide to running ML workloads across clouds with automatic cost optimization using SkyPilot.\n\n## When to use SkyPilot\n\n**Use SkyPilot when:**\n- Running ML workloads across multiple clouds (AWS, GCP, Azure, etc.)\n- Need cost optimization with automatic cloud\u002Fregion selection\n- Running long jobs on spot instances with auto-recovery\n- Managing distributed multi-node training\n- Want unified interface for 20+ cloud providers\n- Need to avoid vendor lock-in\n\n**Key features:**\n- **Multi-cloud**: AWS, GCP, Azure, Kubernetes, Lambda, RunPod, 20+ providers\n- **Cost optimization**: Automatic cheapest cloud\u002Fregion selection\n- **Spot instances**: 3-6x cost savings with automatic recovery\n- **Distributed training**: Multi-node jobs with gang scheduling\n- **Managed jobs**: Auto-recovery, checkpointing, fault tolerance\n- **Sky Serve**: Model serving with autoscaling\n\n**Use alternatives instead:**\n- **Modal**: For simpler serverless GPU with Python-native API\n- **RunPod**: For single-cloud persistent pods\n- **Kubernetes**: For existing K8s infrastructure\n- **Ray**: For pure Ray-based orchestration\n\n## Quick start\n\n### Installation\n\n```bash\npip install \"skypilot[aws,gcp,azure,kubernetes]\"\n\n# Verify cloud credentials\nsky check\n```\n\n### Hello World\n\nCreate `hello.yaml`:\n```yaml\nresources:\n  accelerators: T4:1\n\nrun: |\n  nvidia-smi\n  echo \"Hello from SkyPilot!\"\n```\n\nLaunch:\n```bash\nsky launch -c hello hello.yaml\n\n# SSH to cluster\nssh hello\n\n# Terminate\nsky down hello\n```\n\n## Core concepts\n\n### Task YAML structure\n\n```yaml\n# Task name (optional)\nname: my-task\n\n# Resource requirements\nresources:\n  cloud: aws              # Optional: auto-select if omitted\n  region: us-west-2       # Optional: auto-select if omitted\n  accelerators: A100:4    # GPU type and count\n  cpus: 8+                # Minimum CPUs\n  memory: 32+             # Minimum memory (GB)\n  use_spot: true          # Use spot instances\n  disk_size: 256          # Disk size (GB)\n\n# Number of nodes for distributed training\nnum_nodes: 2\n\n# Working directory (synced to ~\u002Fsky_workdir)\nworkdir: .\n\n# Setup commands (run once)\nsetup: |\n  pip install -r requirements.txt\n\n# Run commands\nrun: |\n  python train.py\n```\n\n### Key commands\n\n| Command | Purpose |\n|---------|---------|\n| `sky launch` | Launch cluster and run task |\n| `sky exec` | Run task on existing cluster |\n| `sky status` | Show cluster status |\n| `sky stop` | Stop cluster (preserve state) |\n| `sky down` | Terminate cluster |\n| `sky logs` | View task logs |\n| `sky queue` | Show job queue |\n| `sky jobs launch` | Launch managed job |\n| `sky serve up` | Deploy serving endpoint |\n\n## GPU configuration\n\n### Available accelerators\n\n```yaml\n# NVIDIA GPUs\naccelerators: T4:1\naccelerators: L4:1\naccelerators: A10G:1\naccelerators: L40S:1\naccelerators: A100:4\naccelerators: A100-80GB:8\naccelerators: H100:8\n\n# Cloud-specific\naccelerators: V100:4         # AWS\u002FGCP\naccelerators: TPU-v4-8       # GCP TPUs\n```\n\n### GPU fallbacks\n\n```yaml\nresources:\n  accelerators:\n    H100: 8\n    A100-80GB: 8\n    A100: 8\n  any_of:\n    - cloud: gcp\n    - cloud: aws\n    - cloud: azure\n```\n\n### Spot instances\n\n```yaml\nresources:\n  accelerators: A100:8\n  use_spot: true\n  spot_recovery: FAILOVER  # Auto-recover on preemption\n```\n\n## Cluster management\n\n### Launch and execute\n\n```bash\n# Launch new cluster\nsky launch -c mycluster task.yaml\n\n# Run on existing cluster (skip setup)\nsky exec mycluster another_task.yaml\n\n# Interactive SSH\nssh mycluster\n\n# Stream logs\nsky logs mycluster\n```\n\n### Autostop\n\n```yaml\nresources:\n  accelerators: A100:4\n  autostop:\n    idle_minutes: 30\n    down: true  # Terminate instead of stop\n```\n\n```bash\n# Set autostop via CLI\nsky autostop mycluster -i 30 --down\n```\n\n### Cluster status\n\n```bash\n# All clusters\nsky status\n\n# Detailed view\nsky status -a\n```\n\n## Distributed training\n\n### Multi-node setup\n\n```yaml\nresources:\n  accelerators: A100:8\n\nnum_nodes: 4  # 4 nodes × 8 GPUs = 32 GPUs total\n\nsetup: |\n  pip install torch torchvision\n\nrun: |\n  torchrun \\\n    --nnodes=$SKYPILOT_NUM_NODES \\\n    --nproc_per_node=$SKYPILOT_NUM_GPUS_PER_NODE \\\n    --node_rank=$SKYPILOT_NODE_RANK \\\n    --master_addr=$(echo \"$SKYPILOT_NODE_IPS\" | head -n1) \\\n    --master_port=12355 \\\n    train.py\n```\n\n### Environment variables\n\n| Variable | Description |\n|----------|-------------|\n| `SKYPILOT_NODE_RANK` | Node index (0 to num_nodes-1) |\n| `SKYPILOT_NODE_IPS` | Newline-separated IP addresses |\n| `SKYPILOT_NUM_NODES` | Total number of nodes |\n| `SKYPILOT_NUM_GPUS_PER_NODE` | GPUs per node |\n\n### Head-node-only execution\n\n```bash\nrun: |\n  if [ \"${SKYPILOT_NODE_RANK}\" == \"0\" ]; then\n    python orchestrate.py\n  fi\n```\n\n## Managed jobs\n\n### Spot recovery\n\n```bash\n# Launch managed job with spot recovery\nsky jobs launch -n my-job train.yaml\n```\n\n### Checkpointing\n\n```yaml\nname: training-job\n\nfile_mounts:\n  \u002Fcheckpoints:\n    name: my-checkpoints\n    store: s3\n    mode: MOUNT\n\nresources:\n  accelerators: A100:8\n  use_spot: true\n\nrun: |\n  python train.py \\\n    --checkpoint-dir \u002Fcheckpoints \\\n    --resume-from-latest\n```\n\n### Job management\n\n```bash\n# List jobs\nsky jobs queue\n\n# View logs\nsky jobs logs my-job\n\n# Cancel job\nsky jobs cancel my-job\n```\n\n## File mounts and storage\n\n### Local file sync\n\n```yaml\nworkdir: .\u002Fmy-project  # Synced to ~\u002Fsky_workdir\n\nfile_mounts:\n  \u002Fdata\u002Fconfig.yaml: .\u002Fconfig.yaml\n  ~\u002F.vimrc: ~\u002F.vimrc\n```\n\n### Cloud storage\n\n```yaml\nfile_mounts:\n  # Mount S3 bucket\n  \u002Fdatasets:\n    source: s3:\u002F\u002Fmy-bucket\u002Fdatasets\n    mode: MOUNT  # Stream from S3\n\n  # Copy GCS bucket\n  \u002Fmodels:\n    source: gs:\u002F\u002Fmy-bucket\u002Fmodels\n    mode: COPY  # Pre-fetch to disk\n\n  # Cached mount (fast writes)\n  \u002Foutputs:\n    name: my-outputs\n    store: s3\n    mode: MOUNT_CACHED\n```\n\n### Storage modes\n\n| Mode | Description | Best For |\n|------|-------------|----------|\n| `MOUNT` | Stream from cloud | Large datasets, read-heavy |\n| `COPY` | Pre-fetch to disk | Small files, random access |\n| `MOUNT_CACHED` | Cache with async upload | Checkpoints, outputs |\n\n## Sky Serve (Model Serving)\n\nDetails, code examples and parameter tables: [references\u002Fsky-serve-model-serving.md](references\u002Fsky-serve-model-serving.md). Read it when this step applies.\n\n## Cost optimization\n\n### Automatic cloud selection\n\n```yaml\n# SkyPilot finds cheapest option\nresources:\n  accelerators: A100:8\n  # No cloud specified - auto-select cheapest\n```\n\n```bash\n# Show optimizer decision\nsky launch task.yaml --dryrun\n```\n\n### Cloud preferences\n\n```yaml\nresources:\n  accelerators: A100:8\n  any_of:\n    - cloud: gcp\n      region: us-central1\n    - cloud: aws\n      region: us-east-1\n    - cloud: azure\n```\n\n### Environment variables\n\n```yaml\nenvs:\n  HF_TOKEN: $HF_TOKEN  # Inherited from local env\n  WANDB_API_KEY: $WANDB_API_KEY\n\n# Or use secrets\nsecrets:\n  - HF_TOKEN\n  - WANDB_API_KEY\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tuning with checkpoints\n\n```yaml\nname: llm-finetune\n\nfile_mounts:\n  \u002Fcheckpoints:\n    name: finetune-checkpoints\n    store: s3\n    mode: MOUNT_CACHED\n\nresources:\n  accelerators: A100:8\n  use_spot: true\n\nsetup: |\n  pip install transformers accelerate\n\nrun: |\n  python train.py \\\n    --checkpoint-dir \u002Fcheckpoints \\\n    --resume\n```\n\n### Workflow 2: Hyperparameter sweep\n\n```yaml\nname: hp-sweep-${RUN_ID}\n\nenvs:\n  RUN_ID: 0\n  LEARNING_RATE: 1e-4\n  BATCH_SIZE: 32\n\nresources:\n  accelerators: A100:1\n  use_spot: true\n\nrun: |\n  python train.py \\\n    --lr $LEARNING_RATE \\\n    --batch-size $BATCH_SIZE \\\n    --run-id $RUN_ID\n```\n\n```bash\n# Launch multiple jobs\nfor i in {1..10}; do\n  sky jobs launch sweep.yaml \\\n    --env RUN_ID=$i \\\n    --env LEARNING_RATE=$(python -c \"import random; print(10**random.uniform(-5,-3))\")\ndone\n```\n\n## Debugging\n\n```bash\n# SSH to cluster\nssh mycluster\n\n# View logs\nsky logs mycluster\n\n# Check job queue\nsky queue mycluster\n\n# View managed job logs\nsky jobs logs my-job\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Quota exceeded | Request quota increase, try different region |\n| Spot preemption | Use `sky jobs launch` for auto-recovery |\n| Slow file sync | Use `MOUNT_CACHED` mode for outputs |\n| GPU not available | Use `any_of` for fallback clouds |\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Multi-cloud, optimization, production patterns\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fdocs.skypilot.co\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fskypilot-org\u002Fskypilot\n- **Slack**: https:\u002F\u002Fslack.skypilot.co\n- **Examples**: https:\u002F\u002Fgithub.com\u002Fskypilot-org\u002Fskypilot\u002Ftree\u002Fmaster\u002Fexamples\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm hardware, framework and server versions, model format, and expected load.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Serve or log a single request or run end to end before scaling.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Measure latency, throughput and output correctness against a reference; check resource usage and costs.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| The server fails to start or OOMs | Check model size versus memory, quantization and parallelism settings. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Do not expose services or credentials publicly; confirm cloud costs before provisioning.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `modal-serverless-gpu`: Serverless GPU cloud platform for running ML workloads.\n- `lambda-labs-gpu-cloud`: Reserved and on-demand GPU cloud instances for ML training and inference.\n- `ray-train`: Distributed training orchestration across clusters.\n","# SkyPilot Multi-Cloud Orchestration\n\nComprehensive guide to running ML workloads across clouds with automatic cost optimization using SkyPilot.\n\n## When to use SkyPilot\n\n**Use SkyPilot when:**\n- Running ML workloads across multiple clouds (AWS, GCP, Azure, etc.)\n- Need cost optimization with automatic cloud\u002Fregion selection\n- Running long jobs on spot instances with auto-recovery\n- Managing distributed multi-node training\n- Want unified interface for 20+ cloud providers\n- Need to avoid vendor lock-in\n\n**Key features:**\n- **Multi-cloud**: AWS, GCP, Azure, Kubernetes, Lambda, RunPod, 20+ providers\n- **Cost optimization**: Automatic cheapest cloud\u002Fregion selection\n- **Spot instances**: 3-6x cost savings with automatic recovery\n- **Distributed training**: Multi-node jobs with gang scheduling\n- **Managed jobs**: Auto-recovery, checkpointing, fault tolerance\n- **Sky Serve**: Model serving with autoscaling\n\n**Use alternatives instead:**\n- **Modal**: For simpler serverless GPU with Python-native API\n- **RunPod**: For single-cloud persistent pods\n- **Kubernetes**: For existing K8s infrastructure\n- **Ray**: For pure Ray-based orchestration\n\n## Quick start\n\n### Installation\n\n```bash\npip install \"skypilot[aws,gcp,azure,kubernetes]\"\n\n# Verify cloud credentials\nsky check\n```\n\n### Hello World\n\nCreate `hello.yaml`:\n```yaml\nresources:\n  accelerators: T4:1\n\nrun: |\n  nvidia-smi\n  echo \"Hello from SkyPilot!\"\n```\n\nLaunch:\n```bash\nsky launch -c hello hello.yaml\n\n# SSH to cluster\nssh hello\n\n# Terminate\nsky down hello\n```\n\n## Core concepts\n\n### Task YAML structure\n\n```yaml\n# Task name (optional)\nname: my-task\n\n# Resource requirements\nresources:\n  cloud: aws              # Optional: auto-select if omitted\n  region: us-west-2       # Optional: auto-select if omitted\n  accelerators: A100:4    # GPU type and count\n  cpus: 8+                # Minimum CPUs\n  memory: 32+             # Minimum memory (GB)\n  use_spot: true          # Use spot instances\n  disk_size: 256          # Disk size (GB)\n\n# Number of nodes for distributed training\nnum_nodes: 2\n\n# Working directory (synced to ~\u002Fsky_workdir)\nworkdir: .\n\n# Setup commands (run once)\nsetup: |\n  pip install -r requirements.txt\n\n# Run commands\nrun: |\n  python train.py\n```\n\n### Key commands\n\n| Command | Purpose |\n|---------|---------|\n| `sky launch` | Launch cluster and run task |\n| `sky exec` | Run task on existing cluster |\n| `sky status` | Show cluster status |\n| `sky stop` | Stop cluster (preserve state) |\n| `sky down` | Terminate cluster |\n| `sky logs` | View task logs |\n| `sky queue` | Show job queue |\n| `sky jobs launch` | Launch managed job |\n| `sky serve up` | Deploy serving endpoint |\n\n## GPU configuration\n\n### Available accelerators\n\n```yaml\n# NVIDIA GPUs\naccelerators: T4:1\naccelerators: L4:1\naccelerators: A10G:1\naccelerators: L40S:1\naccelerators: A100:4\naccelerators: A100-80GB:8\naccelerators: H100:8\n\n# Cloud-specific\naccelerators: V100:4         # AWS\u002FGCP\naccelerators: TPU-v4-8       # GCP TPUs\n```\n\n### GPU fallbacks\n\n```yaml\nresources:\n  accelerators:\n    H100: 8\n    A100-80GB: 8\n    A100: 8\n  any_of:\n    - cloud: gcp\n    - cloud: aws\n    - cloud: azure\n```\n\n### Spot instances\n\n```yaml\nresources:\n  accelerators: A100:8\n  use_spot: true\n  spot_recovery: FAILOVER  # Auto-recover on preemption\n```\n\n## Cluster management\n\n### Launch and execute\n\n```bash\n# Launch new cluster\nsky launch -c mycluster task.yaml\n\n# Run on existing cluster (skip setup)\nsky exec mycluster another_task.yaml\n\n# Interactive SSH\nssh mycluster\n\n# Stream logs\nsky logs mycluster\n```\n\n### Autostop\n\n```yaml\nresources:\n  accelerators: A100:4\n  autostop:\n    idle_minutes: 30\n    down: true  # Terminate instead of stop\n```\n\n```bash\n# Set autostop via CLI\nsky autostop mycluster -i 30 --down\n```\n\n### Cluster status\n\n```bash\n# All clusters\nsky status\n\n# Detailed view\nsky status -a\n```\n\n## Distributed training\n\n### Multi-node setup\n\n```yaml\nresources:\n  accelerators: A100:8\n\nnum_nodes: 4  # 4 nodes × 8 GPUs = 32 GPUs total\n\nsetup: |\n  pip install torch torchvision\n\nrun: |\n  torchrun \\\n    --nnodes=$SKYPILOT_NUM_NODES \\\n    --nproc_per_node=$SKYPILOT_NUM_GPUS_PER_NODE \\\n    --node_rank=$SKYPILOT_NODE_RANK \\\n    --master_addr=$(echo \"$SKYPILOT_NODE_IPS\" | head -n1) \\\n    --master_port=12355 \\\n    train.py\n```\n\n### Environment variables\n\n| Variable | Description |\n|----------|-------------|\n| `SKYPILOT_NODE_RANK` | Node index (0 to num_nodes-1) |\n| `SKYPILOT_NODE_IPS` | Newline-separated IP addresses |\n| `SKYPILOT_NUM_NODES` | Total number of nodes |\n| `SKYPILOT_NUM_GPUS_PER_NODE` | GPUs per node |\n\n### Head-node-only execution\n\n```bash\nrun: |\n  if [ \"${SKYPILOT_NODE_RANK}\" == \"0\" ]; then\n    python orchestrate.py\n  fi\n```\n\n## Managed jobs\n\n### Spot recovery\n\n```bash\n# Launch managed job with spot recovery\nsky jobs launch -n my-job train.yaml\n```\n\n### Checkpointing\n\n```yaml\nname: training-job\n\nfile_mounts:\n  \u002Fcheckpoints:\n    name: my-checkpoints\n    store: s3\n    mode: MOUNT\n\nresources:\n  accelerators: A100:8\n  use_spot: true\n\nrun: |\n  python train.py \\\n    --checkpoint-dir \u002Fcheckpoints \\\n    --resume-from-latest\n```\n\n### Job management\n\n```bash\n# List jobs\nsky jobs queue\n\n# View logs\nsky jobs logs my-job\n\n# Cancel job\nsky jobs cancel my-job\n```\n\n## File mounts and storage\n\n### Local file sync\n\n```yaml\nworkdir: .\u002Fmy-project  # Synced to ~\u002Fsky_workdir\n\nfile_mounts:\n  \u002Fdata\u002Fconfig.yaml: .\u002Fconfig.yaml\n  ~\u002F.vimrc: ~\u002F.vimrc\n```\n\n### Cloud storage\n\n```yaml\nfile_mounts:\n  # Mount S3 bucket\n  \u002Fdatasets:\n    source: s3:\u002F\u002Fmy-bucket\u002Fdatasets\n    mode: MOUNT  # Stream from S3\n\n  # Copy GCS bucket\n  \u002Fmodels:\n    source: gs:\u002F\u002Fmy-bucket\u002Fmodels\n    mode: COPY  # Pre-fetch to disk\n\n  # Cached mount (fast writes)\n  \u002Foutputs:\n    name: my-outputs\n    store: s3\n    mode: MOUNT_CACHED\n```\n\n### Storage modes\n\n| Mode | Description | Best For |\n|------|-------------|----------|\n| `MOUNT` | Stream from cloud | Large datasets, read-heavy |\n| `COPY` | Pre-fetch to disk | Small files, random access |\n| `MOUNT_CACHED` | Cache with async upload | Checkpoints, outputs |\n\n## Sky Serve (Model Serving)\n\nDetails, code examples and parameter tables: [references\u002Fsky-serve-model-serving.md](references\u002Fsky-serve-model-serving.md). Read it when this step applies.\n\n## Cost optimization\n\n### Automatic cloud selection\n\n```yaml\n# SkyPilot finds cheapest option\nresources:\n  accelerators: A100:8\n  # No cloud specified - auto-select cheapest\n```\n\n```bash\n# Show optimizer decision\nsky launch task.yaml --dryrun\n```\n\n### Cloud preferences\n\n```yaml\nresources:\n  accelerators: A100:8\n  any_of:\n    - cloud: gcp\n      region: us-central1\n    - cloud: aws\n      region: us-east-1\n    - cloud: azure\n```\n\n### Environment variables\n\n```yaml\nenvs:\n  HF_TOKEN: $HF_TOKEN  # Inherited from local env\n  WANDB_API_KEY: $WANDB_API_KEY\n\n# Or use secrets\nsecrets:\n  - HF_TOKEN\n  - WANDB_API_KEY\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tuning with checkpoints\n\n```yaml\nname: llm-finetune\n\nfile_mounts:\n  \u002Fcheckpoints:\n    name: finetune-checkpoints\n    store: s3\n    mode: MOUNT_CACHED\n\nresources:\n  accelerators: A100:8\n  use_spot: true\n\nsetup: |\n  pip install transformers accelerate\n\nrun: |\n  python train.py \\\n    --checkpoint-dir \u002Fcheckpoints \\\n    --resume\n```\n\n### Workflow 2: Hyperparameter sweep\n\n```yaml\nname: hp-sweep-${RUN_ID}\n\nenvs:\n  RUN_ID: 0\n  LEARNING_RATE: 1e-4\n  BATCH_SIZE: 32\n\nresources:\n  accelerators: A100:1\n  use_spot: true\n\nrun: |\n  python train.py \\\n    --lr $LEARNING_RATE \\\n    --batch-size $BATCH_SIZE \\\n    --run-id $RUN_ID\n```\n\n```bash\n# Launch multiple jobs\nfor i in {1..10}; do\n  sky jobs launch sweep.yaml \\\n    --env RUN_ID=$i \\\n    --env LEARNING_RATE=$(python -c \"import random; print(10**random.uniform(-5,-3))\")\ndone\n```\n\n## Debugging\n\n```bash\n# SSH to cluster\nssh mycluster\n\n# View logs\nsky logs mycluster\n\n# Check job queue\nsky queue mycluster\n\n# View managed job logs\nsky jobs logs my-job\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Quota exceeded | Request quota increase, try different region |\n| Spot preemption | Use `sky jobs launch` for auto-recovery |\n| Slow file sync | Use `MOUNT_CACHED` mode for outputs |\n| GPU not available | Use `any_of` for fallback clouds |\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Multi-cloud, optimization, production patterns\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fdocs.skypilot.co\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fskypilot-org\u002Fskypilot\n- **Slack**: https:\u002F\u002Fslack.skypilot.co\n- **Examples**: https:\u002F\u002Fgithub.com\u002Fskypilot-org\u002Fskypilot\u002Ftree\u002Fmaster\u002Fexamples\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm hardware, framework and server versions, model format, and expected load.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Serve or log a single request or run end to end before scaling.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Measure latency, throughput and output correctness against a reference; check resource usage and costs.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| The server fails to start or OOMs | Check model size versus memory, quantization and parallelism settings. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Do not expose services or credentials publicly; confirm cloud costs before provisioning.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `modal-serverless-gpu`: Serverless GPU cloud platform for running ML workloads.\n- `lambda-labs-gpu-cloud`: Reserved and on-demand GPU cloud instances for ML training and inference.\n- `ray-train`: Distributed training orchestration across clusters.\n",{"name":5,"license":15,"metadata":44,"description":7},{"tags":45,"version":46,"category":47,"maintainer":48,"dependencies":49},"Infrastructure, Multi-Cloud, Orchestration, GPU, Cost Optimization, SkyPilot","1.0.0","ml-inference-and-ops","Kalaris Labs","skypilot>=0.7.0","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fskypilot-multi-cloud-orchestration",491,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-14 14:06:13",{"source":8,"skillName":5,"commands":61,"prompt":71,"zipAvailable":72},[62,65,68],{"tool":63,"command":64},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill skypilot-multi-cloud-orchestration",{"tool":66,"command":67},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill skypilot-multi-cloud-orchestration",{"tool":69,"command":70},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill skypilot-multi-cloud-orchestration","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fskypilot-multi-cloud-orchestration\u002FSKILL.md",true,[74,80,84],{"path":75,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fadvanced-usage.md","REFERENCE","参考文档",7469,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fskypilot-multi-cloud-orchestration\u002Freferences\u002Fadvanced-usage.md",{"path":81,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fsky-serve-model-serving.md",732,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fskypilot-multi-cloud-orchestration\u002Freferences\u002Fsky-serve-model-serving.md",{"path":85,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Ftroubleshooting.md",10493,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fskypilot-multi-cloud-orchestration\u002Freferences\u002Ftroubleshooting.md",[89,98,106,114,122,130],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":93,"sourceHtmlUrl":94,"categories":95,"keywords":96,"publishedAt":97},"01M7DV332YQMGD8WTRJ9DZQXTB","worktable-review-with-annotations","Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":102,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":97},"01M7DV2YGHCZVBEB1AHYB6CGHE","worktable-find-and-synthesize-context","Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":110,"sourceHtmlUrl":111,"categories":112,"keywords":113,"publishedAt":97},"01M7DV2TQN1H11ME2SAF9MXN7V","worktable-create-or-update-docs","Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FSKILL.md",[],[],{"skillNo":115,"name":116,"nameZh":6,"description":117,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":118,"sourceHtmlUrl":119,"categories":120,"keywords":121,"publishedAt":97},"01M7DV2NE452NEA8TXATKSNAJH","worktable-create-or-manage-records","Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FSKILL.md",[],[],{"skillNo":123,"name":124,"nameZh":6,"description":125,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":126,"sourceHtmlUrl":127,"categories":128,"keywords":129,"publishedAt":97},"01M7DV2HENRBR2KSCVM064WH0Y","worktable-create-or-manage-interactive-html","Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FSKILL.md",[],[],{"skillNo":131,"name":132,"nameZh":6,"description":133,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":134,"sourceHtmlUrl":135,"categories":136,"keywords":137,"publishedAt":97},"01M7DV2CADR5R7RQFFYPNAGR9Y","worktable-collaborate-threads","Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FSKILL.md",[],[]]