[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MB4DBMAZA24VCTKB3VD41V":3,"skill-same:01M6MB4DBMAZA24VCTKB3VD41V":97},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":26,"publishedAt":48,"linkOnly":49,"markdownContent":50,"markdownBody":51,"frontMatter":52,"refName":59,"commitSha":60,"skillDirectory":61,"forksCount":62,"repositoryHtmlUrl":63,"repositoryHomepageUrl":64,"ownerHtmlUrl":65,"ownerBlogUrl":66,"ownerDescription":67,"updatedAt":68,"install":69,"resources":82},"01M6MB4DBMAZA24VCTKB3VD41V","segment-anything-model",null,"Foundation model for image segmentation with zero-shot transfer. Use when you need to segment any object in images using points, boxes, or masks as prompts, or automatically generate all object masks in an image.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1575,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47],"segment anything","model","segment","segment anything model","images","when","for","object","you","Zero-Shot","boxes","as","image","vision","use","computer","segmentation","Transformers","ZeRO","multimodal","generate","2026-11-10 10:12:30",false,"---\nname: segment-anything-model\ndescription: Foundation model for image segmentation with zero-shot transfer. Use when you need to segment any object in images using points, boxes, or masks as prompts, or automatically generate all object masks in an image.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: multimodal-and-emerging\n  maintainer: Kalaris Labs\n  tags: Multimodal, Image Segmentation, Computer Vision, SAM, Zero-Shot\n  dependencies: segment-anything, transformers>=4.30.0, torch>=1.7.0\n---\n\n# Segment Anything Model (SAM)\n\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\n\n## When to use SAM\n\n**Use SAM when:**\n- Need to segment any object in images without task-specific training\n- Building interactive annotation tools with point\u002Fbox prompts\n- Generating training data for other vision models\n- Need zero-shot transfer to new image domains\n- Building object detection\u002Fsegmentation pipelines\n- Processing medical, satellite, or domain-specific images\n\n**Key features:**\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\n- **Flexible prompts**: Points, bounding boxes, or previous masks\n- **Automatic segmentation**: Generate all object masks automatically\n- **High quality**: Trained on 1.1 billion masks from 11 million images\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\n- **ONNX export**: Deploy in browsers and edge devices\n\n**Use alternatives instead:**\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\n- **GroundingDINO + SAM**: For text-prompted segmentation\n- **SAM 2**: For video segmentation tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# From GitHub\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\n\n# Optional dependencies\npip install opencv-python pycocotools matplotlib\n\n# Or use HuggingFace transformers\npip install transformers\n```\n\n### Download checkpoints\n\n```bash\n# ViT-H (largest, most accurate) - 2.4GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\n\n# ViT-L (medium) - 1.2GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\n\n# ViT-B (smallest, fastest) - 375MB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\n```\n\n### Basic usage with SamPredictor\n\n```python\nimport numpy as np\nfrom segment_anything import sam_model_registry, SamPredictor\n\n# Load model\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\nsam.to(device=\"cuda\")\n\n# Create predictor\npredictor = SamPredictor(sam)\n\n# Set image (computes embeddings once)\nimage = cv2.imread(\"image.jpg\")\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\npredictor.set_image(image)\n\n# Predict with point prompts\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\ninput_label = np.array([1])  # 1 = foreground, 0 = background\n\nmasks, scores, logits = predictor.predict(\n    point_coords=input_point,\n    point_labels=input_label,\n    multimask_output=True  # Returns 3 mask options\n)\n\n# Select best mask\nbest_mask = masks[np.argmax(scores)]\n```\n\n### HuggingFace Transformers\n\n```python\nimport torch\nfrom PIL import Image\nfrom transformers import SamModel, SamProcessor\n\n# Load model and processor\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nmodel.to(\"cuda\")\n\n# Process image with point prompt\nimage = Image.open(\"image.jpg\")\ninput_points = [[[450, 600]]]  # Batch of points\n\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\n\n# Generate masks\nwith torch.no_grad():\n    outputs = model(**inputs)\n\n# Post-process masks to original size\nmasks = processor.image_processor.post_process_masks(\n    outputs.pred_masks.cpu(),\n    inputs[\"original_sizes\"].cpu(),\n    inputs[\"reshaped_input_sizes\"].cpu()\n)\n```\n\n## Core concepts\n\n### Model architecture\n\n```\nSAM Architecture:\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\n└─────────────────┘     └─────────────────┘     └─────────────────┘\n        │                       │                       │\n   Image Embeddings      Prompt Embeddings         Masks + IoU\n   (computed once)       (per prompt)             predictions\n```\n\n### Model variants\n\n| Model | Checkpoint | Size | Speed | Accuracy |\n|-------|------------|------|-------|----------|\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\n\n### Prompt types\n\n| Prompt | Description | Use Case |\n|--------|-------------|----------|\n| Point (foreground) | Click on object | Single object selection |\n| Point (background) | Click outside object | Exclude regions |\n| Bounding box | Rectangle around object | Larger objects |\n| Previous mask | Low-res mask input | Iterative refinement |\n\n## Interactive segmentation\n\nDetails, code examples and parameter tables: [references\u002Finteractive-segmentation.md](references\u002Finteractive-segmentation.md). Read it when this step applies.\n\n## Automatic mask generation\n\n### Basic automatic segmentation\n\n```python\nfrom segment_anything import SamAutomaticMaskGenerator\n\n# Create generator\nmask_generator = SamAutomaticMaskGenerator(sam)\n\n# Generate all masks\nmasks = mask_generator.generate(image)\n\n# Each mask contains:\n# - segmentation: binary mask\n# - bbox: [x, y, w, h]\n# - area: pixel count\n# - predicted_iou: quality score\n# - stability_score: robustness score\n# - point_coords: generating point\n```\n\n### Customized generation\n\n```python\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=32,          # Grid density (more = more masks)\n    pred_iou_thresh=0.88,        # Quality threshold\n    stability_score_thresh=0.95,  # Stability threshold\n    crop_n_layers=1,             # Multi-scale crops\n    crop_n_points_downscale_factor=2,\n    min_mask_region_area=100,    # Remove tiny masks\n)\n\nmasks = mask_generator.generate(image)\n```\n\n### Filtering masks\n\n```python\n# Sort by area (largest first)\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\n\n# Filter by predicted IoU\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\n\n# Filter by stability score\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\n```\n\n## Batched inference\n\n### Multiple images\n\n```python\n# Process multiple images efficiently\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\n\nall_masks = []\nfor image in images:\n    predictor.set_image(image)\n    masks, _, _ = predictor.predict(\n        point_coords=np.array([[500, 375]]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks)\n```\n\n### Multiple prompts per image\n\n```python\n# Process multiple prompts efficiently (one image encoding)\npredictor.set_image(image)\n\n# Batch of point prompts\npoints = [\n    np.array([[100, 100]]),\n    np.array([[200, 200]]),\n    np.array([[300, 300]])\n]\n\nall_masks = []\nfor point in points:\n    masks, scores, _ = predictor.predict(\n        point_coords=point,\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks[np.argmax(scores)])\n```\n\n## ONNX deployment\n\n### Export model\n\n```bash\npython scripts\u002Fexport_onnx_model.py \\\n    --checkpoint sam_vit_h_4b8939.pth \\\n    --model-type vit_h \\\n    --output sam_onnx.onnx \\\n    --return-single-mask\n```\n\n### Use ONNX model\n\n```python\nimport onnxruntime\n\n# Load ONNX model\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\n\n# Run inference (image embeddings computed separately)\nmasks = ort_session.run(\n    None,\n    {\n        \"image_embeddings\": image_embeddings,\n        \"point_coords\": point_coords,\n        \"point_labels\": point_labels,\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\n        \"has_mask_input\": np.array([0], dtype=np.float32),\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\n    }\n)\n```\n\n## Common workflows\n\n### Workflow 1: Annotation tool\n\n```python\nimport cv2\n\n# Load model\npredictor = SamPredictor(sam)\npredictor.set_image(image)\n\ndef on_click(event, x, y, flags, param):\n    if event == cv2.EVENT_LBUTTONDOWN:\n        # Foreground point\n        masks, scores, _ = predictor.predict(\n            point_coords=np.array([[x, y]]),\n            point_labels=np.array([1]),\n            multimask_output=True\n        )\n        # Display best mask\n        display_mask(masks[np.argmax(scores)])\n```\n\n### Workflow 2: Object extraction\n\n```python\ndef extract_object(image, point):\n    \"\"\"Extract object at point with transparent background.\"\"\"\n    predictor.set_image(image)\n\n    masks, scores, _ = predictor.predict(\n        point_coords=np.array([point]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n\n    best_mask = masks[np.argmax(scores)]\n\n    # Create RGBA output\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\n    rgba[:, :, :3] = image\n    rgba[:, :, 3] = best_mask * 255\n\n    return rgba\n```\n\n### Workflow 3: Medical image segmentation\n\n```python\n# Process medical images (grayscale to RGB)\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\n\npredictor.set_image(rgb_image)\n\n# Segment region of interest\nmasks, scores, _ = predictor.predict(\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\n    multimask_output=True\n)\n```\n\n## Output format\n\n### Mask data structure\n\n```python\n# SamAutomaticMaskGenerator output\n{\n    \"segmentation\": np.ndarray,  # H×W binary mask\n    \"bbox\": [x, y, w, h],        # Bounding box\n    \"area\": int,                 # Pixel count\n    \"predicted_iou\": float,      # 0-1 quality score\n    \"stability_score\": float,    # 0-1 robustness score\n    \"crop_box\": [x, y, w, h],    # Generation crop region\n    \"point_coords\": [[x, y]],    # Input point\n}\n```\n\n### COCO RLE format\n\n```python\nfrom pycocotools import mask as mask_utils\n\n# Encode mask to RLE\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n# Decode RLE to mask\ndecoded_mask = mask_utils.decode(rle)\n```\n\n## Performance optimization\n\n### GPU memory\n\n```python\n# Use smaller model for limited VRAM\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\n\n# Process images in batches\n# Clear CUDA cache between large batches\ntorch.cuda.empty_cache()\n```\n\n### Speed optimization\n\n```python\n# Use half precision\nsam = sam.half()\n\n# Reduce points for automatic generation\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=16,  # Default is 32\n)\n\n# Use ONNX for deployment\n# Export with --return-single-mask for faster inference\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Out of memory | Use ViT-B model, reduce image size |\n| Slow inference | Use ViT-B, reduce points_per_side |\n| Poor mask quality | Try different prompts, use box + points |\n| Edge artifacts | Use stability_score filtering |\n| Small objects missed | Increase points_per_side |\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\n- **Demo**: https:\u002F\u002Fsegment-anything.com\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `clip`: OpenAI's model connecting vision and language.\n- `generate-image`: Generate or edit images with AI models through the OpenRouter Image API (Gemini, Seedream, Recraft, GPT-Image, Riverflow).\n- `blip-2-vision-language`: Explains how to use Salesforce BLIP-2 (Q-Former bridging a frozen image encoder and an LLM such as OPT or FlanT5) through HuggingFace Trans…\n","# Segment Anything Model (SAM)\n\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\n\n## When to use SAM\n\n**Use SAM when:**\n- Need to segment any object in images without task-specific training\n- Building interactive annotation tools with point\u002Fbox prompts\n- Generating training data for other vision models\n- Need zero-shot transfer to new image domains\n- Building object detection\u002Fsegmentation pipelines\n- Processing medical, satellite, or domain-specific images\n\n**Key features:**\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\n- **Flexible prompts**: Points, bounding boxes, or previous masks\n- **Automatic segmentation**: Generate all object masks automatically\n- **High quality**: Trained on 1.1 billion masks from 11 million images\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\n- **ONNX export**: Deploy in browsers and edge devices\n\n**Use alternatives instead:**\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\n- **GroundingDINO + SAM**: For text-prompted segmentation\n- **SAM 2**: For video segmentation tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# From GitHub\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\n\n# Optional dependencies\npip install opencv-python pycocotools matplotlib\n\n# Or use HuggingFace transformers\npip install transformers\n```\n\n### Download checkpoints\n\n```bash\n# ViT-H (largest, most accurate) - 2.4GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\n\n# ViT-L (medium) - 1.2GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\n\n# ViT-B (smallest, fastest) - 375MB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\n```\n\n### Basic usage with SamPredictor\n\n```python\nimport numpy as np\nfrom segment_anything import sam_model_registry, SamPredictor\n\n# Load model\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\nsam.to(device=\"cuda\")\n\n# Create predictor\npredictor = SamPredictor(sam)\n\n# Set image (computes embeddings once)\nimage = cv2.imread(\"image.jpg\")\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\npredictor.set_image(image)\n\n# Predict with point prompts\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\ninput_label = np.array([1])  # 1 = foreground, 0 = background\n\nmasks, scores, logits = predictor.predict(\n    point_coords=input_point,\n    point_labels=input_label,\n    multimask_output=True  # Returns 3 mask options\n)\n\n# Select best mask\nbest_mask = masks[np.argmax(scores)]\n```\n\n### HuggingFace Transformers\n\n```python\nimport torch\nfrom PIL import Image\nfrom transformers import SamModel, SamProcessor\n\n# Load model and processor\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nmodel.to(\"cuda\")\n\n# Process image with point prompt\nimage = Image.open(\"image.jpg\")\ninput_points = [[[450, 600]]]  # Batch of points\n\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\n\n# Generate masks\nwith torch.no_grad():\n    outputs = model(**inputs)\n\n# Post-process masks to original size\nmasks = processor.image_processor.post_process_masks(\n    outputs.pred_masks.cpu(),\n    inputs[\"original_sizes\"].cpu(),\n    inputs[\"reshaped_input_sizes\"].cpu()\n)\n```\n\n## Core concepts\n\n### Model architecture\n\n```\nSAM Architecture:\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\n└─────────────────┘     └─────────────────┘     └─────────────────┘\n        │                       │                       │\n   Image Embeddings      Prompt Embeddings         Masks + IoU\n   (computed once)       (per prompt)             predictions\n```\n\n### Model variants\n\n| Model | Checkpoint | Size | Speed | Accuracy |\n|-------|------------|------|-------|----------|\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\n\n### Prompt types\n\n| Prompt | Description | Use Case |\n|--------|-------------|----------|\n| Point (foreground) | Click on object | Single object selection |\n| Point (background) | Click outside object | Exclude regions |\n| Bounding box | Rectangle around object | Larger objects |\n| Previous mask | Low-res mask input | Iterative refinement |\n\n## Interactive segmentation\n\nDetails, code examples and parameter tables: [references\u002Finteractive-segmentation.md](references\u002Finteractive-segmentation.md). Read it when this step applies.\n\n## Automatic mask generation\n\n### Basic automatic segmentation\n\n```python\nfrom segment_anything import SamAutomaticMaskGenerator\n\n# Create generator\nmask_generator = SamAutomaticMaskGenerator(sam)\n\n# Generate all masks\nmasks = mask_generator.generate(image)\n\n# Each mask contains:\n# - segmentation: binary mask\n# - bbox: [x, y, w, h]\n# - area: pixel count\n# - predicted_iou: quality score\n# - stability_score: robustness score\n# - point_coords: generating point\n```\n\n### Customized generation\n\n```python\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=32,          # Grid density (more = more masks)\n    pred_iou_thresh=0.88,        # Quality threshold\n    stability_score_thresh=0.95,  # Stability threshold\n    crop_n_layers=1,             # Multi-scale crops\n    crop_n_points_downscale_factor=2,\n    min_mask_region_area=100,    # Remove tiny masks\n)\n\nmasks = mask_generator.generate(image)\n```\n\n### Filtering masks\n\n```python\n# Sort by area (largest first)\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\n\n# Filter by predicted IoU\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\n\n# Filter by stability score\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\n```\n\n## Batched inference\n\n### Multiple images\n\n```python\n# Process multiple images efficiently\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\n\nall_masks = []\nfor image in images:\n    predictor.set_image(image)\n    masks, _, _ = predictor.predict(\n        point_coords=np.array([[500, 375]]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks)\n```\n\n### Multiple prompts per image\n\n```python\n# Process multiple prompts efficiently (one image encoding)\npredictor.set_image(image)\n\n# Batch of point prompts\npoints = [\n    np.array([[100, 100]]),\n    np.array([[200, 200]]),\n    np.array([[300, 300]])\n]\n\nall_masks = []\nfor point in points:\n    masks, scores, _ = predictor.predict(\n        point_coords=point,\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks[np.argmax(scores)])\n```\n\n## ONNX deployment\n\n### Export model\n\n```bash\npython scripts\u002Fexport_onnx_model.py \\\n    --checkpoint sam_vit_h_4b8939.pth \\\n    --model-type vit_h \\\n    --output sam_onnx.onnx \\\n    --return-single-mask\n```\n\n### Use ONNX model\n\n```python\nimport onnxruntime\n\n# Load ONNX model\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\n\n# Run inference (image embeddings computed separately)\nmasks = ort_session.run(\n    None,\n    {\n        \"image_embeddings\": image_embeddings,\n        \"point_coords\": point_coords,\n        \"point_labels\": point_labels,\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\n        \"has_mask_input\": np.array([0], dtype=np.float32),\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\n    }\n)\n```\n\n## Common workflows\n\n### Workflow 1: Annotation tool\n\n```python\nimport cv2\n\n# Load model\npredictor = SamPredictor(sam)\npredictor.set_image(image)\n\ndef on_click(event, x, y, flags, param):\n    if event == cv2.EVENT_LBUTTONDOWN:\n        # Foreground point\n        masks, scores, _ = predictor.predict(\n            point_coords=np.array([[x, y]]),\n            point_labels=np.array([1]),\n            multimask_output=True\n        )\n        # Display best mask\n        display_mask(masks[np.argmax(scores)])\n```\n\n### Workflow 2: Object extraction\n\n```python\ndef extract_object(image, point):\n    \"\"\"Extract object at point with transparent background.\"\"\"\n    predictor.set_image(image)\n\n    masks, scores, _ = predictor.predict(\n        point_coords=np.array([point]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n\n    best_mask = masks[np.argmax(scores)]\n\n    # Create RGBA output\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\n    rgba[:, :, :3] = image\n    rgba[:, :, 3] = best_mask * 255\n\n    return rgba\n```\n\n### Workflow 3: Medical image segmentation\n\n```python\n# Process medical images (grayscale to RGB)\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\n\npredictor.set_image(rgb_image)\n\n# Segment region of interest\nmasks, scores, _ = predictor.predict(\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\n    multimask_output=True\n)\n```\n\n## Output format\n\n### Mask data structure\n\n```python\n# SamAutomaticMaskGenerator output\n{\n    \"segmentation\": np.ndarray,  # H×W binary mask\n    \"bbox\": [x, y, w, h],        # Bounding box\n    \"area\": int,                 # Pixel count\n    \"predicted_iou\": float,      # 0-1 quality score\n    \"stability_score\": float,    # 0-1 robustness score\n    \"crop_box\": [x, y, w, h],    # Generation crop region\n    \"point_coords\": [[x, y]],    # Input point\n}\n```\n\n### COCO RLE format\n\n```python\nfrom pycocotools import mask as mask_utils\n\n# Encode mask to RLE\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n# Decode RLE to mask\ndecoded_mask = mask_utils.decode(rle)\n```\n\n## Performance optimization\n\n### GPU memory\n\n```python\n# Use smaller model for limited VRAM\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\n\n# Process images in batches\n# Clear CUDA cache between large batches\ntorch.cuda.empty_cache()\n```\n\n### Speed optimization\n\n```python\n# Use half precision\nsam = sam.half()\n\n# Reduce points for automatic generation\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=16,  # Default is 32\n)\n\n# Use ONNX for deployment\n# Export with --return-single-mask for faster inference\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Out of memory | Use ViT-B model, reduce image size |\n| Slow inference | Use ViT-B, reduce points_per_side |\n| Poor mask quality | Try different prompts, use box + points |\n| Edge artifacts | Use stability_score filtering |\n| Small objects missed | Increase points_per_side |\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\n- **Demo**: https:\u002F\u002Fsegment-anything.com\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `clip`: OpenAI's model connecting vision and language.\n- `generate-image`: Generate or edit images with AI models through the OpenRouter Image API (Gemini, Seedream, Recraft, GPT-Image, Riverflow).\n- `blip-2-vision-language`: Explains how to use Salesforce BLIP-2 (Q-Former bridging a frozen image encoder and an LLM such as OPT or FlanT5) through HuggingFace Trans…\n",{"name":5,"license":15,"metadata":53,"description":7},{"tags":54,"version":55,"category":56,"maintainer":57,"dependencies":58},"Multimodal, Image Segmentation, Computer Vision, SAM, Zero-Shot","1.0.0","multimodal-and-emerging","Kalaris Labs","segment-anything, transformers>=4.30.0, torch>=1.7.0","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model",491,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-14 14:06:13",{"source":8,"skillName":5,"commands":70,"prompt":80,"zipAvailable":81},[71,74,77],{"tool":72,"command":73},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill segment-anything-model",{"tool":75,"command":76},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill segment-anything-model",{"tool":78,"command":79},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill segment-anything-model","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002FSKILL.md",true,[83,89,93],{"path":84,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Fadvanced-usage.md","REFERENCE","参考文档",15803,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002Freferences\u002Fadvanced-usage.md",{"path":90,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Finteractive-segmentation.md",1598,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002Freferences\u002Finteractive-segmentation.md",{"path":94,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":95,"sourceHtmlUrl":96},"references\u002Ftroubleshooting.md",11914,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002Freferences\u002Ftroubleshooting.md",[98,107,115,123,131,139],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":102,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":106},"01M7DV332YQMGD8WTRJ9DZQXTB","worktable-review-with-annotations","Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",{"skillNo":108,"name":109,"nameZh":6,"description":110,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":111,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":106},"01M7DV2YGHCZVBEB1AHYB6CGHE","worktable-find-and-synthesize-context","Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FSKILL.md",[],[],{"skillNo":116,"name":117,"nameZh":6,"description":118,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":119,"sourceHtmlUrl":120,"categories":121,"keywords":122,"publishedAt":106},"01M7DV2TQN1H11ME2SAF9MXN7V","worktable-create-or-update-docs","Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FSKILL.md",[],[],{"skillNo":124,"name":125,"nameZh":6,"description":126,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":127,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":106},"01M7DV2NE452NEA8TXATKSNAJH","worktable-create-or-manage-records","Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FSKILL.md",[],[],{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":135,"sourceHtmlUrl":136,"categories":137,"keywords":138,"publishedAt":106},"01M7DV2HENRBR2KSCVM064WH0Y","worktable-create-or-manage-interactive-html","Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FSKILL.md",[],[],{"skillNo":140,"name":141,"nameZh":6,"description":142,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":143,"sourceHtmlUrl":144,"categories":145,"keywords":146,"publishedAt":106},"01M7DV2CADR5R7RQFFYPNAGR9Y","worktable-collaborate-threads","Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FSKILL.md",[],[]]