ai-multimodal
Process and generate multimedia content using Google Gemini API. Capabilities include analyze audio files (transcription with timestamps, summarization, speech understanding, music/sound analysis up to 9.5 hours), understand images (captioning, object detection, OCR, visual Q&A, segmentation), process videos (scene detection, Q&A, temporal analysis, YouTube URLs, up to 6 hours), extract from documents (PDF tables, forms, charts, diagrams, multi-page), generate images (text-to-image with Imagen 4, editing, composition, refinement), generate videos (text-to-video with Veo 3, 8-second clips with native audio). Use when working with audio/video files, analyzing images or screenshots, processing PDF documents, extracting structured data from media, creating images/videos from text prompts, or implementing multimodal AI features. Supports Gemini 3/2.5, Imagen 4, and Veo 3 models with context windows up to 2M tokens.
- License 许可
- MIT
- 最近更新
- 2026-04-16
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- ai-multimodal
- license
- MIT
- description
- Process and generate multimedia content using Google Gemini API. Capabilities include analyze audio files (transcription with timestamps, summarization, speech understanding, music/sound analysis up to 9.5 hours), understand images (captioning, object detection, OCR, visual Q&A, segmentation), process videos (scene detection, Q&A, temporal analysis, YouTube URLs, up to 6 hours), extract from documents (PDF tables, forms, charts, diagrams, multi-page), generate images (text-to-image with Imagen 4, editing, composition, refinement), generate videos (text-to-video with Veo 3, 8-second clips with native audio). Use when working with audio/video files, analyzing images or screenshots, processing PDF documents, extracting structured data from media, creating images/videos from text prompts, or implementing multimodal AI features. Supports Gemini 3/2.5, Imagen 4, and Veo 3 models with context windows up to 2M tokens.
- allowed-tools
- [ "Bash", "Read", "Write", "Edit" ]
02 资源文件 26
- 其他.env.example6.8 KB
- 参考文档plans/251120-2323-google-models-update/phase-01-model-catalog.md5.4 KB
- 参考文档plans/251120-2323-google-models-update/phase-02-environment-config.md6.8 KB
- 参考文档plans/251120-2323-google-models-update/phase-03-image-generation.md9.7 KB
- 参考文档plans/251120-2323-google-models-update/phase-04-video-generation.md15.6 KB
- 参考文档plans/251120-2323-google-models-update/phase-05-script-enhancement.md13.8 KB
- 参考文档plans/251120-2323-google-models-update/phase-06-reference-docs.md8.4 KB
- 参考文档plans/251120-2323-google-models-update/phase-07-testing.md16.3 KB
- 参考文档plans/251120-2323-google-models-update/plan.md2.4 KB
- 参考文档references/audio-processing.md9.5 KB
- 参考文档references/image-generation.md18.1 KB
- 参考文档references/video-analysis.md11.6 KB
- 参考文档references/video-generation.md10.8 KB
- 参考文档references/vision-understanding.md10.8 KB
- 脚本scripts/.coverage52.0 KB
- 脚本scripts/check_setup.py8.5 KB
- 脚本scripts/document_converter.py12.2 KB
- 脚本scripts/gemini_batch_process.py28.2 KB
- 脚本scripts/media_optimizer.py16.1 KB
- 脚本scripts/requirements.txt442 B
- 脚本scripts/tests/.coverage52.0 KB
- 脚本scripts/tests/requirements.txt352 B
- 脚本scripts/tests/test_document_converter.py2.2 KB
- 脚本scripts/tests/test_failures.log17.6 KB
- 脚本scripts/tests/test_gemini_batch_process.py12.4 KB
- 脚本scripts/tests/test_media_optimizer.py11.5 KB
03 SKILL.md 正文
AI Multimodal
Process audio, images, videos, documents, and generate images/videos using Google Gemini's multimodal API.
Setup
export GEMINI_API_KEY="your-key" # Get from https://aistudio.google.com/apikey
pip install google-genai python-dotenv pillow
Quick Start
Analyze media: python scripts/gemini_batch_process.py --files <file> --task <analyze|transcribe|extract>
Generate content: python scripts/gemini_batch_process.py --task <generate|generate-video> --prompt "description"
Models
Image generation: imagen-4.0-generate-001 (standard), imagen-4.0-ultra-generate-001 (quality), imagen-4.0-fast-generate-001 (speed)
Video generation: veo-3.1-generate-preview (8s clips with audio)
Analysis: gemini-2.5-flash (recommended), gemini-2.5-pro (advanced)
Scripts
gemini_batch_process.py: Process audio/image/video/PDF, generate images/videos media_optimizer.py: Compress/resize/convert media for API limits document_converter.py: Convert DOCX/XLSX/PPTX to PDF
Use --help for options.
References
Load for detailed guidance:
| Topic | File |
|---|---|
| Audio | references/audio-processing.md |
| Images | references/vision-understanding.md |
| Image Gen | references/image-generation.md |
| Video | references/video-analysis.md |
| Video Gen | references/video-generation.md |
Limits
Formats: Audio (WAV/MP3/AAC, 9.5h), Images (PNG/JPEG/WEBP, 3.6k), Video (MP4/MOV, 6h), PDF (1k pages) Size: 20MB inline, 2GB File API
04 同仓库更多 Skills
web-frameworks
Build modern full-stack web applications with Next.js (App Router, Server Components, RSC, PPR, SSR, SSG, ISR), Turborepo (monorepo management, task pipelines, remote caching, parallel execution), and RemixIcon (3100+ SVG icons in outlined/filled styles). Use when creating React applications, implementing server-side rendering, setting up monorepos with multiple packages, optimizing build performance and caching strategies, adding icon libraries, managing shared dependencies, or working with TypeScript full-stack projects.
ui-styling
Create beautiful, accessible user interfaces with shadcn/ui components (built on Radix UI + Tailwind), Tailwind CSS utility-first styling, and canvas-based visual designs. Use when building user interfaces, implementing design systems, creating responsive layouts, adding accessible components (dialogs, dropdowns, forms, tables), customizing themes and colors, implementing dark mode, generating visual designs and posters, or establishing consistent styling patterns across applications.
threejs
Build immersive 3D web experiences with Three.js - WebGL/WebGPU library for scenes, cameras, geometries, materials, lights, animations, loaders, post-processing, shaders (including node-based TSL), compute, physics, VR/XR, and advanced rendering. Use when creating 3D visualizations, games, interactive graphics, data viz, product configurators, architectural walkthroughs, or WebGL/WebGPU applications. Covers OrbitControls, GLTF/FBX loading, PBR materials, shadow mapping, post-processing effects (bloom, SSAO, SSR), custom shaders, instancing, LOD, animation systems, and WebXR.
skill-creator
Guide for creating effective skills, adding skill references, skill scripts or optimizing existing skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, frameworks, libraries or plugins usage, or API and tool integrations.
sequential-thinking
Apply structured, reflective problem-solving for complex tasks requiring multi-step analysis, revision capability, and hypothesis verification. Use for complex problem decomposition, adaptive planning, analysis needing course correction, problems with unclear scope, multi-step solutions, and hypothesis-driven work.
research
Use when you need to research, analyze, and plan technical solutions that are scalable, secure, and maintainable.