flutter-edge-ai-mediapipe
Use when running .task or .bin models (MediaPipe GenAI, ModelFileType.task or ModelFileType.binary) with flutter_edge_ai_mediapipe on Android, iOS or web. Also use when CocoaPods rejects the iOS platform version, images are ignored in a MediaPipe chat, maxOutputTokens has no effect, or an iOS app is terminated while loading a .task model on the GPU. MediaPipe has no macOS, Windows or Linux support — use a .litertlm model there (flutter-edge-ai-inference).
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 662
- Forks
- 180
01 前言 Front Matter
- name
- flutter-edge-ai-mediapipe
- description
- Use when running .task or .bin models (MediaPipe GenAI, ModelFileType.task or ModelFileType.binary) with flutter_edge_ai_mediapipe on Android, iOS or web. Also use when CocoaPods rejects the iOS platform version, images are ignored in a MediaPipe chat, maxOutputTokens has no effect, or an iOS app is terminated while loading a .task model on the GPU. MediaPipe has no macOS, Windows or Linux support — use a .litertlm model there (flutter-edge-ai-inference).
03 SKILL.md 正文
The MediaPipe engine
Rules
- Depend on
flutter_edge_aiandflutter_edge_ai_mediapipe, and import both. The engine package does not re-export core. - Declare
fileType: ModelFileType.taskfor.taskfiles andModelFileType.binaryfor.binfiles. - An app that includes this package needs iOS 16.0.
- There is no desktop support.
maxTokensis the real context limit — small values are not raised as they are on.litertlm.maxOutputTokensis ignored; stop generation withsession.stopGeneration().- On Android and iOS,
openChatinherits the model's audio support but not its image support — passsupportImage: trueto the chat as well, or the image is dropped. - Upstream MediaPipe LLM Inference is maintenance-only; prefer a
.litertlmmodel (flutter-edge-ai-inference) for new work. - On iOS, do not load a
.taskmodel withPreferredBackend.gpu: Gemma 3 1B reaches the app's memory limit while loading and iOS kills the app with no crash report. Use.litertlm, orPreferredBackend.cpu.
Setup
flutter pub add flutter_edge_ai flutter_edge_ai_mediapipe
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_mediapipe/flutter_edge_ai_mediapipe.dart';
await FlutterEdgeAi.initialize(inferenceEngines: [MediaPipeEngine()]);
await FlutterEdgeAi.installModel(
modelType: ModelType.gemmaIt,
fileType: ModelFileType.task,
).fromNetwork(url).install();
final InferenceModel model = await FlutterEdgeAi.getActiveModel(maxTokens: 1024);
Sessions, chats, streaming and the common traps work as in the flutter-edge-ai-inference skill — except that on web the openSession / openChat sessions (its concurrent-conversation pattern) share the one loaded model (MediaPipe's JS LlmInference object): their turns run one at a time (a turn waits for the other sessions' turns as long as they run, unless they are stopped), each resends that session's whole conversation, and the sampling and LoRA are the ones the first session loaded the model with (a later session that asks for others gets a one-time printed warning).
Response formats are the exception: MediaPipe does not enforce them on mobile
or Web. model.structuredOutputSupport is none, refusalFor(format) returns a
ResponseFormatUnsupportedException, and
openSession / openChat(responseFormat: format) throw it before opening the
MediaPipe session.
iOS
ios/Podfile, declared once:
platform :ios, '16.0'
use_frameworks! :linkage => :static
With Swift Package Manager, also set iOS Deployment Target to 16.0 on the Runner target in Xcode. This package ships no Swift package manifest, so the app gets an ios/Podfile either way.
Core and the other engines build from iOS 15.0, except flutter_edge_ai_llama, which needs 16.4. If the app does not use .task models, leave this package out and stay on 15.
Large models also need Extended Virtual Addressing, Increased Memory Limit and Increased Debugging Memory Limit, added in Xcode under Signing & Capabilities, or the app is killed for memory.
Run .task models on PreferredBackend.cpu on iOS: on the GPU even Gemma 3 1B is killed for memory while loading, with the entitlements present.
Images and audio
final model = await FlutterEdgeAi.getActiveModel(maxTokens: 4096, supportImage: true);
final chat = await model.openChat(supportImage: true);
try {
await chat.addQueryChunk(
Message(text: 'Describe this image.', isUser: true, imageBytes: bytes),
);
final response = await chat.generateChatResponse();
} finally {
await chat.close();
}
On Android and iOS, a chat without supportImage: true drops the image and the model answers the text alone. On web the chat follows the model: an image sent to a model loaded without supportImage: true throws ArgumentError.
Audio input works on Android and iOS only with a .task that carries an audio encoder. The Gemma 3n .task files do not, so for audio use a .litertlm model (flutter-edge-ai-inference).
Bounding output
final session = await model.openSession();
await session.addQueryChunk(Message(text: prompt, isUser: true));
final reply = StringBuffer();
var produced = 0;
await for (final token in session.getResponseAsync()) {
reply.write(token);
if (++produced >= 200) {
await session.stopGeneration();
break;
}
}
await session.close();
Web
Load a web build of the model — the -web.task file, e.g. gemma3-1b-it-int4-web.task rather than gemma3-1b-it-int4.task. Mobile .task files generally do not run on web, so install a separate URL there.
Add to web/index.html <head>, before Flutter boots:
<script type="module">
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai@0.10.30';
window.FilesetResolver = FilesetResolver;
window.LlmInference = LlmInference;
</script>
<script src="cache_api.js"></script>
<script src="opfs_helper.js"></script>
Pin the version — an unpinned import takes whatever was published last. Copy cache_api.js and opfs_helper.js from the flutter_edge_ai package's web/ directory into the app's web/; find it with grep -A1 '"name": "flutter_edge_ai"' .dart_tool/package_config.json.
Web is GPU-only. Models over about 2 GB need OPFS streaming storage:
await FlutterEdgeAi.initialize(
webStorageMode: WebStorageMode.streaming,
inferenceEngines: [MediaPipeEngine()],
);
Android
Text inference runs on arm64-v8a, x86_64 and armeabi-v7a. The release build needs <uses-permission android:name="android.permission.INTERNET"/> in android/app/src/main/AndroidManifest.xml to download a model — Flutter's template declares it only for debug and profile builds.
04 同仓库更多 Skills
send-email
Send an email.
query-wikipedia
Query summary from Wikipedia for a given topic.
qr-code
Generates a QR code for the given url.
kitchen-adventure
Act as a dungeon master for a text-based adventure set in a world where everyone is a sentient kitchen appliance. Trigger when user says "start kitchen adventure".
interactive-map
Show an interactive map view for the given location.
get-current-time
Get the user's current local date, time, and day of the week.