Flutter On-Device AI: What Actually Runs Offline in 2026 and What Still Needs the Cloud

Flutter on-device AI works today through three paths, and none of them covers every phone. Gemini Nano runs only on a short list of recent Android flagships, and only while your app is in the foreground. Apple's Foundation Models need an Apple Intelligence device. A bundled Gemma model runs on more hardware, but it means 0.3 to 4.3 GB of model files, Android 11 or later, and a 64-bit ARM chip. This guide is for teams with a real privacy, compliance, or offline requirement, not a marketing checkbox. For those teams, the answer is a fallback architecture, not a single model.

Which Devices Can Run On-Device AI in Flutter Today?

On-device AI support in Flutter depends heavily on the operating system, device hardware, and model runtime. Before choosing a local AI approach, teams need to understand which devices can actually run the model, how much storage it requires, and whether the system exposes the required AI capabilities.

Gemini Nano is Google's on-device model, delivered to Android apps through the AICore system service. The ML Kit Prompt API, which gives apps free-form access to it, is in Beta. Google's device list for it, last updated September 28, 2026, includes the Pixel 9, 10, and 11 series, the Galaxy S26 series and recent Samsung foldables, plus a handful of devices from other makers.

The Galaxy S25 is not on the Prompt API list. It appears only on the list for Google's narrower feature APIs, such as summarization. Some Flutter plugin documentation still gives "Pixel 9 and Galaxy S25 or newer" as the floor. Treat Google's list as the authority and re-check it before every release.

Apple Foundation Models is Apple's on-device language model framework, available on devices that support Apple Intelligence. It requires iOS 26 or later, and the Flutter plugin documentation lists the iPhone 15 Pro and newer models.

What Are the Three Ways to Run On-Device AI in Flutter?

Flutter teams can approach on-device AI in three fundamentally different ways: use a system-provided model, bundle a smaller model with the application, or run a local LLM through a native inference runtime. Each option trades off device coverage, model control, application size, performance, and maintenance.

Bundled Small Models With flutter_edge_ai

flutter_edge_ai is the Flutter plugin that replaced flutter_gemma on October 5, 2026. The final flutter_gemma release was 1.11.4, and it is now marked as discontinued on pub.dev. Developers who still search for "flutter_gemma" will find that dead package. The replacement is days old, so pin its version, read its changelog before every upgrade, and expect API changes.

The plugin runs Gemma-family models through LiteRT-LM on Android. Its documented model sizes run from 0.3 GB for Gemma 3 270M to 0.5 GB for Gemma 3 1B, 2.4 GB for Gemma 4 E2B, and 4.3 GB for Gemma 4 E4B. On Android, it requires minSdk 30 and arm64-v8a devices only.

Models can come from the network, from app assets, or bundled in the app. Hugging Face downloads cannot resume if interrupted, and Android limits background downloads to about 9 minutes unless you run a foreground service. You gain control over the model and independence from AICore. You lose install size, first-run download time, and any phone below the Android 11 and 64-bit floor.

Choosing a checkpoint is a question of size, quality and language coverage. It is settled by testing candidates on your own task data, which is the core of AI model selection and evaluation work, not by reading benchmarks.

System-Provided Models With flutter_edge_ai_builtin_ai

This plugin, formerly flutter_gemma_builtin_ai, uses Gemini Nano on Android and Apple Foundation Models on iOS and macOS when the device has them. You ship zero model bytes. The cost is that availability is a runtime check, not a guarantee, so every call site needs a bundled or cloud fallback. Treat the plugin as early software: it is days old under its current name.

Full Local LLMs With llama.cpp

llama.cpp is an open-source C++ engine that runs quantized language models on CPUs and mobile GPUs. In Flutter it is usually reached through Dart FFI, for example with the llama_cpp_dart package, with platform channels as the alternative. FFI calls native code directly, and inference runs off the UI thread.

You get the most control: any compatible model, any quantization level, any runtime setting. You also own memory limits, thermal behavior, and model delivery. The package's 0.9.0 release was a rewrite, and its documentation expects at least one more breaking change before 1.0.

When Is On-Device AI the Right Choice for a Flutter App?

On-device inference is most valuable when the product requirement is stronger than simply "add AI." Privacy, offline operation, and response latency can justify the additional complexity of local inference. The decision should start with the business requirement and data flow rather than the model itself.

On-device AI fits three cases.

• Data-residency requirements: The prompt and the response never leave the phone.
• Offline use: Fieldwork, travel, and low-connectivity regions.
• Latency-sensitive features: There is no network round trip, and Gemini Nano supports streaming for faster first output.

Take care with compliance claims. ML Kit processes input and output on the device and does not send them to Google servers. It does send performance and usage metrics to Google, and Google's terms make you responsible for telling users about that processing. Check this against your store disclosures and your compliance team before telling anyone data never leaves the device.

If you are evaluating AI for an existing product, our AI Development Services cover the broader architecture, integration, and development approaches beyond on-device inference.

When Does On-Device AI Fall Short?

Local inference introduces constraints that cloud-based AI largely hides from application developers. Device compatibility, foreground execution, model quotas, context limits, language coverage, and evolving platform APIs can all affect whether an on-device feature is reliable enough for production.

Five limits show up before quality does.

Foreground Execution Limits

Gemini Nano inference is permitted only while your app is the top foreground application. Using it from a foreground service returns an error. A feature that processes documents overnight cannot run on it.

AICore Quotas

AICore enforces a per-app quota, and too many requests in a short period return a busy error.

Language Support Limitations

The Summarization API supports English, Japanese and Korean.

Context Window Constraints

Apple's on-device model launched with a 4,096-token window. Since iOS 26.4 you can read the actual size at runtime, so check it instead of assuming.

Beta and Device Compatibility Risks

The Prompt API is Beta, and Google lists devices by Gemini Nano version. Test on every version you target.

Today, "private AI for everyone" means private AI for owners of some recent Pixel, Galaxy and Apple Intelligence devices, with a fallback plan for everyone else.

What Does a Fallback Architecture Look Like for On-Device AI in Flutter?

A fallback architecture checks availability at runtime and moves down a ranked list of providers. A feature that works on a Pixel 10 and fails silently elsewhere is a defect, not a limitation.

• Define one interface that all providers implement.
• At startup, check whether the system model is available.
• If not, check whether the device can run a bundled model (Android 11 or later, 64-bit ARM, enough free storage). Show the download size before downloading.
• If neither works, call a cloud API, but only if your compliance requirement allows it. Otherwise, disable the feature and tell the user why.
• Treat busy, background-blocked, and quota errors as expected states: back off and retry, never fail silently.

dart

            abstract class AiProvider {
            Futurebool isAvailable();
            StreamString generate(String prompt);
            }
            Future< AiProvider? > pickProvider(List< AiProvider > ranked) async {
            for (final p in ranked) {
            if (await p.isAvailable()) return p;
            }
            return null; // show "not available on this device"
            }
    

The trade-off is three code paths and a larger test matrix. Different models also answer differently, so write prompts for the weakest model in the list, not the strongest. The cloud branch is an ordinary API integration, but production AI applications also need to consider provider abstraction, streaming, security, and cost controls. Our guide to Next.js AI integration and LLM architecture explores these considerations in more detail.

How Should You Choose an On-Device AI Approach for Flutter?

The right implementation depends on what the application must guarantee: data residency, device coverage, offline operation, background processing, model control, or a small application footprint. These requirements often point to different architectures, so there is no single "best" on-device AI solution for every Flutter application.

Four questions decide the path.

If you are weighing these paths for a Flutter app, talk to iSyncEvolution about Flutter app development. Not sure on-device AI is right for your app? Talk to us anyway.

On-Device AI vs Cloud AI: Which Approach Fits Your Requirements?

On-device and cloud AI solve different engineering problems. Local inference provides stronger control over data movement and offline availability, while cloud inference generally offers broader device coverage, larger models, and easier centralized model management. Comparing the two against product requirements makes the trade-off clearer.

Conclusion

On-device AI can improve privacy, offline availability, and response latency, but device fragmentation makes a single-model strategy unreliable. The right approach combines runtime capability detection, local inference where appropriate, and controlled cloud fallback when required. By evaluating device compatibility, model performance, data requirements, and deployment constraints early, teams can build Flutter applications that deliver reliable AI experiences across supported devices. Building an AI-powered Flutter app? Contact iSyncEvolution to discuss your project and find the right on-device and cloud AI architecture for your application.

FAQ

Does Gemini Nano Work on All Android Phones?

No. The ML Kit Prompt API works on a short list of recent devices, including the Pixel 9, 10, and 11 series and the Galaxy S26 series. The Galaxy S25 is not on that list. Check Google's current device list before relying on it.

What Is flutter_gemma?

flutter_gemma was a Flutter plugin for running Gemma models on-device. Its final release, 1.11.4, shipped on October 5, 2026, and it is discontinued. Its replacement is flutter_edge_ai.

Can I Run a Full LLM On-Device in Flutter?

Small ones, yes. The largest model flutter_edge_ai documents is Gemma 4 E4B at 4.3 GB, and Android needs minSdk 30 and a 64-bit ARM chip. Cloud-scale models do not run on phones.

Is On-Device AI Faster Than Cloud AI?

Not reliably. On-device inference removes the network round trip, but small models on phone chips can generate more slowly than cloud hardware. Measure on the devices you target.