On-Device AI

Definition

On-device AI refers to machine learning models — including language models, vision models, and classifiers — that run locally on a user’s smartphone, laptop, or desktop rather than on remote cloud servers. The model weights, inference computation, and data processing all remain on the device. This architecture is marketed as privacy-preserving because user data is not transmitted externally, but it introduces distinct concerns around transparency, consent, and resource consumption.

How It Relates to AI Threats

On-device AI intersects with threats in two directions. Within Privacy & Surveillance, local processing can create a false sense of privacy — users may not know a model is running or what data it analyzes, and consent mechanisms for on-device models lag behind those for cloud services. Within Human-AI Control, on-device models are typically managed and updated by the OS or browser vendor, with limited user visibility into model versions, capabilities, or resource consumption. The silent deployment of on-device models without informed consent represents an emerging threat vector as browsers and operating systems increasingly bundle AI capabilities.

Why It Occurs

• Vendors seek to offer AI features with reduced cloud inference costs
• On-device processing is marketed as privacy-respecting (data stays local)
• Model quantization and hardware advances make local inference feasible on consumer devices
• Automatic background updates and downloads are the default pattern for browser and OS vendors
• Consent mechanisms for on-device AI are less developed than those for cloud-based data processing

Real-World Context

In May 2026, privacy researcher Alexander Hanff documented that Google Chrome had been silently downloading an approximately 4GB Gemini Nano model to users’ devices since 2024, without clear consent or disclosure (INC-26-0098). The model powers features including scam detection, Help me write, and summaries. The practice raised questions about whether on-device model downloads require prior consent under the EU ePrivacy Directive.