On-device AI means a model processes an input on the user's hardware rather than sending that input to a remote model for inference. Inference is simply the step that produces an answer from the model.
That definition describes where one operation happens. It leaves several product questions open: how the model arrives, whether the feature is supported, what the rest of the application sends over the network, and what happens when local processing is unavailable.
Here are five answers for teams considering an AI feature in a web app. The browser examples refer to Chrome's published documentation, reviewed on October 22, 2026; this is an explainer, not a hands-on performance review or a new release announcement.
1. Does local AI mean the app works without the internet?
The model operation may work offline. The complete app might still need a connection.
Chrome's getting-started guide explains that its built-in models need an initial download. Subsequent model use does not require a network connection. The same guide describes device requirements and APIs at different stages of availability.
Now consider an illustrative writing feature inside a customer portal. Rephrasing a sentence locally would not automatically make the portal's sign-in, customer database, or document loading available offline. Those features have their own requirements.
A useful product description names the operation: “This rewrite is processed on your device.” An offline promise needs a separate explanation of what remains usable when the connection disappears, including whether the user can save the result.
2. Is everything in the app private because the model is local?
Local inference can keep the model input out of a remote inference request. It does not describe every other data flow in the application.
A support form might process a draft locally and then submit the finished message to a server. That is an ordinary part of the product, but the wording should make the distinction clear. Analytics, synchronization, error reports and third-party integrations also need their own review.
Google's Chrome help page distinguishes features that rely on on-device generative models from other AI features that may still run when those models are disabled. A browser-wide AI label therefore tells you less than the documentation for the specific feature.
For a buyer, the useful question is which information goes where during the actual task. A diagram showing only the model and the laptop is incomplete if the application also uploads the source document.
3. Will every customer have the same experience?
Availability is a product state, not just a developer setup step.
Chrome documents four readiness states for its built-in AI APIs: unavailable, downloadable, downloading and available. Hardware, requested options and model readiness can affect the result. Support for a feature on a developer's laptop is not evidence that the intended customer devices support it.
That changes the interface. Someone waiting for a model download needs a clear explanation of what is happening. Someone whose device cannot use the feature needs an alternative that does not make them feel they broke the application.
For the writing example, leaving the normal text editor usable is often a better starting point than blocking the entire form. AI can remain optional while the core task still works. This is a design recommendation, not a claim about a tested implementation.
4. Can the app quietly switch to a cloud model?
It may be technically possible to build that switch, but it changes where processing happens.
Chrome's built-in AI overview includes resources for hybrid applications with a cloud fallback. A fallback deserves a product decision of its own: when it is offered, what it sends, who provides the remote service, and what the user is told.
“Use cloud processing instead” is more informative than presenting the same button while silently changing the data destination. Where local processing is a stated requirement, preserving that requirement may mean leaving AI unavailable rather than automatically routing the content elsewhere.
The right answer depends on the feature and its commitments. A convenience tool and a workflow designed around local processing do not necessarily need the same fallback.
5. Is local automatically faster or cheaper?
Neither conclusion follows from the location alone.
Local processing uses the customer's hardware and may involve a first download. Remote processing involves a network request and a service with its own costs. The comparison needs to include waiting time, output quality, device coverage, support work and whatever happens when the preferred route fails.
For a proposed rewriting tool, measure whether the suggested text remains accurate and useful as well as how quickly it appears. Keep first-use download time separate from subsequent requests. Report results for the devices and tasks actually evaluated, rather than turning one successful demonstration into a general performance claim.
“Runs locally” is a useful architectural fact. A complete product promise explains the feature's boundaries in language the person pressing the button can understand.