On-device
Quant.npu runs 4-bit LLMs up to 15.1 percent faster on a Qualcomm NPU
Eight authors report a fully static integer quantisation method that runs 4-bit models up to 15.1 percent faster on a Qualcomm SM8650 NPU.
Tencent open-sources a 440 MB offline translation model for phones
Hy-MT1.5-1.8B-1.25bit compresses a 1.8-billion-parameter translation model from 3.3 GB to 440 MB and runs offline on a phone. Weights and an Android demo are public.
Airgap is a React Native kit for support chatbots that answer offline
Xavier Puspus published a React Native kit for support chatbots that answer without a network, running a 2.4 GB Gemma 4 E2B file through llama.rn.
Gemma 4 lands on the edge with Agent Skills in Google AI Edge Gallery
Google says Gemma 4 E2B runs in under 1.5 GB on some devices and reaches 3,700 prefill tokens per second on a Qualcomm Dragonwing IQ8 NPU.
iPhone 16 Pro settles 41.5 percent below peak over 20 back-to-back prompts
Four authors ran Qwen 2.5 1.5B 20 times in a row on four edge platforms and report the iPhone 16 Pro settling 41.5 percent below its peak throughput.
Meta's MobileLLM-Flash goes shallow and wide for 1.8 times faster prefill
Meta designed 350M, 650M and 1.4B models by measuring latency on a Galaxy S25, reversing the deep-and-thin rule of the first MobileLLM.
FlexServe runs on-device LLM inference inside TrustZone for 4.41 percent more latency
Shanghai Jiao Tong University researchers kept model weights and prompts inside ARM TrustZone at 4.41 percent more time to first token.
Alibaba adds 0.8B and 2B sizes to Qwen3.5, with 262K context and a vision encoder
Alibaba released Qwen3.5-0.8B and Qwen3.5-2B, dense vision-language models with a 262,144-token context, Apache 2.0 weights and 4-bit MNN builds.
ClawMobile tries system commands before screen taps and finishes all six test tasks
An agent runtime that runs on the phone itself routes tasks to ADB commands before UI taps, and the authors report 100 percent completion on six tasks.
Apple ships Python bindings for the on-device Foundation Models framework
The apple-fm-sdk package calls the on-device Apple Intelligence model from Python on macOS 26, for scripting and batch evaluation outside Swift.
Show HN: Off Grid runs text, image, vision and speech models offline on phones
An MIT-licensed Android and iOS app bundles llama.cpp, Stable Diffusion and Whisper for offline use. The Show HN thread reached 124 points and 66 comments.
Intelligence per watt puts local model coverage at 88.7% of real queries
Stanford and Together AI measured 20+ local models on 1M real queries and propose accuracy per watt as the metric for local inference.