Mobile AI news ticker

Google has detailed LiteRT-LM, the runtime it uses to run Gemma 4 on-device in Chrome, ChromeOS, the Pixel Watch and the Google AI Edge Gallery app. It sits on top of LiteRT, formerly TensorFlow Lite, and adds GenAI libraries over the XNNPACK and ML Drift kernels.

Running Gemma 4 E2B without multi-token prediction, Google measures 52 decode tokens per second on the Android GPU through OpenCL, 56 on iOS through Metal, and up to 76 in the browser on WebGPU. The Android figures come from a Samsung S26 Ultra, iOS from an iPhone 17 Pro, and web from Chrome on a 2024 MacBook Pro with an M4 Max.

The published comparison puts LiteRT-LM prefill at 3,808 tokens per second against 292 for llama.cpp on the Android GPU, 2,878 against 1,493 for MLX on iOS, and 4,182 against 1,127 for ONNX on the web. Decode figures in the same chart are 52 against 19, 56 against 32 and 76 against 29.

Multi-token prediction raises decode further. Google reports up to a 2.2x speedup, taking Gemma 4 E2B from 52 to 85 tokens per second and E4B from 21 to 47, measured on a Samsung S26 Ultra GPU. The drafter and the primary model run on the same hardware block so the shared KV cache and activations stay in local memory.

On memory, LiteRT-LM keeps per-layer embeddings out of memory and loads image and audio encoders only when a task needs them. Google states that the 2.58 GB Gemma 4 E2B model runs with a physical footprint of 607 MB on Apple mobile CPUs using XNNPACK weight caching.

For agentic use the runtime supports Thinking Mode, which developers can stream to the interface or strip to save KV cache space:

By dedicating a scratchpad for step-by-step reasoning before the model commits to an action, LiteRT-LM can significantly improve the output quality.

Constrained decoding enforces JSON schemas on tool payloads, and native function calling pauses execution, returns the tool-call request to the app and resumes on the tool’s output. Session save and restore serialise KV cache state so long conversations skip the prefill phase on return.

Beyond the existing Kotlin and C++ interfaces on Android, Google is adding an open-source Swift API for Apple platforms and a JavaScript API for the web through WebAssembly and WebGPU.

Eight authors published Quant.npu on May 19, 2026, a way of quantising 1B to 3B language models so they run on a phone’s neural processing unit with no rescaling at runtime. On a Qualcomm SM8650 NPU they measure their 4-bit weight, 8-bit activation build of Llama-3.2-3B-Instruct finishing up to 15.1 percent faster than the same model under ExecuTorch’s 4-bit per-block weights with 16-bit activations, for about 1 percentage point of zero-shot accuracy. Across every model and task in the paper the authors put the average cost at a 2.58 percent accuracy drop and a 1.23 rise in perplexity.

Quantisation stores a tensor as small integers plus a scale factor that maps them back to real values. Most published 4-bit methods, including the QuaRot and SpinQuant work Quant.npu is measured against, work that scale out while the model runs by scanning each activation tensor for its largest value. A mobile NPU compiles its graph before the model ships and expects every scale to be fixed by then, because that scan is a reduction the hardware is poor at, at least 7 vector instruction cycles for a 128-element 8-bit vector by the authors’ count, and because two of the four rotation steps these methods use against outliers need floating-point matrix multiplications during inference, which the paper puts at roughly double the compute. Anything left dynamic therefore has to run outside the compiled integer pipeline, and the authors report that simply freezing a model optimised under dynamic quantisation costs 15.61 percent accuracy on SmolLM2-1.7B-Instruct.

Quant.npu learns the scales during optimisation instead and folds its two rotation matrices into the weights offline, so the shipped graph is integer arithmetic end to end. In the paper’s main table for Llama-3.2-3B-Instruct at 4-bit weights and 8-bit activations, that lifts average zero-shot accuracy across six benchmarks to 58.27 percent against 46.23 for SpinQuant and 47.66 for QuaRot in the same static setting, with perplexity on C4 falling from 28.78 to 19.16. Against the slower ExecuTorch configuration it beats on latency, though, Quant.npu is the one behind, scoring lower on five of the six benchmarks, 1.08 points lower on average and 0.97 higher in perplexity, and FP32 stays ahead of both at 61.3 percent.

On the newer SM8750 NPU the authors report peak memory of 1876 MB to 1985 MB for their 3B builds against 2216 MB to 2435 MB for ExecuTorch, and prefill of 969.72 tok/s for Qwen2.5-3B-Instruct on the DroidTask set against 808.10. They name two limits of their own, that the pipeline still keeps a few operations such as the SiLU activation at 16 bits to hold accuracy, which they call a bottleneck against a fully low-bit pipeline, and that they use no method for picking calibration data even though the optimisation is sensitive to it. Quantising Qwen3-1.7B takes about 2.5 hours on NVIDIA A40 GPUs, and the preprint announces no code release.

Tencent Hunyuan has open-sourced Hy-MT1.5-1.8B-1.25bit, a translation model that runs fully offline on a phone. Quantisation to 1.25 bits cuts the file from 3.3 GB at FP16 to 440 MB; a 2-bit build is 574 MB. The announcement states the result is 25 percent smaller and around 10 percent faster than prior 1.67-bit approaches with no loss of accuracy.

The compression uses Sherry, a ternary quantisation framework with 3:4 fine-grained sparsity: of every four weights, the three most important are stored in 1 bit as -1 or +1 and the fourth is zeroed. A custom STQ kernel aligns the format to the SIMD instruction set of mobile CPUs. Running it requires llama.cpp with STQ1_0 kernel support, added in pull request 22836, and the model card links a side-by-side speed demo recorded on a Snapdragon 888 with 8 GB of RAM.

The model covers 33 languages, 5 dialects and minority languages, and 1,056 translation directions, including Tibetan and Mongolian. On the FLORES-200 benchmark Tencent reports it ahead of Tower-Plus-72B, Qwen3-32B, Microsoft Translator and Doubao on Chinese-to-foreign pairs, and the announcement claims it outperforms Google Translate and matches 235B-scale models.

Weights are on Hugging Face alongside an Android demo APK for offline translation. The code is released under the AngelSlim licence rather than a standard open-source licence.

Xavier Puspus published Airgap on April 29, 2026, a React Native starter kit for branded customer support apps that have to keep answering when the network is slow, missing or deliberately blocked. Its configuration file pins a Gemma 4 E2B instruction-tuned model in Q3_K_S GGUF form, about 2.4 GB, which the app downloads on first launch and runs through llama.rn, the React Native binding for llama.cpp. The README states that Airgap searches bundled local documents first and uses a model only to phrase what that retrieval returned, and it lists Apple Foundation Models and Google’s ML Kit GenAI Prompt API as two further on-device answer providers an operator can put ahead of the downloaded file.

The repository ships seven example configurations, for airline, banking, electric utility, healthcare, insurance, telecom and water utility support, each with its own branding, prompts and JSON knowledge files, and the downloaded-model path targets iOS 15.1 and newer and Android API 24 and newer. Airgap is published under the MIT license. It is an early single-author project, with 54 commits and no tagged release, and almost all of the code arrived in one squashed initial commit followed by a burst of work in August 2026. The README lists physical-device evaluation of the Apple and Android system providers as an open release gate and labels the published demo recordings as Android emulator and iOS simulator footage rather than evidence of either system model running on a phone.

Researchers at Meta AI and KAUST published Tempo on April 9, 2026, a way to hand a long video to a small vision-language model first and let a larger language model answer the question afterwards. On LVBench, a benchmark whose clips average 4101 seconds, the authors report their 6B system scoring 52.7 against 48.2 for VideoChat-Flash 7B, the strongest specialised baseline in their table, and against 30.8 for GPT-4o and 33.1 for Gemini 1.5 Pro.

The small model does the reading and the large one does the reasoning. A Qwen3-VL-2B-Instruct compressor takes an eight-frame chunk of video together with the user’s question and writes out a short run of memory tokens that hold what the chunk contains, and a Qwen3 4B language model then reads the whole condensed sequence with timestamps attached and produces the answer. A linear projector joins the two, for 6B parameters in total. The compressor is the part that could in principle sit on a phone, since it never sees more than a few seconds of video at a time.

How many tokens each chunk gets is decided during the same forward pass. The authors insert a yes or no question about whether the segment is relevant to the query, read the difference between the two logits as a score, and keep more memory tokens for chunks that score high, spending between 0.5 and 16 tokens per frame under a fixed overall budget of 4K or 8K visual tokens. On LVBench under the 4K budget the measured average comes to 2.9 tokens per frame. The tighter budget also scores better than the looser one on that benchmark, 52.7 against 52.3, and on the long subset of Video-MME, 57.8 against 57.0. Raising the cap to 2048 frames and a 12K budget brings LVBench to 53.7.

On the other three benchmarks the authors report 67.8 on Video-MME under the 4K budget against 65.3 for VideoChat-Flash and 61.9 for the Qwen3-VL 2B model the compressor starts from, plus 75.6 on MLVU and 65.1 on LongVideoBench. All of these are accuracy scores run through lmms-eval, not speed or memory figures. Nothing in the paper was measured on a phone, a tablet or any other edge device. Training used 32 and then 64 Nvidia H100 GPUs, the paper reports no latency, throughput or memory footprint for the compressor on mobile hardware, and the on-device case rests on the size of the model rather than on a measurement.

The authors state that all training datasets they used are publicly available, and they link code and a project page from the paper, with the repository carrying an Apache 2.0 license. They describe no weights release in the paper itself. Their stated next steps are to let the compressor decide for itself when it has gathered enough evidence to stop, which they say would bottleneck inference latency as things stand.

Google DeepMind has launched Gemma 4 under the Apache 2.0 licence, and the Google AI Edge team has described how the E2B and E4B variants run on edge hardware. The models support more than 140 languages and a 128K context window.

Gemma 4 enables multi-step planning, autonomous action, offline code generation, and even audio-visual processing, all without specialized fine-tuning.

The Google AI Edge Gallery app on iOS and Android gains Agent Skills, which Google calls one of the first applications to run multi-step, autonomous agentic workflows entirely on-device. The examples given are querying Wikipedia, turning speech input into charts and flashcards, pairing photos with generated music, and building a small app that plays animal calls through conversation alone.

For in-app deployment Google points to LiteRT-LM. It reports that Gemma 4 E2B runs in under 1.5 GB of memory on some devices using 2-bit and 4-bit weights with memory-mapped per-layer embeddings, and that GPU optimisations process 4,000 input tokens across two skills in under 3 seconds. Constrained decoding enforces structured output, and dynamic context lengths let a single model span CPUs and GPUs.

On smaller hardware, LiteRT-LM reaches 133 prefill and 7.6 decode tokens per second on a Raspberry Pi 5 CPU, and 3,700 prefill and 31 decode tokens per second on the NPU of a Qualcomm Dragonwing IQ8, the processor behind the Arduino VENTUNO Q.

Gemma 4 is available with CPU and GPU support on Android and iOS, system-wide on Android through the new AICore Developer Preview, on Windows, Linux and macOS via Metal, and in the browser through WebGPU. Google also released a Python package and a litert-lm CLI for Linux, macOS and Raspberry Pi, with tool calling included.

Four authors at Conscious Engines published a benchmark of Qwen 2.5 1.5B at 4-bit across four edge platforms on March 24, 2026, feeding each device the same 258-token prompt 20 times in a row with a 1-second gap instead of once. On an iPhone 16 Pro the first two runs peak at 40.49 tokens per second, and from run 17 onwards the phone holds a plateau of 23.67 tok/s, which the authors put at 41.5 percent below that peak.

The iPhone passes through the three thermal states iOS reports to an app. Runs 1 and 2 sit in Normal and average 36.83 tok/s, runs 3 to 16 sit in Warm and average 24.96 tok/s, and runs 17 to 20 sit in Hot at 23.67 tok/s with a variation of 1.2 percent. The authors write that the 1-second gap allows no thermal recovery, and they measure 5 percent of battery drained over the 20 runs, which they project to roughly 400 replies of this length per charge.

A Samsung Galaxy S24 Ultra degrades more gently in the same test. Its Adreno 750 GPU boosts to 1,000 MHz on runs 4 and 5 for a peak of 12.21 tok/s, then steps down to a 720 to 770 MHz plateau from run 8 where the authors measure 10.38 tok/s, 15 percent off the peak, with a low of 9.55 tok/s on the final run. They stress that reaching this plateau took deliberate tuning, with the display off, the context capped at 2,048 tokens and MLC-LLM’s prefill chunk cut to 128 tokens, and that earlier unconstrained runs hit an OS-enforced GPU frequency floor.

Neither phone ran the model on its NPU. The paper states that MLX dispatches to the A18 Pro GPU through Metal and does not target Apple’s Neural Engine, and that the S24 Ultra ran through MLC-LLM on the Adreno GPU, leaving the Snapdragon 8 Gen 3’s Hexagon NPU outside the measurement. The one NPU that did run the model, a Hailo-10H module in a Raspberry Pi 5, was also the only part that never throttled, holding 6.914 tok/s across all 20 runs with a variation of 0.04 percent at under 2 W, its CPU at 52.7°C and the NPU at 58.5°C, with attention layers that hailo-ollama does not offload falling back to the Pi’s Cortex-A76 CPU. A user running llama.cpp’s Hexagon NPU backend on a Snapdragon 8 Gen 3 phone reported 12.5 tok/s of generation on Gemma 3 4B and said the phone stayed cool.

A laptop RTX 4050 on battery led on speed at 131.7 tok/s and 34.12 W, its GPU climbing from 55°C to 70°C without throttling. Energy per token comes out at 270.5 mJ for the Hailo module measured across the whole system, 297.3 mJ for the RTX 4050 measured at the GPU, and 143.0 mJ for the S24 Ultra plateau derived from the Android fuel gauge with the display off. The authors state that the three readings differ in scope and call the resemblance suggestive rather than settled, and report no energy figure for the iPhone because iOS exposes no per-component power reading. They tested one model, one prompt and one unit per platform across four different runtimes, and describe the results as characterising hardware and software together rather than hardware on its own.

Meta AI published MobileLLM-Flash on March 16, 2026, three models of 350M, 650M and 1.4B parameters whose shapes were picked by measuring how fast each candidate actually ran on a phone. Meta reports the family reaching up to 1.8 times faster prefill and 1.6 times faster decode than Liquid AI’s LFM2 models on mobile CPUs, at comparable or higher accuracy, with support for up to 8k tokens of context.

Meta’s first MobileLLM paper argued in 2024 that deep and thin networks win below a billion parameters, and the Flash paper states that such deep-and-thin structures often fail to improve on-device latency. Meta exported 100 architectures to a phone and found parameter count a poor predictor of measured prefill and decode speed, at a Kendall rank correlation of 0.40, with FLOPs little better at 0.46 and 0.55. The 1.4B model therefore runs 16 layers at a width of 2048, against the 30 layers at 1280 of MobileLLM-Pro-1B.

Meta ran the latency tests on a Samsung Galaxy S25, a phone with a Snapdragon 8 Elite chipset, an octa-core CPU and 12 GB of memory, exporting each model through ExecuTorch to the XNNPACK CPU backend at 4-bit weights, 8-bit dynamic activations and a quantised KV cache, on 4 CPU threads and averaged over three runs after a warmup. At a 2k context the 650M model reaches its first token in 3.34 s against 6.01 s for LFM2 700M, and decodes at 85.35 tok/s against 53.57 tok/s. Meta repeated the 1k-context test on an iPhone 17, where the 650M model takes 2.81 s to the first token against 3.40 s for LFM2 700M.

Most of the speed comes from letting some layers skip attention, so a token passes through those blocks without being compared to every earlier token. The 350M model keeps full attention in 7 of its 12 layers and the 650M model in 8 of 13, while the 1.4B model attends in all 16. Meta reports that skipping consistently beat sliding window attention, which instead limits each token to a fixed span of recent ones, and that more than three skipped layers in a row costs accuracy at the same latency, dropping TriviaQA exact match from 33.2% to 8.8%.

Meta built the three models by pruning a shallow 1.8B variant of MobileLLM-Pro and inheriting its weights, which the paper says took about 35% of the tokens that pretraining MobileLLM-Pro required. Across nine reasoning and knowledge tasks Meta reports averages of 45.46, 48.57 and 55.06 for the three sizes, against 44.92, 48.52 and 50.48 for LFM2 at 350M, 700M and 1.2B, 51.79 for Gemma 3 1B and 49.01 for Llama 3.2 1B. After instruction tuning, the 1.4B model scores 47.89 on MMLU and 46.34 on HumanEval, against 49.30 and 37.80 for Llama 3.2 1B. The paper announces no release of the weights.

Researchers at Shanghai Jiao Tong University published FlexServe on March 10, 2026, a serving system that keeps an on-device language model’s weights and the text going through it inside ARM TrustZone’s secure world. Running Llama 3.2 3B, Llama 3.1 8B, Qwen3 1.7B and Qwen3 8B quantised to 8-bit on a NanoPC-T6 board with a Rockchip RK3588, 16 GB of memory and a 6 TOPS NPU, the authors measure time to first token 4.41 percent above the same inference run without any protection, averaged over their prompt lengths. Against a TrustZone baseline that already uses a secure NPU and a loading pipeline, they report FlexServe 1.43 to 2.42 times faster, 1.85 times on average.

TrustZone splits the chip into a normal world, where Android and every app runs, and a secure world the normal world cannot read. FlexServe puts the inference engine, the weights, the prompt, the generated text and the KV cache on the secure side, so a kernel an attacker has taken over cannot read any of it out of memory, and the authors justify that assumption with Linux’s 40 million lines of code and 9756 CVEs. Both halves are worth stealing in their account, since a vendor’s fine-tuned weights cost real money to train yet sit on a stranger’s device, and the apps calling a shared model feed it chat history and screen contents. The authors name the boundary they cannot cross, which is that an app in the normal world still receives its own input and output there, so a compromised kernel can take those, and only the weights stay protected in that case.

Plain TrustZone is slow at this because it hands out secure memory as a few large contiguous regions. Reserving 8 GB at runtime therefore means the kernel’s contiguous allocator compacting scattered pages, which the authors time at 6440.67 ms against 568.58 ms for FlexServe’s page-granular secure memory, and the secure world cannot normally touch the NPU at all, leaving 30.06 s of CPU computation where the NPU needs 1.94 s. FlexServe carves secure pages out with the stage-2 page tables that ARM’s virtualisation extension already provides, and flips the NPU between protected and unprotected mode in 0.21 ms. Handing 8 GB back drops from 732.85 ms to 80.50 ms, because the pages are released lazily rather than zeroed in one pass.

A scheduler decides which models stay resident when several share the device, holding the earliest layers of each in secure memory so the pipeline does not stall and prefetching the next model while the current one is still generating. With a 4 GB cache and requests drawn at random from UltraChat, OpenAssistant, Dolly and Alpaca, the authors measure pairs of models answering 8.84 times faster than the plain secure-world baseline and 2.53 times faster than the optimised one. Across ten agent workflows, where one model’s finished answer feeds the next, the gains reach 24.30 times and 4.05 times, averaging 14.15 and 2.94. Serving one phone-side model to several callers is also what ElastiLM works on, by resizing the shared model to each request’s latency budget rather than isolating it.

The protection costs the rest of the device something as well. A SQLite instance holding 3 GB in the normal world runs 1.31 times slower while FlexServe holds 8 GB in the background, against 3.27 times slower with the contiguous allocator, and the authors put the standing virtualisation cost at 2.46 percent on SPEC CPU, which they switch off when no secure inference is running. They list the prototype’s limits themselves, a development board rather than a phone, Llama 3 and Qwen3 series models only, no GPU in the pipeline, and an NPU runtime slow enough that computation becomes the bottleneck, so a faster NPU would widen their lead. Side-channel attacks, physical attacks and denial of service are out of scope, and the paper names no code release.

Alibaba’s Qwen team released Qwen3.5-0.8B and Qwen3.5-2B on March 2, 2026, together with 4B and 9B versions, according to the Qwen release notes. Both are dense vision-language models with a 262,144-token context and Apache 2.0 weights. The model card states that the released files work with Hugging Face Transformers, vLLM, SGLang and KTransformers, and that in light of the parameter scale the intended uses are prototyping, task-specific fine-tuning and other research or development purposes.

The benchmark table in that model card puts Qwen3.5-2B at 55.3 on MMLU-Pro with reasoning switched off, where the same table gives 40.2 for last generation’s Qwen3-1.7B and 69.6 for the larger Qwen3-4B-2507, and Qwen3.5-0.8B at 29.7. With reasoning on, Alibaba reports 66.5 for the 2B and 42.3 for the 0.8B on the same test, and 51.6 against 11.9 on GPQA, a set of graduate-level science questions. The same table has Qwen3-1.7B ahead of both new sizes on IFEval, which checks whether a model obeys explicit formatting instructions, at 68.2 against 61.2 for the 2B and 52.1 for the 0.8B with reasoning off. Alibaba ran all of these figures itself.

Both sizes ship a vision encoder in the same checkpoint rather than as a separate release, and Alibaba’s vision table gives Qwen3.5-2B 64.2 on MMMU, a college-level image question set, above the 61.4 it lists for the previous Qwen3-VL-2B, with 49 for Qwen3.5-0.8B. The two models stack 24 layers in which three linear-attention blocks, whose memory cost does not grow with the sequence, alternate with one full-attention block. Input embedding and output layer are tied, so a single 248,320-entry matrix does both jobs, and Alibaba puts language coverage at 201 languages and dialects.

Alibaba published the weights on Hugging Face and ModelScope under Apache 2.0, and its MNN team put 4-bit exports of both sizes up the same day. The 0.8B build carries a 470 MB language weight file plus 63 MB for the vision encoder, and the 2B build 1.18 GB plus 196 MB, against 1.75 GB and 4.55 GB for the full-precision checkpoints. For other local runtimes the Qwen repository points readers to llama.cpp, which it says supports the series for text and vision, and to mlx-lm and mlx-vlm on Apple silicon, without Alibaba publishing GGUF or MLX builds of these sizes itself.

Seven researchers at MBZUAI, City University of Hong Kong and one working independently published ClawMobile on February 26, 2026, an agent runtime that runs on the Android phone itself with no tethering to a host machine. The model is not on the device. The authors state that ClawMobile runs locally while model inference is performed remotely, and all agents in their tests use GPT-5.2, so the split is an on-device runtime driving a cloud model. Across six real-life phone tasks the authors report ClawMobile completing all six at 100 percent, against 33 to 100 percent for the UI agent they compare against.

What separates it from screen-reading GUI agents is that the reasoning loop never touches the screen. The orchestrator issues tool calls to control backends instead, and the authors describe a deterministic-first policy in which the runtime first checks its memory for a structured interface, reaching for ADB system commands and the Termux hardware API before anything visual. A semantic UI agent is invoked only when a step genuinely depends on reading dynamic interface state, and raw taps on interface elements sit below that as a fallback. After every action the runtime re-queries device state to confirm the step landed, which the authors contrast with inferring execution success purely through model reasoning.

The tests ran on a Google Pixel 9 on Android 16, with DroidRun as the UI agent baseline and a stripped ClawMobile without DroidRun as a second baseline, all three on GPT-5.2 and scored by human annotators. In the authors’ table DroidRun reaches 100 percent on switching the system to dark theme and on playing a YouTube video, 85 percent on posting a YouTube comment, 73 percent on a Chrome search and on a cross-app task that searches football results and writes them into Notes, and 33 percent on installing an app from the Play Store, where ClawMobile scores 100 percent throughout. The authors put the cost of that at 57.5 seconds slower per task on average, with ClawMobile taking 21 seconds for the dark theme switch and 235 seconds for the YouTube comment. Their deterministic-only variant hits the 600-second timeout on both YouTube tasks.

The authors name the remote model as the main open problem, writing that fully remote inference introduces network latency and privacy concerns and calling on-device inference a promising direction they have not taken. They also flag that serialising full UI trees or screenshots into model context can dominate token use and latency on mobile, and present the results as preliminary, six tasks on one device. The code is open source on GitHub, built on the OpenClaw agent framework with a Telegram bot as the chat channel, and the paper was accepted at EuroMLSys 2026 under a Creative Commons Attribution 4.0 license.

Apple published python-apple-fm-sdk, Python bindings for its Foundation Models framework, with a first beta release on February 25, 2026 according to the repository’s release history. The package reaches the on-device model at the core of Apple Intelligence from Python on macOS, and installs with pip install apple-fm-sdk.

The Python API follows the Swift one. SystemLanguageModel reports whether the model is available and, if not, why, while LanguageModelSession holds a conversation and its async respond call returns text or streams it. A @fm.generable decorator marks a Python class for the model to fill in, and fm.guide adds per-field constraints such as a numeric range, as the README shows with a Cat class whose age is bounded to 0 through 20. Tool calling and session transcripts are exposed as well.

The bindings do not reimplement inference. The repository carries a Swift package named foundation-models-c that wraps the framework behind a C interface and builds it as a dynamic library, which the Python layer then calls. Apple writes in the documentation that the bindings run the Swift framework underneath, so evaluations reflect real on-device performance and behaviour, and its evaluation guide notes that inference calls are processed one at a time rather than in parallel at the macOS hardware level.

Apple lists macOS 26.0 or later, Xcode 26.0 or later with its agreement accepted in the Xcode app, Python 3.10 or later, and Apple Intelligence switched on. PyPI carries only source distributions for the package, so the Swift component is compiled locally during installation, which is what the Xcode requirement covers. Apple Intelligence on the Mac runs on Apple silicon machines only.

The code is under the Apache 2.0 license and the package metadata on PyPI classifies it as alpha. Apple pushed 0.1.0 to PyPI on March 8, 2026, added the Attachment API for sending images alongside text in 0.2.0 on June 8, 2026 for WWDC 2026, and exposed the model’s context size and the token count of a given input in 0.2.1 on June 29, 2026. The README states the project is not yet taking contributions.