Apple Silicon
19 updates on Apple Silicon.
Edge0 releases MoE expert-offloading framework, demos 35B model on iPhone
Edge0 streams mixture-of-experts weights from storage. Its 35B tier reports 2.9 GiB peak memory on a Mac mini M4 Pro; a launch post shows a 35B model on an iPhone.
Apertus Mini distils an open-data 8B into 0.5B, 1.5B and 4B models
Swiss AI distilled its fully open Apertus 8B into 0.5B, 1.5B and 4B models on 1.7T tokens, with 3-bit to 6-bit MLX builds for Apple devices.
Apple ships Python bindings for the on-device Foundation Models framework
The apple-fm-sdk package calls the on-device Apple Intelligence model from Python on macOS 26, for scripting and batch evaluation outside Swift.
Intelligence per watt puts local model coverage at 88.7% of real queries
Stanford and Together AI measured 20+ local models on 1M real queries and propose accuracy per watt as the metric for local inference.
A19 Pro puts Neural Accelerators in every GPU core
Apple says the iPhone 17 Pro chip pairs Neural Accelerators in each of six GPU cores with a 16-core Neural Engine to run large local language models.
Apple puts the cost of 2-bit compression at 3.4 MMLU points
Apple measures its on-device model at 67.8 MMLU in 16 bits and 64.4 after compression to 2 bits per weight, and details the distillation pipeline behind it.
Locally AI runs Llama, Gemma and Qwen offline on iPhone and iPad through MLX
Adrien Grondin shipped a free iPhone and iPad app that downloads open-weight models and runs them on device, built on Apple silicon through MLX.
HuggingSnap describes what the iPhone camera sees with a 500M model on the phone
Hugging Face released an iPhone app that runs SmolVLM2 at 500M parameters through MLX, describing camera scenes, photos and video with no cloud call.
Flower Intelligence runs models on device, with remote handoff off by default
Flower Labs released a preview library that runs Llama 3.2 and SmolLM2 locally via WebLLM or MLX Swift, and calls its remote service only if an app enables it.
EXO Labs benchmarks put Llama 3.1 8B at 14 tok/s on an iPhone 15 Pro
EXO Labs published automated inference benchmarks from real devices, covering iPhone 15 Pro, Galaxy S24 Ultra, Mac mini M4 Pro clusters and desktop GPUs.
Apple Intelligence pairs a 3-billion-parameter on-device model with a server model
Apple reports 0.6 ms per prompt token and 30 tokens per second on an iPhone 15 Pro for a model compressed to an average of 3.7 bits per weight.
Apple releases OpenELM at 270M to 3B, with parameters spread unevenly across layers
Apple published four models from 270M to 3B parameters with the full training framework and code to run them through MLX on Apple silicon.