iPhone
17 updates on iPhone.
Microsoft runs Phi-3-mini offline on an iPhone 14 at over 12 tokens per second
The 3.8-billion-parameter model takes about 1.8 GB at 4-bit and scores 69 percent on MMLU, which Microsoft compares to Mixtral 8x7B and GPT-3.5.
Meta's MobileLLM trades width for depth and gains 2.7 and 4.3 points
Meta Reality Labs built 125M and 350M models around deep and thin layers and profiled them on an iPhone 13 through ExecuTorch.
LLMFarm runs llama.cpp models offline on iOS and macOS
The iOS and macOS app loads a quantised model file off the device and chats with it offline, with 3B the practical size on an iPhone.
MLC LLM brings local language models to iPhone, browsers and consumer GPUs
A compiler stack built on Apache TVM deploys chat models natively to iOS, browsers and consumer GPUs, with an iPhone build handed out through TestFlight.
Hugging Face's DistilBERT is 40% smaller than BERT and 71% faster on an iPhone
The 66M-parameter distilled BERT keeps 97% of its teacher on GLUE and weighs 207 MB inside a question-answering app on an iPhone 7 Plus.