Open source
35 updates on Open source.
TinyLlama pretrains a 1.1B model on 3 trillion tokens
Singapore University of Technology and Design trained a 1.1B model on 3 trillion tokens with 16 A100-40G GPUs and released it under Apache 2.0.
Apple publishes MLX, where CPU and GPU share arrays without copies
Apple machine learning research released an array framework for Apple silicon with a unified memory model, lazy evaluation and Swift bindings for iOS.
LLM.swift wraps llama.cpp for on-device text generation in Swift apps
A Swift package that wraps llama.cpp so iOS, macOS, watchOS, tvOS and visionOS apps can load a GGUF file and generate text on the device.
MIT HAN Lab publishes TinyChatEngine for 4-bit LLMs on laptops and Raspberry Pi
The from-scratch C and C++ engine from Song Han's group at MIT runs 4-bit LLaMA2-7B on x86, Apple silicon, Raspberry Pi and Nvidia GPUs.
Hugging Face publishes swift-transformers for Core ML models in Swift apps
A Swift package with tokenizers, Hub downloads and a Core ML generation loop, demonstrated with Llama 2, Falcon, StarCoder and GPT models.
LLMFarm runs llama.cpp models offline on iOS and macOS
The iOS and macOS app loads a quantised model file off the device and chats with it offline, with 3B the practical size on an iPhone.
RWKV trains like a transformer and runs with constant memory per token
The paper releases pretrained RNN weights from 169M to 14B parameters and reports constant time and memory per token during inference.
MLC LLM brings local language models to iPhone, browsers and consumer GPUs
A compiler stack built on Apache TVM deploys chat models natively to iOS, browsers and consumer GPUs, with an iPhone build handed out through TestFlight.
Sherpa runs LLaMA on an Android phone through a Flutter chat app
The Flutter app loads a ggml model file off the phone and chats with it offline, and development stopped three weeks after it started.
Georgi Gerganov publishes llama.cpp, LLM inference in plain C and C++
The dependency-free C and C++ inference engine supports 1.5-bit to 8-bit quantisation and treats Apple silicon as a first-class target.
George Hotz starts tinygrad, a framework that ports to an accelerator in about 25 ops
The tiny corp framework caps its repository at 26,500 lines, ships Metal, Adreno and WebGPU backends, and runs openpilot on a Snapdragon 845 GPU.