Research
65 updates on Research.
Alibaba builds Qwen2's 0.5B and 1.5B sizes for phones, earphones and glasses
Alibaba built Qwen2-0.5B and Qwen2-1.5B for smartphones, earphones and smart glasses, with 32K context and Apache 2.0 weights.
llm.npu prefills 1,106 tok/s for Qwen1.5-1.8B on a Snapdragon 8 Gen 3
Peking University and BUPT move prompt processing onto the Hexagon NPU and report up to 43.6x faster prefill and up to 59.5x lower energy use.
Apple's MobileCLIP-S0 encodes an image in 1.5 ms on an iPhone 12 Pro Max
Apple timed its image-text models on an iPhone and released four variants, the weights and the reinforced DataCompDR dataset.
BUPT measures 22 LLMs on four Android phones at about 200 ms per token
A BUPT team ran 22 models from 0.5B to 7B on four Android phones with llama.cpp and found 7B models take about 4 GB and 200 ms per token.
PowerInfer-2 runs a 47B model on a OnePlus 12 at 11.68 tokens per second
Shanghai Jiao Tong University researchers report a 47B model decoding at 11.68 tokens per second on a OnePlus 12, with weights streamed from flash.
BUPT proposes one 9.2B model in the OS that all apps call through adapters
A BUPT team proposes that the phone OS ship one 9.2B multimodal model all apps call through small adapters, and matched app models on 85% of 50 datasets.
Apple releases OpenELM at 270M to 3B, with parameters spread unevenly across layers
Apple published four models from 270M to 3B parameters with the full training framework and code to run them through MLX on Apple silicon.
Octopus v3 picks an action from an image and a query in under 1B parameters
A technical report describes a sub-billion multimodal agent that turns an image and a text query into a function call, shown through ten worked examples.
Octopus v2 is a 2B model that calls Android APIs with one token per function
A 2B Gemma fine-tune gives every Android API its own token, and the authors report 99.524% accuracy and 0.38 seconds per call, ahead of GPT-4.
Octopus fine-tunes a 2B model to 93 percent on API function calls
Stanford and Harvard authors fine-tuned open 2B to 7B models on 20,000 RapidAPI functions and report up to 97 percent function call accuracy.
One shared on-device LLM keeps a context per app and switches in 0.27 seconds
Peking University, BUPT and Tsinghua researchers compress and swap each app conversation context in 16-token chunks, switching up to 20 times faster.
MeRino designs sub-100M language models that run 4.9 times faster than OPT-350M
Researchers design 52M to 64M parameter transformers by maximising entropy under a compute budget, matching OPT-350M accuracy on an NVIDIA Jetson Nano.