FBLayout fine-tunes transformers on phone GPUs 2.2 to 5.7 times faster

Researchers at the University of Macau, the University of Georgia and the Hong Kong University of Science and Technology published FBLayout on July 7, 2026, a system that fine-tunes transformer models on a phone’s own GPU, accepted at ACM MobiSys 2026. On a OnePlus Ace 5 Pro with a Snapdragon 8 Elite and its Adreno 830 GPU, the authors measure training on Llama 3.2 1B, Qwen2.5 1.5B and Gemma 2 2B at 3.9 to 4.1 times the speed of MNN, 4.3 to 4.9 times TFLite and 5.4 to 5.7 times TVM. Over the same three baselines they count 3.5 times fewer global memory reads and 4.2 times fewer cache misses.

The authors time inference on the Snapdragon 8 Elite at roughly 6.5 times faster on the GPU than on the CPU, while fine-tuning models of 0.5B to 2B parameters on the GPU comes out level with the CPU or up to 43 percent worse. Their explanation is memory layout, meaning the order in which a tensor’s numbers physically sit in memory. Mobile GPUs hold tensors in a 2D texture grid whose cache rewards reading neighbouring cells, and training reuses forward-pass activations and weights in the backward pass, which sums along a different dimension and so walks the same tensor in strides instead of in rows. Each stride that misses the cache becomes a fetch from DRAM at under 100 GB/s of mobile memory bandwidth, and the GPU stalls waiting for it.

Existing frameworks take one of two ways out, and the paper prices both. MNN reuses the forward layout in the backward pass, which the authors put at up to a 5.3 times slowdown on key operations and at under 21 percent arithmetic-unit utilisation in a matrix-multiply benchmark. TFLite instead inserts explicit transpose and reshape operators, and the paper puts that route at 40.8 to 50.0 percent of total runtime across the three LLMs and at roughly a third of all memory allocations. FBLayout defines one tile-based layout, called R-Tile, that keeps both directions contiguous, then rewrites the remaining reshapes as coordinate arithmetic so no data moves, which they report covers over 86 percent of all layout transformations and over 95 percent of those in the LLMs.

The evaluation covers seven models, adding BERT-Large, ViT-Large, Whisper-Large and Stable Diffusion 1.5 to the three LLMs, with LoRA adapters in the attention and MLP blocks for everything except ViT. Beyond the Snapdragon 8 Elite phone the authors test a OnePlus Ace 10 Pro with a Snapdragon 8 Gen 1 and an Adreno 730 and a OnePlus Ace 5 Ultra with a Dimensity 9400+ and an Arm Mali Immortalis-G925, and report gains on both. Preparing the Llama 3.2 1B training graph takes 6.1 seconds in their measurement, against 723 seconds in MNN, 862 in TFLite and over 4,800 in TVM, and a Monsoon power monitor puts energy use over five training rounds at 3.5 to 6.3 times below the baselines.

FBLayout is built on top of Alibaba’s MNN as about 9.2K lines of C++ and OpenCL, extending it with automatic differentiation so the training graphs exist in the first place. The paper names no code release and points to no repository. It appeared at MobiSys 2026 in Cambridge in June 2026 and is on arXiv under a CC BY-NC-ND 4.0 license.