https://llmobile.kavents.com/tags/qwen/A concise news ticker covering AI on mobile devices, local models, apps and hardware.en-GBhttps://llmobile.kavents.com/ticker/rikkahub-agent-local-llm/https://llmobile.kavents.com/ticker/rikkahub-agent-local-llm/Sat, 08 Aug 2026 00:00:00 +0200Adam Conway at XDA has tested RikkaHub Agent, a fork by ExTV of the open-source Android LLM client RikkaHub, which adds an agent layer with more than 80 device tools. The phone runs the agent loop and executes the tools; the model itself runs elsewhere on the local network over an OpenAI-compatible endpoint. The test device was an Oppo Find N5 with a Snapdragon 8 Elite and 16 GB of RAM. Inference ran on a Radeon RX 7900 XTX with Qwen 3.6 27B at Q4_K_M, roughly 16 GB, at 40 to 50 tokens per second. A second configuration used Qwen 3.6 35B on a Lenovo ThinkStation PGX with Nvidia’s GB10 Grace Blackwell. Screenshot: XDA. The tool set covers tapping, swiping, scrolling and typing on screen, reading notifications, sending SMS, file management, browser control, SSH, scheduled jobs, a JavaScript runtime and Telegram. All tools are off by default and enabled per assistant, actions with side effects require approval, and a deterministic blocklist rejects destructive commands. The default context is 7,000 input tokens, rising to 32,000 with every tool enabled. Asked whether whisper.cpp was installed, the agent determined that neither the command-line tool nor a model was present, requested permission, then installed the dependencies, cloned the repository and compiled it inside Termux. The build took seven minutes, and the agent chose the 75 MB small model on speed and accuracy grounds. It then transcribed a Telegram voice message, installing ffmpeg, converting the OGG/Opus file to 16 kHz mono WAV and returning a transcript in about two seconds. Running the model on the phone itself was slower. With Gemma 4 E2B, a 2.6 GB build accelerated by LiteRT-LM, Conway reports chat worked but agentic tasks were painful, and tool calls were emitted as plain text until an update fixed the behaviour. Frontier intelligence is never what I’d want from a phone agent, which is why a local LLM works perfectly. Source: https://www.xda-developers.com/turned-phone-ai-agent-powered-local-llm-installed-software/ Read the article: https://llmobile.kavents.com/ticker/rikkahub-agent-local-llm/AndroidAgentsQwenOpen sourceSpeechhttps://llmobile.kavents.com/ticker/small-vlms-video-compressors/https://llmobile.kavents.com/ticker/small-vlms-video-compressors/Thu, 09 Apr 2026 13:40:25 +0200Researchers at Meta AI and KAUST published Tempo on April 9, 2026, a way to hand a long video to a small vision-language model first and let a larger language model answer the question afterwards. On LVBench, a benchmark whose clips average 4101 seconds, the authors report their 6B system scoring 52.7 against 48.2 for VideoChat-Flash 7B, the strongest specialised baseline in their table, and against 30.8 for GPT-4o and 33.1 for Gemini 1.5 Pro. The small model does the reading and the large one does the reasoning. A Qwen3-VL-2B-Instruct compressor takes an eight-frame chunk of video together with the user’s question and writes out a short run of memory tokens that hold what the chunk contains, and a Qwen3 4B language model then reads the whole condensed sequence with timestamps attached and produces the answer. A linear projector joins the two, for 6B parameters in total. The compressor is the part that could in principle sit on a phone, since it never sees more than a few seconds of video at a time. How many tokens each chunk gets is decided during the same forward pass. The authors insert a yes or no question about whether the segment is relevant to the query, read the difference between the two logits as a score, and keep more memory tokens for chunks that score high, spending between 0.5 and 16 tokens per frame under a fixed overall budget of 4K or 8K visual tokens. On LVBench under the 4K budget the measured average comes to 2.9 tokens per frame. The tighter budget also scores better than the looser one on that benchmark, 52.7 against 52.3, and on the long subset of Video-MME, 57.8 against 57.0. Raising the cap to 2048 frames and a 12K budget brings LVBench to 53.7. Diagram: Fei et al., Figure 2 of the paper, licensed CC BY 4.0. On the other three benchmarks the authors report 67.8 on Video-MME under the 4K budget against 65.3 for VideoChat-Flash and 61.9 for the Qwen3-VL 2B model the compressor starts from, plus 75.6 on MLVU and 65.1 on LongVideoBench. All of these are accuracy scores run through lmms-eval, not speed or memory figures. Nothing in the paper was measured on a phone, a tablet or any other edge device. Training used 32 and then 64 Nvidia H100 GPUs, the paper reports no latency, throughput or memory footprint for the compressor on mobile hardware, and the on-device case rests on the size of the model rather than on a measurement. The authors state that all training datasets they used are publicly available, and they link code and a project page from the paper, with the repository carrying an Apache 2.0 license. They describe no weights release in the paper itself. Their stated next steps are to let the compressor decide for itself when it has gathered enough evidence to stop, which they say would bottleneck inference latency as things stand. Source: https://arxiv.org/abs/2604.08120 Read the article: https://llmobile.kavents.com/ticker/small-vlms-video-compressors/MetaQwenResearchBenchmarksDistillationhttps://llmobile.kavents.com/ticker/qwen-3-5/https://llmobile.kavents.com/ticker/qwen-3-5/Mon, 02 Mar 2026 10:00:00 +0100Alibaba’s Qwen team released Qwen3.5-0.8B and Qwen3.5-2B on March 2, 2026, together with 4B and 9B versions, according to the Qwen release notes. Both are dense vision-language models with a 262,144-token context and Apache 2.0 weights. The model card states that the released files work with Hugging Face Transformers, vLLM, SGLang and KTransformers, and that in light of the parameter scale the intended uses are prototyping, task-specific fine-tuning and other research or development purposes. The benchmark table in that model card puts Qwen3.5-2B at 55.3 on MMLU-Pro with reasoning switched off, where the same table gives 40.2 for last generation’s Qwen3-1.7B and 69.6 for the larger Qwen3-4B-2507, and Qwen3.5-0.8B at 29.7. With reasoning on, Alibaba reports 66.5 for the 2B and 42.3 for the 0.8B on the same test, and 51.6 against 11.9 on GPQA, a set of graduate-level science questions. The same table has Qwen3-1.7B ahead of both new sizes on IFEval, which checks whether a model obeys explicit formatting instructions, at 68.2 against 61.2 for the 2B and 52.1 for the 0.8B with reasoning off. Alibaba ran all of these figures itself. Both sizes ship a vision encoder in the same checkpoint rather than as a separate release, and Alibaba’s vision table gives Qwen3.5-2B 64.2 on MMMU, a college-level image question set, above the 61.4 it lists for the previous Qwen3-VL-2B, with 49 for Qwen3.5-0.8B. The two models stack 24 layers in which three linear-attention blocks, whose memory cost does not grow with the sequence, alternate with one full-attention block. Input embedding and output layer are tied, so a single 248,320-entry matrix does both jobs, and Alibaba puts language coverage at 201 languages and dialects. Alibaba published the weights on Hugging Face and ModelScope under Apache 2.0, and its MNN team put 4-bit exports of both sizes up the same day. The 0.8B build carries a 470 MB language weight file plus 63 MB for the vision encoder, and the 2B build 1.18 GB plus 196 MB, against 1.75 GB and 4.55 GB for the full-precision checkpoints. For other local runtimes the Qwen repository points readers to llama.cpp, which it says supports the series for text and vision, and to mlx-lm and mlx-vlm on Apple silicon, without Alibaba publishing GGUF or MLX builds of these sizes itself. Source: https://huggingface.co/Qwen/Qwen3.5-2B Read the article: https://llmobile.kavents.com/ticker/qwen-3-5/AlibabaQwenOpen weightsQuantisationBenchmarkshttps://llmobile.kavents.com/ticker/mobile-agent-v3-5/https://llmobile.kavents.com/ticker/mobile-agent-v3-5/Sun, 15 Feb 2026 02:52:00 +0100Alibaba’s Tongyi Lab published Mobile-Agent-v3.5 on February 15, 2026, describing GUI-Owl-1.5, a family of agent models that read a screenshot and emit the next tap, type or scroll. The family spans 2B, 4B, 8B, 32B and 235B-A22B parameters in instruct and reasoning variants, all built on Qwen3-VL. Every benchmark in the paper was run server-side against cloud sandboxes and emulators, and the authors give no on-device deployment, latency or hardware figures anywhere, so the small sizes are the only thing that points at a phone. The authors write that the smaller instruct variants, which skip the reasoning step, are fast enough to be deployed on edge devices and to work alongside a larger cloud model. Across platforms, the authors report their 32B-Instruct model reaching 56.5 on OSWorld-Verified for desktop tasks, 47.6 on OSWorld-MCP and 46.8 on Mobile-World, the last two of which also require calling external tools. On AndroidWorld, which scores an agent driving real Android apps in an emulator, their 8B reasoning model reaches 71.6, which the paper puts just behind UI-TARS-2 at 73.3. For browsers, the paper’s table has the 32B reasoning model at 48.4 on WebArena, 46.6 on VisualWebArena and 82.1 on WebVoyager. The small checkpoints carry a large part of that. The authors report 2B-Instruct at 43.5 on OSWorld-Verified and 67.9 on AndroidWorld, and 4B-Instruct at 48.2 and 69.8, which puts the 4B model within 2 points of their own 32B on Android tasks. On ScreenSpot-Pro, which tests whether a model can point at the right element in a high-resolution screenshot, they report 72.9 for 32B-Instruct and 80.3 when the model first crops a region and looks again. The paper’s prose and its tables disagree on which variant owns several headline numbers, and the figures above follow the tables. Charts: Alibaba Tongyi Lab, the authors' own benchmark figures. Six checkpoints are on Hugging Face under the MIT license, covering 2B, 4B and 8B instruct, 8B reasoning, and 32B in both variants. The 235B-A22B model named in the paper is not among them. The code is MIT as well, and the repository points at a hosted cloud-sandbox demo rather than a local runtime. Source: https://arxiv.org/abs/2602.16855 Read the article: https://llmobile.kavents.com/ticker/mobile-agent-v3-5/AlibabaAgentsQwenOpen weightsAndroidhttps://llmobile.kavents.com/ticker/intelligence-per-watt/https://llmobile.kavents.com/ticker/intelligence-per-watt/Tue, 11 Nov 2025 07:33:00 +0100Researchers at Stanford University and Together AI profiled more than 20 local models on 1M real queries and report that at least one model with 20B or fewer active parameters answered 88.7% of them correctly. The best single model in their set, GPT-OSS-120B, reached 71.4% on its own, averaged across the four query collections. Their 2025 efficiency figure comes from that model running on an Apple M4 Max. The paper proposes intelligence per watt, mean task accuracy divided by the mean power the accelerator draws while answering, so a pairing that gets more answers right on the same power budget scores higher. The best local pairing each year moved from 7.92×10⁻⁴ accuracy per watt for Mixtral 8x7B on an NVIDIA Quadro RTX 6000 in 2023, to 1.80×10⁻³ for Llama 3.1 8B on an RTX 6000 Ada in 2024, to 4.18×10⁻³ for GPT-OSS-120B on an Apple M4 Max in 2025, which the authors put at a 5.3 times gain over two years. Query coverage for that single best pairing rose from 23.2% to 71.3% across the same three years. A companion metric divides accuracy by the energy a query consumes instead, so slow generation counts against a device. Diagram: Saad-Falcon et al., Figure 1 of the paper, licensed CC BY 4.0. The hardware panel separates the accelerators the authors treat as local from the cloud parts. The 1M queries come from four collections. The team took 500K prompts from WildChat, a month of real ChatGPT traffic, 500K from NaturalReasoning, 12K from MMLU Pro and 26.5K from SuperGPQA, all single-turn. WildChat has no ground truth, so GPT-4o judged whether a local model’s answer matched one written by Qwen3-235B, while the other three collections supplied reference answers. Every run used a batch size of 1, one query at a time. The hardware the authors call local is desktop and workstation class rather than mobile. Their 2025 numbers ran on an Apple Mac Studio with an M4 Max, 128 GB of unified memory and 480 W of continuous power, and the 2023 and 2024 points used NVIDIA workstation cards. No phone, tablet or wearable was profiled. Running the same Qwen3 models, the M4 Max reached 1.40 times lower accuracy per watt than an NVIDIA B200 and 1.6 to 2.3 times lower accuracy per joule, a gap the authors read as headroom for local accelerator design. For the savings case the team simulated 80.2M queries over 24 hours split between four small models on M4 Max machines and Qwen3-235B in the cloud. Perfect routing to the smallest capable model cut energy by 80.4% against sending everything to the frontier model, and a router that picks correctly 80% of the time cut it by 64.3%, with the perfect case described as a theoretical maximum. The authors note that coverage falls in technical domains such as architecture and engineering, that GPT-OSS-120B answered 88.9% of WildChat queries against 64.9% of NaturalReasoning ones, and that they release their profiling harness so the measurements can be repeated on new hardware. Source: https://arxiv.org/abs/2511.07885 Read the article: https://llmobile.kavents.com/ticker/intelligence-per-watt/ResearchBenchmarksApple SiliconNvidiaQwenhttps://llmobile.kavents.com/ticker/shadownpu/https://llmobile.kavents.com/ticker/shadownpu/Fri, 22 Aug 2025 09:41:00 +0200Researchers from Peking University and BUPT report up to 4.5 times faster end-to-end inference on a Xiaomi 14 with a Snapdragon 8 Gen 3, and 3.5 times faster on average for the attention kernel alone, with a peak of 6.9 times. The baseline is the arrangement most on-device frameworks use today, float32 full attention running on the CPU or GPU while the rest of the model sits on the NPU. The authors measured Qwen2-0.5B, Qwen2-1.5B, PhoneLM-0.5B and PhoneLM-1.5B on summarisation and two mobile agent tasks, and gave their own system only one of the phone’s middle CPU cores. Against the native attention of llm.npu, which uses four CPU cores or the GPU, they report up to 3.0 times lower latency from that one core. The paper is due at MobiSys 2026. The starting problem is that attention keeps falling back off the NPU. Attention multiplies three activation tensors together, and mobile NPUs compile a static graph that fixes one quantisation scale factor for a whole tensor, so the authors measure an average 18 percentage point accuracy drop when the operator runs in INT8 on the NPU. Their trade is to stop asking the NPU for the answer and ask it only for the ranking. The NPU computes the query-key scores in INT8 purely to find which tokens matter, which the authors report recovers over 99 percent of the positions a float run would pick, since picking the largest values needs relative magnitudes rather than exact ones. The CPU then computes real attention in float over roughly 20 percent of the tokens and throws the other 80 percent away, at an average accuracy cost of 0.4 percentage points against the float baseline. Each attention head gets its own sparsity ratio, fixed by an offline profiling run the paper puts at about five minutes on a single A100. On a Redmi K60 Champion Edition with a Snapdragon 8 Gen 2 the authors read battery voltage and current from the kernel and report a single attention kernel at 1024 tokens dropping from 8.59 J to 1.12 J for PhoneLM-1.5B, a factor of 7.66. Speedups on that older phone are smaller, 2 times, 1.25 times and 1.22 times across the three datasets, which the authors attribute to its Hexagon V73 NPU against the V75 in the Xiaomi 14. The paper reports no memory footprint figures. It does report that end-to-end inference time on the Xiaomi 14 stayed between 14.5 and 15.5 seconds while the phone was also taking photos, playing video, playing music or running a shopping app, since the LLM work is not competing for the cores those apps use. Diagram: Yin et al. The authors state a prototype of more than 10,000 lines of C++ and Python, built on Qualcomm’s QNN and Hexagon SDK for the NPU side and integrated into llm.npu, which they call the only fully open-source framework running on NPUs. The paper names no repository and announces no artefact release, and the title calls the system ShadowNPU while the text throughout calls the module shadowAttn. The arXiv listing carries a CC BY 4.0 license. For a hands-on point of comparison on the same silicon family, a user test of llama.cpp’s Hexagon NPU backend on a Snapdragon 8 Gen 3 phone put generation at about the same speed as the chip’s CPU, without the heat. Source: https://arxiv.org/abs/2508.16703 Read the article: https://llmobile.kavents.com/ticker/shadownpu/QualcommNPUResearchQwenAndroidhttps://llmobile.kavents.com/ticker/qwen-3/https://llmobile.kavents.com/ticker/qwen-3/Wed, 14 May 2025 15:41:00 +0200Alibaba’s Qwen Team published the Qwen3 Technical Report on May 14, 2025, covering six dense models from 0.6B to 32B parameters and two mixture-of-experts models, where only part of the model is active per token. The two smallest dense sizes, Qwen3-0.6B and Qwen3-1.7B, carry the same mode switch as the rest of the family, so one set of weights either works through a problem step by step before answering or replies straight away. Alibaba reports Qwen3-1.7B scoring 93.4 on MATH-500, a 500-problem maths set, with reasoning on, against 73.0 with it off. Earlier Qwen releases needed two separate models for that, a chat model and a dedicated reasoning model such as QwQ-32B. In Qwen3 the user picks the behaviour with a flag in the prompt or the system message, /think or /no_think, and the report states that reasoning is the default and that across a multi-turn conversation the model follows the last flag it saw. The same section describes a reasoning budget, where the model’s working is cut off once it passes a token limit the user set and it answers from what it has so far, which the authors say they never trained for and which emerged from merging the two modes. The benchmark tables give both small sizes in both modes. For Qwen3-0.6B, Alibaba reports 55.6 on MMLU-Redux, a corrected version of the MMLU knowledge test, with reasoning on against 44.6 with it off, and 77.6 against 55.2 on MATH-500. For Qwen3-1.7B the same pair of tables gives 73.9 and 64.4 on MMLU-Redux, 48.3 against 13.4 on the AIME 2024 competition maths set, and 33.2 against 11.6 on LiveCodeBench v5. Against the reasoning baselines in the report, Qwen3-0.6B is ahead of DeepSeek-R1-Distill-Qwen-1.5B on MMLU-Redux, 55.6 to 45.4, and behind it on MATH-500, 77.6 to 83.9. Alibaba built the small sizes by copying behaviour from the large ones rather than by running the full reinforcement learning pipeline on each. The report describes two stages of distillation from Qwen3-32B and Qwen3-235B-A22B, first on stored teacher outputs generated in both modes, then by matching the small model’s output probabilities against the teacher’s while the small model generates, and it credits the first stage with teaching the smaller models to switch modes at all. Pre-training ran on about 36 trillion tokens covering 119 languages and dialects, up from 29 in Qwen2.5. Context length is where the two small sizes sit apart from the family. Table 1 of the report puts Qwen3-0.6B and Qwen3-1.7B at 32K tokens against 128K for Qwen3-8B and every larger model, and both tie the input embedding to the output layer so one weight matrix does both jobs. All eight models are published under the Apache 2.0 license on Hugging Face, ModelScope and Kaggle. The Qwen3 blog post of April 29, 2025 names Ollama, LM Studio, MLX, llama.cpp and KTransformers as the tools it recommends for running them locally. Source: https://arxiv.org/abs/2505.09388 Read the article: https://llmobile.kavents.com/ticker/qwen-3/AlibabaQwenOpen weightsResearchBenchmarkshttps://llmobile.kavents.com/ticker/alibaba-mnn/https://llmobile.kavents.com/ticker/alibaba-mnn/Thu, 23 Jan 2025 13:30:00 +0100Alibaba published the first public build of its MNN-LLM Android app on January 23, 2025, a client that downloads open models and runs them entirely on the phone, covering text-to-text, image-to-text, audio-to-text and text-to-image generation through diffusion models. It sits on MNN, the inference and training framework Alibaba has kept open source since 2019 and uses in more than 30 of its own apps, among them Taobao, Youku and DingTalk. Alibaba reports the runtime prefilling 8.6 times faster than llama.cpp on an Android CPU, with decoding 2.3 times faster. Models reach the device through llmexport, which converts a PyTorch checkpoint to ONNX and then to MNN’s own format, splitting the weights into a separate file and quantising them to 4-bit or 8-bit by default, with block size configurable and 5-bit and 6-bit available through the MNNConvert tool. The same README lists Qwen, Gemma, Llama including TinyLlama and MobileLLM, Baichuan, Yi, DeepSeek, InternLM, Phi, ReaderLM and SmolLM as supported families. LoRA adapters can be merged into the base model at export or kept as a separate file that the runtime loads and swaps while running. The MNN-LLM paper, presented at the ACM Multimedia Asia 2024 workshops, describes a DRAM-Flash split that leaves the embedding table, roughly 15% of the weights, in bfloat16 on flash storage and reads only the single token needed for each decode step. The authors put the saving for Qwen2 7B at about 2.18 GB of DRAM. Layers and the LM head stay in memory at int4 or int8 with asymmetric quantisation, computed as W4A8 or W8A8 on CPU and W4A16 or W8A16 on GPU, while the KV cache quantises keys to int4 or int8 and values to fp8 before spilling to flash with prefetching. The team measured the speed comparisons on a Xiaomi 14 against llama.cpp, MLC-LLM and fastllm, using Qwen2 1.5B, Qwen2 7B and Llama 3 8B. Android builds run on the CPU with ARMv8.2 fp16 instructions and add an OpenCL path for the GPU, and the same runtime compiles for iOS with Metal, for macOS, Linux and Windows, and for browsers through WebAssembly. Alibaba distributes the app as a downloadable APK and warns that the first version was tested only on a OnePlus 13 and a Xiaomi 14 Ultra, so low-spec phones may run slowly or not at all. MNN is published under the Apache 2.0 license. Diagram: Alibaba. The architecture overview from the MNN repository. Source: https://github.com/alibaba/MNN Read the article: https://llmobile.kavents.com/ticker/alibaba-mnn/AlibabaAndroidQuantisationQwenOpen sourcehttps://llmobile.kavents.com/ticker/qwen2/https://llmobile.kavents.com/ticker/qwen2/Mon, 15 Jul 2024 14:35:00 +0200Alibaba’s Qwen Team published the Qwen2 Technical Report on July 15, 2024, covering five models from 0.5B to 72B parameters. The report states that the two smallest, Qwen2-0.5B and Qwen2-1.5B, are designed for easy deployment on portable devices such as smartphones, earphones and smart glasses, while the larger sizes target GPUs. Qwen2-1.5B scores 56.5 on MMLU, a multiple-choice knowledge test, against 52.7 for Microsoft’s Phi-2, which the same table credits with about twice the non-embedding parameters. Both small sizes tie the input embedding to the output layer, so a single weight matrix does both jobs and the shared vocabulary of 151,646 tokens is stored once rather than twice. The report puts non-embedding parameters at 0.3B for Qwen2-0.5B and 1.2B for Qwen2-1.5B. All five models use grouped query attention, where several query heads share one set of key and value heads, with 14 query heads over 2 key-value heads at 0.5B and 12 over 2 at 1.5B, which the authors say lowers the key-value cache size per token against Qwen1.5 and with it the memory footprint during long-context inference. Alibaba trained Qwen2-0.5B on a 12 trillion token dataset and the other dense models on a 7 trillion token one, reporting that the larger corpus did not pay off at the bigger sizes. For the instruction-tuned versions the report gives Qwen2-1.5B-Instruct 52.4 on MMLU, 61.6 on the GSM8K grade-school maths set and 29.0 on IFEval, which checks whether a model obeys explicit formatting instructions, against 37.9, 40.1 and 20.0 for Qwen2-0.5B-Instruct. The authors attribute the gain over the similarly sized Qwen1.5 models mainly to that data scaling. Context length differs across the family. The authors tested retrieval from long documents with Needle in a Haystack and report that the two small models handle 32K tokens, Qwen2-57B-A14B-Instruct 64K, and the 7B and 72B instruction-tuned models 128K with the YARN extension applied. The pre-training data covers approximately 30 languages, among them English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai and Vietnamese. Alibaba put the weights on Hugging Face and ModelScope. The Qwen2 blog post of June 7, 2024 states that Qwen2-0.5B, 1.5B, 7B and 57B-A14B carry the Apache 2.0 license while Qwen2-72B and its instruction-tuned variant stay on Alibaba’s own Qianwen License, and it names MLX, llama.cpp, Ollama and LM Studio among the projects that run the models locally. Source: https://arxiv.org/abs/2407.10671 Read the article: https://llmobile.kavents.com/ticker/qwen2/AlibabaQwenOpen weightsResearchWearables