On-device AI deployment has taken a major step forward. Taobao's MNN team has officially announced the open-source release of MNN-Pocket-0.3B, a lightweight large language model designed for mobile applications. With about 300 million parameters, the model is tailored to the resource budgets and task requirements of mobile devices such as phones, delivering cloud-like high-speed responses and strong overall task capabilities while using extremely low memory and energy.
Extreme lightweight design: size nearly halved, peak memory kept under 300MB
To make large models truly run "imperceptibly" on mobile devices, Taobao's MNN team refined the idea of a "native large model for mobile applications" into four core design targets: decode throughput above 200 token/s, peak memory below 300MB, extremely low energy consumption per inference, and powerful instruction-following and multi-task extensibility.
In terms of actual performance, the model delivered impressive results:
Amazing offline speed: In real-device tests on an Apple iPhone 18 Pro, running fully offline through the MNN inference engine, average decode speed reached about 350 token/s. In tests on the iPad Pro M5 Metal backend, Prefill throughput reached 8,721 token/s, and Decode throughput reached 466 token/s, about 1.63x that of Qwen3-0.6B under the same conditions.
Extremely small size and memory usage: The INT4 MNN deployment file is only 173MB. Under the iPad Pro M5 test configuration, its peak memory increment is only 228MB. By comparison, Qwen3-0.6B in the same test has a deployment size of 340MB and a peak memory increment of 427MB. MNN-Pocket-0.3B reduces deployment size by about 49% and memory increment by about 47%, leaving ample space for the phone system's photos, videos, and other business modules.
Ultra-low power consumption: The team tested the power consumption of the 4-bit model on a OnePlus 13 (Snapdragon 8 Elite). Under baseline load, the cumulative incremental energy for executing 100 equivalent-load tasks accounted for only about 0.3% of the phone's total battery capacity, completely eliminating users' concerns about on-device AI power drain and heat.
Streamlined and efficient vocabulary architecture: To better adapt to the Chinese internet environment, the team reduced the vocabulary from about 151K to 64K, a 57% reduction, effectively lowering the parameter overhead of the embedding and output layers.
Comprehensive capability leapfrog challenge: topping multiple evaluations among models of the same scale
In actual task performance and academic benchmark evaluations, MNN-Pocket-0.3B demonstrated explosive capability far beyond its parameter size.
Evaluation data shows that the model achieved an English IFEval score of 73.20 and a self-built Chinese instruction-following IFEval-zh score of 74.59, ranking among the top in comparable models. Across 11 comprehensive evaluations covering Chinese and English knowledge, reasoning, math, code, and instruction following, its average score reached 50.00, comprehensively surpassing larger comparable models such as MiniCPM4-0.5B and Qwen3-0.6B. Among them, it scored 51.20 on Chinese math MGSM-zh and 53.66 on code generation HumanEval.
Full-version, fully open-source matrix: empowering researchers and the developer ecosystem
To enable rapid conversion of technical results into real business productivity, the MNN team has open-sourced multiple versions across the full pipeline from pretraining and post-training to on-device deployment, meeting deep adaptation needs across different scenarios:
MNN-Pocket-0.3B-Base: designed for continued pretraining, domain adaptation, and basic capability research.
MNN-Pocket-0.3B-SFT: intended for post-training research, reinforcement learning, distillation, and ablation experiments.
MNN-Pocket-0.3B: suitable for general instruction invocation and deep adaptation to various business tasks.
MNN-Pocket-0.3B-MNN: an on-device dedicated deployment version using W4A16 QAT. Combined with the MNN inference engine's low-bit operators and hardware backend optimizations, decode throughput can max out at 466 token/s on specific devices.
MNN-Pocket-0.3B-DFlash2: designed for the DFlash2 speculative decoding draft model. With block-level generation and verification, decode throughput under specific tasks can reach 2.06x that of regular autoregressive decoding.
MNN-Pocket-0.3B-Dec: accurately completes candidate selection tasks through a decision head.
