Alibaba has officially launched the Qwen-UI-Agent, a GUI agent foundation model centered on the real world. This model comprehensively covers mobile, desktop, web, and deep search environments, aiming to break the limitations of traditional simulation testing, enabling the model to seamlessly operate on complex real devices.
Comprehensive Performance Exceeding Industry Leaders
In various authoritative tests and real-world environments, Qwen-UI-Agent has demonstrated strong capabilities:
- Mobile Performance: Achieved an impressive score of 82.1% in the MobileWorld test, surpassing multiple major international flagship models; in its self-built benchmark test MobileWorld-Real with over 100 real devices, the success rate reached as high as 92.2%, and it was nearly perfect in Android Daily.
- Desktop Performance: Scored 79.5% in OSWorld-Verified, and significantly reduced the number of execution steps compared to the baseline in multiple tasks.
- Web and General Capabilities: Ranked first among all comparison models in the WebArena web test, and its general and Agentic capabilities fully surpassed the training base model, capable of handling long-tail demands effortlessly.
Moving from Simulation to Real Devices
To bridge the final gap between "simulation and reality," Qwen-UI-Agent has built a real mobile environment covering more than 100 real phones and 150+ applications for task building, trajectory collection, and model training. It also introduced the MobileWorld-Real real-device benchmark with over 400 tasks, allowing the development team to make precise selections based on real-device success rates.
Efficient Instructions and Security Control
In terms of functional features, this model not only supports conventional GUI interface operations but can also directly execute command-line operations. It outputs batch actions in a single decision, significantly shortening the execution trajectory and improving efficiency. At the same time, it has a complete security judgment mechanism throughout the process: when facing illegal or high-risk requests, it will directly refuse and terminate the task; when involving sensitive scenarios such as payments, data deletion, and privacy authorization, it will stop at key steps and wait for user confirmation before continuing.
Additionally, this model supports online reinforcement learning training on trajectories longer than 100 steps, combined with adaptive curriculum learning, continuously tackling challenging long-term tasks.
Project Homepage: https://tongyi-mai.github.io/Qwen-UI-Agent/
