Recently, the XLANG Lab at the University of Hong Kong, together with Moonshot AI, Stanford University, and other institutions, has open-sourced a new framework called OpenCUA. The goal of this project is to help developers easily build and expand computer usage agents (CUAs), allowing everyone to have their own personal computer assistant.
The release of the OpenCUA framework marks another leap forward in computer usage agents. It not only provides a seamless annotation infrastructure for capturing human demonstrations on computers, but also integrates a large-scale computer usage task dataset called AgentNet. This dataset covers over 200 applications and websites, as well as three major operating systems, offering developers rich data support.
In addition, OpenCUA features an extensible workflow that can convert demonstrations into "state-action" pairs, promoting long-chain reasoning capabilities. This means developers can easily build personalized intelligent assistants according to their needs, helping users complete tasks more efficiently.
The project leader, Professor Yu Tao, stated that OpenCUA aims to enable "everyone to create their own personalized computer agent" by opening up complete data, tools, and models. The framework performs exceptionally well on multiple key benchmarks, even surpassing the current state-of-the-art GPT-4o, becoming the most powerful open-source CUA solution.
With the release of OpenCUA, the application of computer agents will become more widespread and convenient. Developers can not only customize according to their own needs, but also use this framework to enhance the user's computer experience. Notably, this project has received participation from several renowned universities and companies, demonstrating collaboration and mutual benefit in the field of agent research within the tech industry.
OpenCUA's open source provides new possibilities for the development of future intelligent assistants. Let's look forward to how technology will further improve our work and life efficiency.
Project URL: https://opencua.xlang.ai/
Paper URL: https://arxiv.org/pdf/2508.09123