Baidu has officially released the ERNIE Bot 4.5 series model and has made it fully open source. Users can immediately experience this latest open-source technology through ERNIE Bot (https://yiyan.baidu.com).
This series of models includes various parameter configurations, such as a mixture of experts (MoE) model with 47B and 3B activation parameters, and a dense model with 0.3B parameters, totaling ten different models.
In terms of training and inference, the ERNIE Bot 4.5 series models use the PaddlePaddle deep learning framework, achieving a model FLOPs utilization rate of 47%. Experimental results show that this series of models performs well in multiple text and multimodal benchmark tests, especially achieving SOTA (state-of-the-art) levels in tasks such as instruction following, world knowledge memory, visual understanding, and multimodal reasoning.
The weights of the ERNIE Bot 4.5 series models are open-sourced under the Apache 2.0 license, allowing for academic research and industrial applications. In addition, PaddlePaddle provides open-source industrial-grade development kits that are compatible with various hardware, reducing the barriers to post-training and deployment.
To facilitate developers, the ERNIE Bot 4.5 series also offers a one-stop usage guide, including how to quickly master model fine-tuning and alignment. Developers can easily implement model training and deployment through tools such as ERNIEKit and FastDeploy. FastDeploy supports multiple hardware platforms, providing a high-performance and low-barrier inference deployment experience.
This open source initiative not only includes pre-trained weights and inference code for the models, but also provides download and deployment services on multiple platforms. Users can find and use these models on well-known platforms such as the PaddlePaddle Starry Community and HuggingFace. Notably, Baidu has also launched an API service for open models, which users can call and use on the Baidu Intelligent Cloud Qianfan Large Model Platform. This means that both researchers and developers can more conveniently access the latest AI technologies.
The open source of the ERNIE Bot 4.5 series model includes pre-trained weights and inference code, and all projects have been uploaded to:
Hugging Face (https://huggingface.co/baidu),
GitHub (https://github.com/PaddlePaddle/ERNIE)
PaddlePaddle Starry Community (https://aistudio.baidu.com/overview).
If you need to understand the technical details, you can download the technical report (https://yiyan.baidu.com/blog/publication) for reference.
Key Points:
🌟 The ERNIE Bot 4.5 series model was officially open-sourced on June 30th, offering 10 different versions of the model.
📈 The model adopts an innovative multimodal heterogeneous architecture, enhancing multimodal understanding capabilities.
🛠️ Developers can quickly perform model training and deployment using tools such as ERNIEKit and FastDeploy.
