> [!NOTE] > Note: "**-Paddle**" models use PaddlePaddle weights, while "**-PT**" models use Transformer-style PyTorch weights. The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations: 1. **Multimodal Heterogeneous MoE Pre-Training:** Our models are jointly trained on both textual and visual modalities to better capture the nuances of multimodal information and improve performance on tasks involving text understanding and generation, image understanding, and cross-modal reasoning. To achieve this without one modality hindering the learning of another, we designed a *heterogeneous MoE structure*, incorporated *modality-isolated routing*, and employed *router orthogonal loss* and *multimodal token-balanced loss*. These architectural choices ensure that both modalities are effectively represented, allowing for mutual reinforcement during training.
Over the past three months, we have continued to scale the **thinking capability** of ERNIE-4.5-21B-A3B, improving both the **quality and depth** of reasoning, thereby advancing the competitiveness of ERNIE **lightweight models** in complex reasoning tasks. We are pleased to introduce **ERNIE-4.5-21B-A3B-Thinking**, featuring the following key enhancements: * **Significantly improved performance** on reasoning tasks, including logical reasoning, mathematics, science, coding, text generation, and academic benchmarks that typically require human expertise. * **Efficient tool usage** capabilities. * **Enhanced 128K long-context understanding** capabilities. > [!NOTE] > Note: This version has an increased thinking length. We strongly recommend its use in highly complex reasoning tasks.
- [2026/07/21] 🤝 Thanks to the ms-swift community for their support, our model now supports training with ms-swift. - [2026/07/03] 🤝 Thanks to the Baidu Cloud team for their support. Our model is now available on Baidu Cloud. - [2026/06/28] 🤝 Thanks to the vLLM community and Tianyu Guo for their support, our model now supports vLLM inference. - [2026/06/24] 🤝 Thanks to AK for creating a demo for us. It is now available at Hugging Face Spaces. - [2026/06/23] 📄 Our paper is now available on arXiv. - [2026/06/23] 🤝 Thanks to the ModelScope community for their support. Our model is now available at ModelScope. - [2026/06/22] 🚀 We present Unlimited-OCR, aiming to push Deepseek-OCR one step further. Inference using Huggingface transformers on NVIDIA GPUs. Requirements tested on python 3.12.3 + CUDA12.9： **Recipe:** https://recipes.vllm.ai/baidu/Unlimited-OCR
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
