ERNIE 4.5 21B A3B Base
This sparse language model keeps a larger expert pool while using about 3B active parameters for each token. This entry covers the base checkpoint.
Model design and access
The base release retains the pretrained text model before chat post-training. The 21B total size governs expert-weight storage. Its active count describes the work selected for each token.ERNIE-4.5-21B-A3B-Base-PT official model card
Practical use
Practical guidance. Choose this base when you have a specific training objective and enough evaluation data to measure it. It offers a smaller adaptation target than the 300B version while retaining expert routing. Check whether tuning alters routing balance or general language behavior.ERNIE-4.5-21B-A3B-Base-PT official model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | ERNIE-4.5-21B-A3B-Base-PT |
| Context window | 131,072 tokens Official checkpoint context window. Input and generation share this limit. ERNIE-4.5-21B-A3B-Base-PT official model card |
| Parameters | 21B total parameters; 3B active per tokenERNIE-4.5-21B-A3B-Base-PT official model card |
| Architecture | 28 layers; 64 routed text experts with six selected and two shared expertsERNIE-4.5-21B-A3B-Base-PT official model card |
| License | Apache 2.0ERNIE-4.5-21B-A3B-Base-PT official model card |
Related models
- ERNIE 4.5 21B A3B
Baidu ยท ERNIE 4.5
Post-trained sparse language model with 21 billion total parameters and about three billion active per token.
Launch date not confirmedDownloadable