ERNIE 4.5 21B A3B
This sparse language model keeps a larger expert pool while using about 3B active parameters for each token. This entry covers the post-trained checkpoint.
Model design and access
The post-trained release supports general language understanding and generation in English and Chinese. Its 131,072-token window is the model limit. PT denotes the PyTorch-format distribution, not a different generation of model.ERNIE-4.5-21B-A3B-PT official model card
Practical use
Practical guidance. Plan memory from the full model and the expected cache, then measure throughput with realistic concurrency. The 3B active label does not mean the checkpoint fits wherever a dense 3B model fits. Include code and long-answer cases in the comparison.ERNIE-4.5-21B-A3B-PT official model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | ERNIE-4.5-21B-A3B-PT |
| Context window | 131,072 tokens Official checkpoint context window. Input and generation share this limit. ERNIE-4.5-21B-A3B-PT official model card |
| Parameters | 21B total parameters; 3B active per tokenERNIE-4.5-21B-A3B-PT official model card |
| Architecture | 28 layers; 64 routed text experts with six selected and two shared expertsERNIE-4.5-21B-A3B-PT official model card |
| License | Apache 2.0ERNIE-4.5-21B-A3B-PT official model card |
Related models
- ERNIE 4.5 21B A3B Base
Baidu ยท ERNIE 4.5
Sparse language base model with a 21-billion-parameter expert pool and about three billion active per token.
Launch date not confirmedDownloadable