LLM-jp-4 32B-A3B Base
LLM-jp-4 32B-A3B Base is the pretrained and mid-trained checkpoint for this size. It is the starting point for adaptation before instruction alignment.
Research release
Pretraining before instruction alignment
The training pipeline used 11.7 trillion tokens across pretraining and mid-training. The project releases much of the corpus, but excludes some material because of license restrictions. Use the supplied tokenizer; the format resembles Harmony, but its token IDs differ.LLM-jp-4 32B-A3B Base official model card
Choose the checkpoint for adaptation
Practical guidance. Active parameters describe computation, not the full weight storage requirement. Plan memory for the complete model and its serving state.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-4-32b-a3b-base |
| Context window | 65,536 tokens Maximum sequence length stated in the model card. LLM-jp-4 32B-A3B Base official model card |
| Architecture | Mixture of experts; 128 routed experts, eight active per token.LLM-jp-4 32B-A3B Base official model card |
| Parameters | 32,139,028,992 total; 3,827,476,992 active parameters.LLM-jp-4 32B-A3B Base official model card |
| License | Apache 2.0; downloadable research weights.LLM-jp-4 32B-A3B Base official model card |
Related models
- LLM-jp-4.1 32B-A3B Thinking
LLM-jp / NII LLMC · LLM-jp-4.1
LLM-jp-4.1 32B-A3B Thinking is a Japanese-English reasoning release with new SFT and preference training.
Launch date Research releaseDownloadable - LLM-jp-4 32B-A3B Thinking
LLM-jp / NII LLMC · LLM-jp-4
LLM-jp-4 32B-A3B Thinking provides Japanese and English text reasoning weights with a 65,536-token context.
Launch date not confirmedResearch releaseDownloadable