LLM-jp-4 33B Base
LLM-jp-4 33B Base is the pretrained and mid-trained checkpoint for this size. It is the starting point for adaptation before instruction alignment.
Research release
Pretraining before instruction alignment
The training pipeline used 11.7 trillion tokens across pretraining and mid-training. The project releases much of the corpus, but excludes some material because of license restrictions. Use the supplied tokenizer; the format resembles Harmony, but its token IDs differ.LLM-jp-4 33B Base official model card
Choose the checkpoint for adaptation
Practical guidance. Use this dense checkpoint to compare full-model computation with the 32B-A3B expert model. Similar total parameter counts do not imply similar memory traffic or token latency.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-4-33b-base |
| Context window | 65,536 tokens Maximum sequence length stated in the model card. LLM-jp-4 33B Base official model card |
| Architecture | Dense; 64 layers, hidden size 5,120.LLM-jp-4 33B Base official model card |
| Parameters | 33,219,548,160 total parameters.LLM-jp-4 33B Base official model card |
| License | Apache 2.0; downloadable research weights.LLM-jp-4 33B Base official model card |
Related models
- LLM-jp-4.1 33B Thinking
LLM-jp / NII LLMC · LLM-jp-4.1
LLM-jp-4.1 33B Thinking is a Japanese-English reasoning release with new SFT and preference training.
Launch date Research releaseDownloadable - LLM-jp-4 33B Thinking
LLM-jp / NII LLMC · LLM-jp-4
LLM-jp-4 33B Thinking provides Japanese and English text reasoning weights with a 65,536-token context.
Launch date not confirmedResearch releaseDownloadable