LLM-jp-3.1 1.8B Instruct4
LLM-jp-3.1 1.8B Instruct4 starts from the corresponding LLM-jp-3 checkpoint. The family adds 400 billion mid-training tokens, with instruction-response data mixed into the pretraining corpus.
Research release
Instruct4 adds two stages of DPO
After supervised fine-tuning, the project generated multiple answers and scored them with Qwen2.5-32B-Instruct. It used those preferences for DPO in two stages. The second stage also adds the Japanese safety preference dataset ac-self-inst.LLM-jp-3.1 1.8B Instruct4 official model card
Compare alignment at the same size
Practical guidance. Check short Japanese prompts and format constraints before selecting this compact aligned release. Its larger relatives do not establish its own task accuracy.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3.1-1.8b-instruct4 |
| Context window | 4,096 tokens Maximum sequence length stated in the model card. LLM-jp-3.1 1.8B Instruct4 official model card |
| Architecture | 24 layers; hidden size 2,048; 16 attention heads.LLM-jp-3.1 1.8B Instruct4 official model card |
| Size | 1,867,216,896 parameters.LLM-jp-3.1 1.8B Instruct4 official model card |
| License | Apache 2.0.LLM-jp-3.1 1.8B Instruct4 official model card |
| Pretraining total | 2.5 trillion tokens, including the 400-billion-token mid-training stage.LLM-jp-3.1 1.8B Instruct4 official model card |
Related models
- LLM-jp-3 1.8B Instruct3
LLM-jp / NII LLMC · LLM-jp-3
LLM-jp-3 1.8B Instruct3 is a preference-aligned instruction release with 4,096-token context and Apache 2.0 weights.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3.1 1.8B Base
LLM-jp / NII LLMC · LLM-jp-3.1
LLM-jp-3.1 1.8B Base adds instruction pretraining to LLM-jp-3.
Launch date not confirmedResearch releaseDownloadable