LLM-jp-3 8×1.8B Instruct2
LLM-jp-3 8×1.8B Instruct2 has 9,266,362,368 total parameters and 2,924,279,808 active parameters. Its 24-layer architecture retains a 4,096-token context.
Research release
The supervised stage remains available
The downloadable Instruct2 checkpoint provides the supervised alignment stage. Its family card lists ichikara, AnswerCarefully, Magpie, English instruction data and code examples.LLM-jp-3 8×1.8B Instruct2 official model cardLLM-jp explains Instruct2 and Instruct3
Use the SFT checkpoint as a comparison
Practical guidance. Compare 8×1.8B Instruct2 and Instruct3 with identical generation settings. The pair is useful for studying whether preference tuning changes the responses an application needs.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3-8x1.8b-instruct2 |
| Context window | 4,096 tokens Maximum sequence length stated in the model card. LLM-jp-3 8×1.8B Instruct2 official model card |
| Architecture | 24 layers; hidden size 2048; eight experts, two active.LLM-jp-3 8×1.8B Instruct2 official model card |
| License | Apache 2.0.LLM-jp-3 8×1.8B Instruct2 official model card |
| Pretraining | 2.1 trillion seen tokens reported in the model card.LLM-jp-3 8×1.8B Instruct2 official model card |
| Ancestry | Drop-Upcycling from LLM-jp-3 1.8B; expert reinitialization ratio 0.5.LLM-jp-3 MoE training and release |
Related models
- LLM-jp-3 8×1.8B Base
LLM-jp / NII LLMC · LLM-jp-3 MoE
LLM-jp-3 8×1.8B Base is an eight-expert language model with two experts selected per token.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3 8×1.8B Instruct3
LLM-jp / NII LLMC · LLM-jp-3 MoE
LLM-jp-3 8×1.8B Instruct3 is an eight-expert language model with two experts selected per token.
Launch date not confirmedResearch releaseDownloadable