LLM-jp-3 8×13B Instruct2
LLM-jp-3 8×13B Instruct2 has 73,162,828,800 total parameters and 22,200,806,400 active parameters. Its 40-layer architecture retains a 4,096-token context.
Research release
The supervised stage remains available
The downloadable Instruct2 checkpoint provides the supervised alignment stage. Its family card lists ichikara, AnswerCarefully, Magpie, English instruction data and code examples.LLM-jp-3 8×13B Instruct2 official model cardLLM-jp explains Instruct2 and Instruct3
Use the SFT checkpoint as a comparison
Practical guidance. Compare 8×13B Instruct2 and Instruct3 with identical generation settings. The pair is useful for studying whether preference tuning changes the responses an application needs.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3-8x13b-instruct2 |
| Context window | 4,096 tokens Maximum sequence length stated in the model card. LLM-jp-3 8×13B Instruct2 official model card |
| Architecture | 40 layers; hidden size 5120; eight experts, two active.LLM-jp-3 8×13B Instruct2 official model card |
| License | Apache 2.0.LLM-jp-3 8×13B Instruct2 official model card |
| Pretraining | 2.1 trillion seen tokens reported in the model card.LLM-jp-3 8×13B Instruct2 official model card |
| Ancestry | Drop-Upcycling from LLM-jp-3 13B; expert reinitialization ratio 0.5.LLM-jp-3 MoE training and release |
Related models
- LLM-jp-3 8×13B Base
LLM-jp / NII LLMC · LLM-jp-3 MoE
LLM-jp-3 8×13B Base is an eight-expert language model with two experts selected per token.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3 8×13B Instruct3
LLM-jp / NII LLMC · LLM-jp-3 MoE
LLM-jp-3 8×13B Instruct3 is an eight-expert language model with two experts selected per token.
Launch date not confirmedResearch releaseDownloadable