LLM-jp-3.1 8×13B 32K Instruct4
LLM-jp-3.1 8×13B 32K Instruct4 extends the 8×13B line from 4K to 32K context with another 50 billion training tokens.
Research release
Long-context training changes the checkpoint
The release combines the 32K continued-training variant with Instruct4 post-training. The card describes supervised fine-tuning and two-stage DPO. Its inherited family table still lists 4,096, so the specific 32K release statement is the relevant context claim.LLM-jp-3.1 8×13B 32K Instruct4 official model card
Evaluate instructions across long inputs
Practical guidance. Check whether answers remain grounded when the relevant passage is far from the prompt's end. A larger window allows more input; it does not itself show that every part is used reliably.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3.1-8x13b-32K-instruct4 |
| Context window | 32,768 tokens max_position_embeddings in the release configuration; the introductory card describes the 4K-to-32K extension. LLM-jp-3.1 8×13B 32K Instruct4 configuration |
| Architecture | Eight experts; two selected per token; 73,162,828,800 total parameters.LLM-jp-3.1 8×13B 32K Instruct4 official model card |
| License | Apache 2.0.LLM-jp-3.1 8×13B 32K Instruct4 official model card |
| Training distinction | A continued-training release, not a serving configuration that changes a context limit.LLM-jp-3.1 8×13B 32K Instruct4 official model card |
Related models
- LLM-jp-3.1 8×13B 32K Base
LLM-jp / NII LLMC · LLM-jp-3.1
LLM-jp-3.1 8×13B 32K Base is a separately trained long-context variant of the 8×13B model.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3.1 8×13B Instruct4
LLM-jp / NII LLMC · LLM-jp-3.1
LLM-jp-3.1 8×13B Instruct4 adds instruction pretraining to LLM-jp-3 and SFT plus two-stage preference alignment.
Launch date not confirmedResearch releaseDownloadable