LLM-jp-3.1 8×13B 32K Base
LLM-jp-3.1 8×13B 32K Base extends the 8×13B line from 4K to 32K context with another 50 billion training tokens.
Research release
Long-context training changes the checkpoint
The additional 50 billion tokens follow LLM-jp-3.1 8×13B training. The card's introductory release statement identifies the longer context, while the shared architecture table remains unchanged.LLM-jp-3.1 8×13B 32K Base official model card
Use the extension in controlled comparisons
Practical guidance. Compare this checkpoint with the 4K model on documents that fit both, then on longer inputs. That separates behavior changes from the simple ability to accept more tokens.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3.1-8x13b-32K |
| Context window | 32,768 tokens max_position_embeddings in the release configuration; the introductory card describes the 4K-to-32K extension. LLM-jp-3.1 8×13B 32K Base configuration |
| Architecture | Eight experts; two selected per token; 73,162,828,800 total parameters.LLM-jp-3.1 8×13B 32K Base official model card |
| License | Apache 2.0.LLM-jp-3.1 8×13B 32K Base official model card |
| Training distinction | A continued-training release, not a serving configuration that changes a context limit.LLM-jp-3.1 8×13B 32K Base official model card |
Related models
- LLM-jp-3.1 8×13B 32K Instruct4
LLM-jp / NII LLMC · LLM-jp-3.1
LLM-jp-3.1 8×13B 32K Instruct4 is a separately trained long-context variant of the 8×13B model.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3.1 8×13B Base
LLM-jp / NII LLMC · LLM-jp-3.1
LLM-jp-3.1 8×13B Base adds instruction pretraining to LLM-jp-3.
Launch date not confirmedResearch releaseDownloadable