LLM-jp / NII LLMCLLM-jp-3
LLM-jp-3 7.2B Base
LLM-jp-3 7.2B Base is the 7.2B pretrained text model in LLM-jp's generation-3 size series. The underlying model was exposed to 2.1 trillion pretraining tokens.
Research release
The size changes the architecture
The 32-layer model uses hidden width 4,096 and 32 attention heads. Pretraining includes Japanese web and archive material, English Dolma sources and code from The Stack.LLM-jp-3 7.2B Base official model card
Use the size as an experimental control
Practical guidance. This is a dense comparison point for the smaller expert model. Compare total memory and response latency, not only active parameter labels.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3-7.2b |
| Context window | 4,096 tokens Maximum sequence length stated in the model card. LLM-jp-3 7.2B Base official model card |
| Architecture | 32 layers; hidden size 4,096; 32 attention heads.LLM-jp-3 7.2B Base official model card |
| Parameters | 7,291,269,120 total, from the card's embedding and non-embedding counts.LLM-jp-3 7.2B Base official model card |
| License | Apache 2.0; research weights.LLM-jp-3 7.2B Base official model card |
Related models
- LLM-jp-3 7.2B Instruct3
LLM-jp / NII LLMC ยท LLM-jp-3
LLM-jp-3 7.2B Instruct3 is a preference-aligned instruction release with 4,096-token context and Apache 2.0 weights.
Launch date not confirmedResearch releaseDownloadable