LLM-jp-3 172B alpha2
LLM-jp-3 172B alpha2 is an early 172B release whose pretraining consumed 1.4 trillion tokens. The project retains these weights despite performance below its expected level.
Research release
A failed training setting is part of the record
The card warns that the alpha models perform substantially below the project's other released models. This version retains the affected pretraining checkpoint. It remains downloadable for research into training behavior.LLM-jp-3 172B alpha2 official model card
Use it to study the failure
Practical guidance. Treat alpha2 as an experimental artifact. It is useful for understanding the training run, but its large parameter count is not a reason to select it over a later model.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3-172b-alpha2 |
| Context window | 4,096 tokens Maximum sequence length stated in the model card. LLM-jp-3 172B alpha2 official model card |
| Architecture | 96 layers; hidden size 12,288; 96 attention heads.LLM-jp-3 172B alpha2 official model card |
| License | Apache 2.0.LLM-jp-3 172B alpha2 official model card |
| Release role | Pretrained alpha checkpoint.LLM-jp-3 172B alpha2 official model card |
Related models
- LLM-jp-3 172B alpha2 Instruct
LLM-jp / NII LLMC · LLM-jp-3 172B
LLM-jp-3 172B alpha2 Instruct preserves an early research release with a documented training problem.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3 172B Instruct3
LLM-jp / NII LLMC · LLM-jp-3 172B
The 172B preference-aligned release has a separate license that permits redistribution subject to attribution and other terms.
Launch date not confirmedResearch releaseDownloadable