LLM-jp-3 172B alpha1 Instruct
LLM-jp-3 172B alpha1 Instruct is an early 172B release whose pretraining consumed 700 billion tokens. The project retains these weights despite performance below its expected level.
Research release
A failed training setting is part of the record
The card warns that the alpha models perform substantially below the project's other released models. This version adds instruction tuning to the affected base. It remains downloadable for research into training behavior.LLM-jp-3 172B alpha1 Instruct official model card
Use it to study the failure
Practical guidance. Treat alpha1 as an experimental artifact. It is useful for understanding the training run, but its large parameter count is not a reason to select it over a later model.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-3-172b-alpha1-instruct |
| Context window | 4,096 tokens Maximum sequence length stated in the model card. LLM-jp-3 172B alpha1 Instruct official model card |
| Architecture | 96 layers; hidden size 12,288; 96 attention heads.LLM-jp-3 172B alpha1 Instruct official model card |
| License | Apache 2.0.LLM-jp-3 172B alpha1 Instruct official model card |
| Release role | Instruction-tuned alpha checkpoint.LLM-jp-3 172B alpha1 Instruct official model card |
Related models
- LLM-jp-3 172B alpha1
LLM-jp / NII LLMC · LLM-jp-3 172B
LLM-jp-3 172B alpha1 preserves an early research release with a documented training problem.
Launch date not confirmedResearch releaseDownloadable - LLM-jp-3 172B Instruct3
LLM-jp / NII LLMC · LLM-jp-3 172B
The 172B preference-aligned release has a separate license that permits redistribution subject to attribution and other terms.
Launch date not confirmedResearch releaseDownloadable