Polyglot-Ko 5.8B
A Korean autoregressive language model developed by the EleutherAI Polyglot team. The released weights support text completion and further adaptation.
Research release
Korean modeling at this scale
Polyglot-Ko 5.8B has fewer layers than the 3.8B release but a wider hidden state of 4,096. Parameter scale therefore does not increase through depth alone. The network uses 16 attention heads.polyglot-ko-5.8b model cardpolyglot-ko-5.8b model configuration
Training data and use
The model generates continuations from Korean text. It was not trained to follow a chat role template. KoAlpaca adaptations are separate fine-tuning work, and loading the base checkpoint does not load those adaptations.polyglot-ko-5.8b model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | EleutherAI/polyglot-ko-5.8b |
| Context window | 2,048 tokens Maximum position count in the released model configuration. polyglot-ko-5.8b model configuration |
| Parameters | 5,885,059,072polyglot-ko-5.8b model card |
| Language | Koreanpolyglot-ko-5.8b model card |
| License | Apache 2.0polyglot-ko-5.8b model card |
Related models
- Polyglot-Ko 1.3B
EleutherAI · Polyglot-Ko
A compact Korean base model from the Polyglot project.
Launch date not confirmedResearch releaseDownloadable - Polyglot-Ko 12.8B
EleutherAI · Polyglot-Ko
The largest documented Polyglot-Ko base model, trained on Korean text.
Launch date not confirmedResearch releaseDownloadable - Polyglot-Ko 3.8B
EleutherAI · Polyglot-Ko
A Korean text-completion model with 32 Transformer layers.
Launch date not confirmedResearch releaseDownloadable