Polyglot-Ko 12.8B
A Korean autoregressive language model developed by the EleutherAI Polyglot team. The released weights support text completion and further adaptation.
Research release
Korean modeling at this scale
The 40-layer network uses a hidden width of 5,120 and 40 attention heads. It was trained for 167 billion tokens across 301,000 steps on 256 A100 GPUs with GPT-NeoX.polyglot-ko-12.8b model cardpolyglot-ko-12.8b model configuration
Training data and use
The card describes masking bank-account, resident-registration and phone numbers during preprocessing. This is a training-data step, not a guarantee that generated text contains no personal information. The source corpus remains unavailable for public use.polyglot-ko-12.8b model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | EleutherAI/polyglot-ko-12.8b |
| Context window | 2,048 tokens Maximum position count in the released model configuration. polyglot-ko-12.8b model configuration |
| Parameters | 12,898,631,680polyglot-ko-12.8b model card |
| Language | Koreanpolyglot-ko-12.8b model card |
| License | Apache 2.0polyglot-ko-12.8b model card |
Related models
- Polyglot-Ko 1.3B
EleutherAI · Polyglot-Ko
A compact Korean base model from the Polyglot project.
Launch date not confirmedResearch releaseDownloadable - Polyglot-Ko 3.8B
EleutherAI · Polyglot-Ko
A Korean text-completion model with 32 Transformer layers.
Launch date not confirmedResearch releaseDownloadable - Polyglot-Ko 5.8B
EleutherAI · Polyglot-Ko
A wider Korean base model with 28 layers and downloadable weights.
Launch date not confirmedResearch releaseDownloadable