FinGPT XL
FinGPT XL is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series. It uses the BLOOM architecture and is not an OpenAI GPT-3 checkpoint.
Model design and training
24 layers, width 2,064 and 24 attention heads. The card reports 300 billion training tokens. The Finnish corpus combines web text, news, books, Wikipedia and discussion data. This is a pure language model, without dialogue or question-answering instruction tuning.gpt3-finnish-xl: official model card
Practical assessment
Practical guidance. XL crosses into the billion-parameter range without adding layers over Large. Estimate memory from the actual precision and runtime rather than the size label. For task adaptation, retain a baseline on general Finnish so improvements do not hide loss of broader language behavior.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | gpt3-finnish-xlFinnish GPT-3 XL |
| Model size | 1.5 billion parameters.gpt3-finnish-xl: official model card |
| License | Apache 2.0.gpt3-finnish-xl: official model card |
| Training state | Pretrained base; not instruction-tuned.gpt3-finnish-xl: official model card |
Related models
- FinGPT 13B
TurkuNLP · FinGPT
FinGPT 13B is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series.
Launch date not confirmedDownloadable - FinGPT Small
TurkuNLP · FinGPT
FinGPT Small is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series.
Launch date not confirmedDownloadable