FinGPT 13B
FinGPT 13B is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series. It uses the BLOOM architecture and is not an OpenAI GPT-3 checkpoint.
Model design and training
40 layers, width 5,120 and 40 attention heads. The card reports 300 billion training tokens. The Finnish corpus combines web text, news, books, Wikipedia and discussion data. This is a pure language model, without dialogue or question-answering instruction tuning.gpt3-finnish-13B: official model card
Practical assessment
Practical guidance. This is the largest Finnish-only model in the released FinGPT series. Its larger weight memory is separate from the resources needed for long inputs and batching. Use it as a scale comparison or an adaptation base, and evaluate whether gains appear on the actual Finnish task.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | gpt3-finnish-13BFinnish GPT-3 13B |
| Model size | 13.3 billion parameters.gpt3-finnish-13B: official model card |
| License | Apache 2.0.gpt3-finnish-13B: official model card |
| Training state | Pretrained base; not instruction-tuned.gpt3-finnish-13B: official model card |
Related models
- BLUUMI 176B
TurkuNLP · BLUUMI
BLUUMI adds Finnish capability to BigScience's multilingual BLOOM 176B model through continued pretraining.
Launch date not confirmedDownloadable FinGPT 8B is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series.
Launch date not confirmedDownloadable