FinGPT 3B
FinGPT 3B is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series. It uses the BLOOM architecture and is not an OpenAI GPT-3 checkpoint.
Model design and training
32 layers, width 2,560 and 32 attention heads. The card reports 300 billion training tokens. The Finnish corpus combines web text, news, books, Wikipedia and discussion data. This is a pure language model, without dialogue or question-answering instruction tuning.gpt3-finnish-3B: official model card
Practical assessment
Practical guidance. Use the actual 2.8-billion count when comparing this model with another model called 3B. Its main role is a monolingual base for adaptation or research. Check whether the intended task benefits from instruction tuning before designing a chat interface around it.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | gpt3-finnish-3BFinnish GPT-3 3B |
| Model size | 2.8 billion parameters in the architecture table.gpt3-finnish-3B: official model card |
| License | Apache 2.0.gpt3-finnish-3B: official model card |
| Training state | Pretrained base; not instruction-tuned.gpt3-finnish-3B: official model card |
Related models
- FinGPT 13B
TurkuNLP · FinGPT
FinGPT 13B is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series.
Launch date not confirmedDownloadable - FinGPT Small
TurkuNLP · FinGPT
FinGPT Small is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series.
Launch date not confirmedDownloadable