FinGPT Small
FinGPT Small is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series. It uses the BLOOM architecture and is not an OpenAI GPT-3 checkpoint.
Model design and training
12 layers, width 768 and 12 attention heads. The card reports 300 billion training tokens. The Finnish corpus combines web text, news, books, Wikipedia and discussion data. This is a pure language model, without dialogue or question-answering instruction tuning.gpt3-finnish-small: official model card
Practical assessment
Practical guidance. The small model can support controlled studies of Finnish text continuation with a lower weight-memory requirement. Use it as a baseline when the experiment matters more than chat quality. Keep training data and prompts comparable when measuring changes from fine-tuning.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | gpt3-finnish-smallFinnish GPT-3 Small |
| Model size | 186 million parameters.gpt3-finnish-small: official model card |
| License | Apache 2.0.gpt3-finnish-small: official model card |
| Training state | Pretrained base; not instruction-tuned.gpt3-finnish-small: official model card |
Related models
- FinGPT Medium
TurkuNLP ยท FinGPT
FinGPT Medium is a Finnish text-generation base model in TurkuNLP's Finnish GPT-3 series.
Launch date not confirmedDownloadable