FinBERT Base Uncased
FinBERT Base Uncased is TurkuNLP's Finnish BERT encoder. It is a pretrained model for adaptation to text tasks, rather than an instruction-following text generator.
Finnish pretraining
The uncased release provides a separate Finnish checkpoint that normalizes case. It belongs to the final 1.0 release, rather than the earlier 2019 beta. Its training draws on Finnish news, discussion and web text.FinBERT Base Uncased: model card and weights
Earlier beta release
The project also lists the 24 October 2019 uncased 0.2 beta. That earlier checkpoint is separate from this final v1 release. Its older archive is not presented here as a new current release.FinBERT: releases and training project
Use the matching tokenizer
Practical guidance. This release is useful when input case is inconsistent. Check whether normalization removes distinctions needed by your labels, especially for names. A task head trained with the cased model cannot be assumed to transfer unchanged.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | |
| Access | Downloadable |
| Model identifiers | bert-base-finnish-uncased-v1 |
| Context window | 512 tokens Position limit in the released encoder configuration, including special tokens. FinBERT Base Uncased: model configuration |
| Version | 1.0FinBERT: releases and training project |
| Architecture | 12 encoder layers, hidden width 768 and 12 attention heads.FinBERT Base Uncased: model configuration |
| Vocabulary | 50,101 entries in the published configuration.FinBERT Base Uncased: model configuration |
| Weight license | CC BY 4.0, as stated by the FinBERT project.FinBERT: explicit license for trained models |
Related models
- FinBERT Base Cased
TurkuNLP · FinBERT
FinBERT Base Cased is a Finnish masked-language encoder with a dedicated WordPiece vocabulary.
Launch date Downloadable - FinBERT Large Cased
TurkuNLP · FinBERT
FinBERT Large Cased increases encoder depth and width while using the same Finnish training data as FinBERT Base.
Launch date not confirmedDownloadable