FinBERT Base Cased
FinBERT Base Cased is TurkuNLP's Finnish BERT encoder. It is a pretrained model for adaptation to text tasks, rather than an instruction-following text generator.
Finnish pretraining
The cased release keeps distinctions between uppercase and lowercase text. Its Finnish training data cover news, online discussion and web crawls. The authors report more than three billion source tokens and one million pretraining steps. The project recommends this cased release for general use.FinBERT Base Cased: model card and weights
Earlier beta release
The project also lists the 30 September 2019 cased 0.1 beta. That earlier checkpoint is separate from this final v1 release. Its older archive is not presented here as a new current release.FinBERT: releases and training project
Use the matching tokenizer
Practical guidance. Keep the supplied tokenizer with the weights. Casing can affect names and abbreviations, so compare errors on those items before replacing this release with the uncased model.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | |
| Access | Downloadable |
| Model identifiers | bert-base-finnish-cased-v1 |
| Context window | 512 tokens Position limit in the released encoder configuration, including special tokens. FinBERT Base Cased: model configuration |
| Version | 1.0FinBERT: releases and training project |
| Architecture | 12 encoder layers, hidden width 768 and 12 attention heads.FinBERT Base Cased: model configuration |
| Vocabulary | 50,105 entries in the published configuration.FinBERT Base Cased: model configuration |
| Weight license | CC BY 4.0, as stated by the FinBERT project.FinBERT: explicit license for trained models |
Related models
- FinBERT Base Uncased
TurkuNLP · FinBERT
FinBERT Base Uncased is a Finnish masked-language encoder with a dedicated WordPiece vocabulary.
Launch date Downloadable - FinBERT Large Cased
TurkuNLP · FinBERT
FinBERT Large Cased increases encoder depth and width while using the same Finnish training data as FinBERT Base.
Launch date not confirmedDownloadable