Finnish ModernBERT Tiny Short
Finnish ModernBERT Tiny Short is a multilingual masked-language encoder with 129 million parameters. It uses the family's 128,000-entry vocabulary and an 8,192-token configuration.
Training and configuration
Training covers Finnish, Swedish, English, code, Latin and Northern Sámi. The card reports 362.2 billion tokens. This Short architecture has more vocabulary entries than the corresponding longer-window model.Finnish ModernBERT Tiny Short: model card and weightsFinnish ModernBERT Tiny Short: released configuration
Checkpoint for continued training
finnish-modernbert-tiny-short-cpt is a downloadable checkpoint from before learning-rate decay. Its card reports 357.3 billion tokens and recommends it for continued pretraining. It is recorded here as a training-stage variant of this model, rather than another final release.Finnish ModernBERT Tiny Short: checkpoint for continued pretraining
Practical assessment
Practical guidance. The Short label does not mean a smaller weight file. Its larger vocabulary makes this six-layer model larger than Tiny. Use the full published identifier when recording deployment memory and tokenizer choices.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | finnish-modernbert-tiny-short |
| Context window | 8,192 tokens max_position_embeddings in this release's configuration; theoretical extrapolation is not recorded as an established input limit. Finnish ModernBERT Tiny Short: released configuration |
| License | Apache 2.0.Finnish ModernBERT Tiny Short: model card and weights |
| Architecture | 6 layers, hidden width 768 and 12 attention heads.Finnish ModernBERT Tiny Short: released configuration |
| Vocabulary | 128,000 entries in the released configuration.Finnish ModernBERT Tiny Short: released configuration |
| Training volume | 362.2 billion tokens, as stated by this release's card.Finnish ModernBERT Tiny Short: model card and weights |
Related models
- Finnish ModernBERT Tiny
TurkuNLP · Finnish ModernBERT
Finnish ModernBERT Tiny provides 16,000-token encoder inputs with 51 million parameters.
Launch date not confirmedDownloadable - Finnish ModernBERT Tiny Short EDU
TurkuNLP · Finnish ModernBERT
Finnish ModernBERT Tiny Short EDU provides 8,192-token encoder inputs with 129 million parameters and educational-data annealing.
Launch date not confirmedDownloadable