Meet the team that turns your ideas into work.

Discover EightBots
EIGHTBOTS
Contact salesSign up
TurkuNLPFinnish ModernBERT

Finnish ModernBERT Tiny EDU

Finnish ModernBERT Tiny EDU is a multilingual masked-language encoder with 51 million parameters. Its final training stage uses multilingual educational data.

Training and configuration

The card identifies a separate educational-data annealing release. It reports 439.3 billion training tokens. This release keeps the 51M-parameter size-specific-vocabulary architecture; it is not a quantized format of the baseline model.Finnish ModernBERT Tiny EDU: model card and weightsFinnish ModernBERT Tiny EDU: released configuration

Practical assessment

Practical guidance. Compare Tiny EDU with Tiny on the same labelled examples. The final data mixture changes, but the model remains a masked-language encoder. A classification or retrieval application still needs its own task setup.

Specifications & access

Finnish ModernBERT Tiny EDU specifications
DetailValue and conditions
AccessDownloadable
Model identifiersfinnish-modernbert-tiny-edu
Context window16,000 tokens

max_position_embeddings in this release's configuration; theoretical extrapolation is not recorded as an established input limit.

Finnish ModernBERT Tiny EDU: released configuration
LicenseApache 2.0.Finnish ModernBERT Tiny EDU: model card and weights
Architecture6 layers, hidden width 768 and 12 attention heads.Finnish ModernBERT Tiny EDU: released configuration
Vocabulary27,264 entries in the released configuration.Finnish ModernBERT Tiny EDU: released configuration
Training volume439.3 billion tokens, as stated by this release's card.Finnish ModernBERT Tiny EDU: model card and weights

Related models

  • Finnish ModernBERT Tiny

    TurkuNLP ยท Finnish ModernBERT

    Finnish ModernBERT Tiny provides 16,000-token encoder inputs with 51 million parameters.

    Launch date not confirmedDownloadable