Meet the team that turns your ideas into work.

Discover EightBots
EIGHTBOTS
Contact salesSign up
TurkuNLPFinnish ModernBERT

Finnish ModernBERT Base Short EDU

Finnish ModernBERT Base Short EDU is a multilingual masked-language encoder with 209 million parameters. Its final training stage uses multilingual educational data.

Training and configuration

The card identifies a separate educational-data annealing release. It reports 365.0 billion training tokens. This release keeps the 209M-parameter 128,000-vocabulary architecture; it is not a quantized format of the baseline model.Finnish ModernBERT Base Short EDU: model card and weightsFinnish ModernBERT Base Short EDU: released configuration

Practical assessment

Practical guidance. This release is suitable for a controlled comparison with Base Short. Keep document truncation identical in both runs; otherwise the input selection can hide the effect of the educational-data stage.

Specifications & access

Finnish ModernBERT Base Short EDU specifications
DetailValue and conditions
AccessDownloadable
Model identifiersfinnish-modernbert-base-short-edu
Context window8,192 tokens

max_position_embeddings in this release's configuration; theoretical extrapolation is not recorded as an established input limit.

Finnish ModernBERT Base Short EDU: released configuration
LicenseApache 2.0.Finnish ModernBERT Base Short EDU: model card and weights
Architecture22 layers, hidden width 768 and 12 attention heads.Finnish ModernBERT Base Short EDU: released configuration
Vocabulary128,000 entries in the released configuration.Finnish ModernBERT Base Short EDU: released configuration
Training volume365.0 billion tokens, as stated by this release's card.Finnish ModernBERT Base Short EDU: model card and weights

Related models

  • Finnish ModernBERT Base Short

    TurkuNLP ยท Finnish ModernBERT

    Finnish ModernBERT Base Short provides 8,192-token encoder inputs with 209 million parameters.

    Launch date not confirmedDownloadable