Meet the team that turns your ideas into work.

Discover EightBots
EIGHTBOTS
Contact salesSign up
TurkuNLPFinnish ModernBERT

Finnish ModernBERT Tiny Short EDU

Finnish ModernBERT Tiny Short EDU is a multilingual masked-language encoder with 129 million parameters. Its final training stage uses multilingual educational data.

Training and configuration

The card identifies a separate educational-data annealing release. It reports 362.2 billion training tokens. This release keeps the 129M-parameter 128,000-vocabulary architecture; it is not a quantized format of the baseline model.Finnish ModernBERT Tiny Short EDU: model card and weightsFinnish ModernBERT Tiny Short EDU: released configuration

Practical assessment

Practical guidance. Keep the Short and EDU parts of the identifier in an experiment record. They identify different vocabulary and training-data choices. Compare against Tiny Short to isolate the effect of the final data mixture.

Specifications & access

Finnish ModernBERT Tiny Short EDU specifications
DetailValue and conditions
AccessDownloadable
Model identifiersfinnish-modernbert-tiny-short-edu
Context window8,192 tokens

max_position_embeddings in this release's configuration; theoretical extrapolation is not recorded as an established input limit.

Finnish ModernBERT Tiny Short EDU: released configuration
LicenseApache 2.0.Finnish ModernBERT Tiny Short EDU: model card and weights
Architecture6 layers, hidden width 768 and 12 attention heads.Finnish ModernBERT Tiny Short EDU: released configuration
Vocabulary128,000 entries in the released configuration.Finnish ModernBERT Tiny Short EDU: released configuration
Training volume362.2 billion tokens, as stated by this release's card.Finnish ModernBERT Tiny Short EDU: model card and weights

Related models

  • Finnish ModernBERT Tiny Short

    TurkuNLP ยท Finnish ModernBERT

    Finnish ModernBERT Tiny Short provides 8,192-token encoder inputs with 129 million parameters.

    Launch date not confirmedDownloadable