Meet the team that turns your ideas into work.

Discover EightBots
EIGHTBOTS
Contact salesSign up
TurkuNLPFinnish ModernBERT

Finnish ModernBERT Large Short EDU

Finnish ModernBERT Large Short EDU is a multilingual masked-language encoder with 475 million parameters. Its final training stage uses multilingual educational data.

Training and configuration

The card identifies a separate educational-data annealing release. It reports 362.2 billion training tokens. This release keeps the 475M-parameter 128,000-vocabulary architecture; it is not a quantized format of the baseline model.Finnish ModernBERT Large Short EDU: model card and weightsFinnish ModernBERT Large Short EDU: released configuration

Practical assessment

Practical guidance. Use Large Short EDU when studying the final training data in the 475-million-parameter design. Do not substitute its weights into a retrieval index built with the MS MARCO derivative.

Specifications & access

Finnish ModernBERT Large Short EDU specifications
DetailValue and conditions
AccessDownloadable
Model identifiersfinnish-modernbert-large-short-edu
Context window8,192 tokens

max_position_embeddings in this release's configuration; theoretical extrapolation is not recorded as an established input limit.

Finnish ModernBERT Large Short EDU: released configuration
LicenseApache 2.0.Finnish ModernBERT Large Short EDU: model card and weights
Architecture28 layers, hidden width 1,024 and 16 attention heads.Finnish ModernBERT Large Short EDU: released configuration
Vocabulary128,000 entries in the released configuration.Finnish ModernBERT Large Short EDU: released configuration
Training volume362.2 billion tokens, as stated by this release's card.Finnish ModernBERT Large Short EDU: model card and weights

Related models

  • Finnish ModernBERT Large Short

    TurkuNLP ยท Finnish ModernBERT

    Finnish ModernBERT Large Short provides 8,192-token encoder inputs with 475 million parameters.

    Launch date not confirmedDownloadable