Meet the team that turns your ideas into work.

Discover EightBots
EIGHTBOTS
Contact salesSign up
TurkuNLPFinnish ModernBERT

Finnish ModernBERT Large EDU

Finnish ModernBERT Large EDU is a multilingual masked-language encoder with 401 million parameters. Its final training stage uses multilingual educational data.

Training and configuration

The card identifies a separate educational-data annealing release. It reports 393.8 billion training tokens. This release keeps the 401M-parameter size-specific-vocabulary architecture; it is not a quantized format of the baseline model.Finnish ModernBERT Large EDU: model card and weightsFinnish ModernBERT Large EDU: released configuration

Practical assessment

Practical guidance. Large EDU can support comparisons of the final training mixture while keeping the architecture fixed. Use the same tokenizer, batching and downstream training settings when comparing it with Large.

Specifications & access

Finnish ModernBERT Large EDU specifications
DetailValue and conditions
AccessDownloadable
Model identifiersfinnish-modernbert-large-edu
Context window16,000 tokens

max_position_embeddings in this release's configuration; theoretical extrapolation is not recorded as an established input limit.

Finnish ModernBERT Large EDU: released configuration
LicenseApache 2.0.Finnish ModernBERT Large EDU: model card and weights
Architecture28 layers, hidden width 1,024 and 16 attention heads.Finnish ModernBERT Large EDU: released configuration
Vocabulary55,616 entries in the released configuration.Finnish ModernBERT Large EDU: released configuration
Training volume393.8 billion tokens, as stated by this release's card.Finnish ModernBERT Large EDU: model card and weights

Related models

  • Finnish ModernBERT Large

    TurkuNLP ยท Finnish ModernBERT

    Finnish ModernBERT Large provides 16,000-token encoder inputs with 401 million parameters.

    Launch date not confirmedDownloadable