WikiBERT Language Models
WikiBERT is a collection of language-specific BERT models trained from Wikipedia. Each language identifier selects separate trained weights. This article lists the published language releases together; it does not describe one multilingual checkpoint.
Separate encoders for separate languages
The original paper introduced 42 monolingual models. The live catalog now lists 44 WikiBERT language repositories. The paper reports that parsing improvements vary by language, even though average performance improves. That finding does not justify selecting the same model family for every language without checking the downstream task.WikiBERT Models: Deep Transfer Learning for Many LanguagesTurkuNLP: public model repositories and artifact inventory
Published language releases from Afrikaans to Estonian
Afrikaans: wikibert-base-af-cased.WikiBERT Afrikaans: trained model repository
Arabic: wikibert-base-ar-cased.WikiBERT Arabic: trained model repository
Belarusian: wikibert-base-be-cased.WikiBERT Belarusian: trained model repository
Bulgarian: wikibert-base-bg-cased.WikiBERT Bulgarian: trained model repository
Catalan: wikibert-base-ca-cased.WikiBERT Catalan: trained model repository
Czech: wikibert-base-cs-cased.WikiBERT Czech: trained model repository
Danish: wikibert-base-da-cased.WikiBERT Danish: trained model repository
German: wikibert-base-de-cased.WikiBERT German: trained model repository
Greek: wikibert-base-el-cased.WikiBERT Greek: trained model repository
English: wikibert-base-en-cased.WikiBERT English: trained model repository
Spanish: wikibert-base-es-cased.WikiBERT Spanish: trained model repository
Estonian: wikibert-base-et-cased.WikiBERT Estonian: trained model repository
Published releases from Basque to Italian
Basque: wikibert-base-eu-cased.WikiBERT Basque: trained model repository
Persian: wikibert-base-fa-cased.WikiBERT Persian: trained model repository
Finnish: wikibert-base-fi-cased.WikiBERT Finnish: trained model repository
French: wikibert-base-fr-cased.WikiBERT French: trained model repository
Irish: wikibert-base-ga-cased.WikiBERT Irish: trained model repository
Galician: wikibert-base-gl-cased.WikiBERT Galician: trained model repository
Hebrew: wikibert-base-he-cased.WikiBERT Hebrew: trained model repository
Hindi: wikibert-base-hi-cased.WikiBERT Hindi: trained model repository
Croatian: wikibert-base-hr-cased.WikiBERT Croatian: trained model repository
Hungarian: wikibert-base-hu-cased.WikiBERT Hungarian: trained model repository
Armenian: wikibert-base-hy-cased.WikiBERT Armenian: trained model repository
Indonesian: wikibert-base-id-cased.WikiBERT Indonesian: trained model repository
Italian: wikibert-base-it-cased.WikiBERT Italian: trained model repository
Published releases from Japanese to Norwegian
Japanese: wikibert-base-ja-cased.WikiBERT Japanese: trained model repository
Korean: wikibert-base-ko-cased.WikiBERT Korean: trained model repository
Lithuanian: wikibert-base-lt-cased.WikiBERT Lithuanian: trained model repository
Latvian: wikibert-base-lv-cased.WikiBERT Latvian: trained model repository
Dutch: wikibert-base-nl-cased.WikiBERT Dutch: trained model repository
Norwegian: wikibert-base-no-cased.WikiBERT Norwegian: trained model repository
Published releases from Polish to Vietnamese
Polish: wikibert-base-pl-cased.WikiBERT Polish: trained model repository
Portuguese: wikibert-base-pt-cased.WikiBERT Portuguese: trained model repository
Romanian: wikibert-base-ro-cased.WikiBERT Romanian: trained model repository
Russian: wikibert-base-ru-cased.WikiBERT Russian: trained model repository
Slovak: wikibert-base-sk-cased.WikiBERT Slovak: trained model repository
Slovenian: wikibert-base-sl-cased.WikiBERT Slovenian: trained model repository
Serbian: wikibert-base-sr-cased.WikiBERT Serbian: trained model repository
Swedish: wikibert-base-sv-cased.WikiBERT Swedish: trained model repository
Tamil: wikibert-base-ta-cased.WikiBERT Tamil: trained model repository
Turkish: wikibert-base-tr-cased.WikiBERT Turkish: trained model repository
Ukrainian: wikibert-base-uk-cased.WikiBERT Ukrainian: trained model repository
Urdu: wikibert-base-ur-cased.WikiBERT Urdu: trained model repository
Vietnamese: wikibert-base-vi-cased.WikiBERT Vietnamese: trained model repository
Keep each model with its own vocabulary
Practical guidance. Use the language-specific vocabulary distributed with the selected weights. Equal layer counts and vocabulary sizes do not make token IDs interchangeable. For Finnish, compare WikiBERT with FinBERT using the same task data, because their training collections differ.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | WikiBERT |
| Context window | 512 tokens Position limit in the reviewed WikiBERT configurations; the cited Finnish file shows this shared design. WikiBERT Finnish: released BERT configuration |
| Model design | 12 encoder layers, hidden width 768 and 12 attention heads in the reviewed configurations.WikiBERT Finnish: released BERT configuration |
| Vocabulary | 20,101 entries per reviewed configuration; equal size does not mean shared token IDs.WikiBERT Finnish: released BERT configuration |
| Published coverage | 44 language repositories in the reviewed TurkuNLP catalog.TurkuNLP: public model repositories and artifact inventory |
| License evidence | The model repositories do not declare individual weight licenses. The project's GitHub repository has an MIT license, whose scope should not be assumed to supply missing per-weight terms.TurkuNLP: public model repositories and artifact inventoryWikiBERT: official project and language-specific model distribution |
Related models
- FinBERT Base Cased
TurkuNLP ยท FinBERT
FinBERT Base Cased is a Finnish masked-language encoder with a dedicated WordPiece vocabulary.
Launch date Downloadable