ECCO-BERT Base Cased
ECCO-BERT was pretrained only on Eighteenth Century Collections Online. The authors designed it for later fine-tuning on tasks that use that historical collection.
A model for a defined historical collection
The training domain is digitized documents published in the United Kingdom during the eighteenth century. The card describes a cased BERT base-sized model. It does not provide a full training recipe or a general-purpose benchmark claim.ECCO-BERT Base Cased: model card
Check the source text
Practical guidance. Use historical text when assessing this encoder. Compare results on original transcription and corrected text if both are available. A model trained on this collection should not be assumed to handle modern English in the same way.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | eccobert-base-cased-v1 |
| Context window | 512 tokens Position limit in the released configuration. ECCO-BERT Base Cased: configuration |
| Architecture | BertForPreTraining; 12 layers, hidden width 768 and 12 attention heads.ECCO-BERT Base Cased: configuration |
| Vocabulary | 50,000 entries.ECCO-BERT Base Cased: configuration |
| License | No explicit weight license is stated in the reviewed model card.ECCO-BERT Base Cased: model card |
Related models
- FinBERT Base Cased
TurkuNLP ยท FinBERT
FinBERT Base Cased is a Finnish masked-language encoder with a dedicated WordPiece vocabulary.
Launch date Downloadable