Register Ablation Models
TurkuNLP publishes the models from its Register Always Matters study as a controlled set of training-data experiments. The branches represent different trained models, not quantized copies.
Research release
What changes across the experiments
The study changes the register, or text type, used for pretraining. Its results show that single-register datasets and combinations have different strengths. The findings concern this controlled training setup; they do not establish that a preferred mixture will remain best at every model size.Register Always Matters: authors' study of pretraining text types
Exact published branch names
The live repository exposes main, ne, SP, OP, NA, MT, LY, IP, IN, ID, HI-IN_HI_dtp_OP, HI-IN_HI_dtp_NA_OP_ID_SP, HI-IN_HI_dtp_NA_OP_ID, HI-IN_HI_dtp_NA_OP, HI-IN_HI_dtp, HI-IN and HI. Preserve case and punctuation when selecting a revision.Register ablations: published branch identifiers
Select the experiment, then the checkpoint
Practical guidance. Record the repository branch as well as the iteration directory. Loading the main branch alone does not reproduce every register experiment. Compare the models under the same prompts and evaluation conditions to keep data-selection effects separate from inference changes.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | register-ablations |
| Context window | 2,048 tokens Position limit in the main branch's released configuration. Register ablations: main checkpoint configuration |
| Architecture | The main checkpoint uses LlamaForCausalLM: 24 layers, hidden width 2,048 and 32 attention heads.Register ablations: main checkpoint configuration |
| Published location | Weights are stored in iter_0050000 within the selected branch.Register ablations: main checkpoint configuration |
| License scope | The main repository card declares Apache 2.0; no independent terms are inferred for older branch cards.Register ablations: research model repository |
Related models
- Multilingual Web Register Classifier
TurkuNLP ยท Web register classification
This XLM-RoBERTa Large classifier assigns hierarchical text-type labels to web documents.
Launch date not confirmedDownloadable