Sakana AIRePo
RePo-OLMo2-1B-stage2-L5
Sakana continues training Ai2's OLMo-2-0425-1B using its RePo technique and the Dolmino mixture.
Research release
RePo changes how the model uses context
The source checkpoint had completed its first pretraining stage. Sakana applied context repositioning during further training on stage-two data. This remains a base language-model release, not a separately documented chat fine-tune.RePo-OLMo2-1B-stage2-L5 official model card
Working with this release
Practical guidance. Use completion tasks and the project's evaluation setup when studying the method. Do not assume that a base checkpoint has instruction-model behavior.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | SakanaAI/RePo-OLMo2-1B-stage2-L5 |
| Base model | allenai/OLMo-2-0425-1BRePo-OLMo2-1B-stage2-L5 official model card |
| Further training | 50 billion tokens from allenai/dolmino-mix-1124.RePo-OLMo2-1B-stage2-L5 official model card |
| Terms | Apache 2.0; research prototype with commercial and critical deployment discouraged.RePo-OLMo2-1B-stage2-L5 official model card |
Related models
An OLMo 2 derivative trained with context repositioning.
Launch date not confirmedResearch releaseDownloadable