Sakana AIRePo
RePo-OLMo2-7B-stage2-L10
Sakana continues training Ai2's OLMo-2-1124-7B using its RePo technique and the Dolmino mixture.
Research release
RePo changes how the model uses context
The source checkpoint had completed its first pretraining stage. Sakana applied context repositioning during further training on stage-two data. This remains a base language-model release, not a separately documented chat fine-tune.RePo-OLMo2-7B-stage2-L10 official model card
Working with this release
Practical guidance. Use completion tasks and the project's evaluation setup when studying the method. Do not assume that a base checkpoint has instruction-model behavior.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | SakanaAI/RePo-OLMo2-7B-stage2-L10 |
| Base model | allenai/OLMo-2-1124-7BRePo-OLMo2-7B-stage2-L10 official model card |
| Further training | 50 billion tokens from allenai/dolmino-mix-1124.RePo-OLMo2-7B-stage2-L10 official model card |
| Terms | Apache 2.0; research prototype with commercial and critical deployment discouraged.RePo-OLMo2-7B-stage2-L10 official model card |
Related models
- RePo-OLMo2-1B-stage2-L5
Sakana AI ยท RePo
An OLMo 2 derivative trained with context repositioning.
Launch date not confirmedResearch releaseDownloadable