Ai2 (Allen Institute for AI)Olmo Hybrid
Olmo Hybrid 7B
Olmo Hybrid 7B is a pretrained hybrid recurrent and attention model.
Research release
What the release provides
Three quarters of its layers use gated DeltaNet operations. Each group of three recurrent layers is followed by an attention layer. The card gives a 65,536-token context and Apache 2.0 terms.Olmo Hybrid 7B official model card
How to use it well
Practical guidance. Treat the base checkpoint as a training foundation. Compare memory use and throughput across input lengths; do not assume a recurrent design removes all attention costs.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | allenai/Olmo-Hybrid-7B |
| Context window | 65,536 tokens Pretrained checkpoint context Olmo Hybrid 7B official model card |