KyutaiHelium
Sequential Helium 6B
This Helium 6B research family compares training on text in date order with training on shuffled data. It studies how a model retains changing facts and associates answers with the correct time period.
Research release
The repository includes temporal comparisons
The sequential checkpoints cover stages from 2020 to 2024 before the main release. Three shuffled controls use token budgets matched to selected sequential stages. The names refer to training conditions, not live knowledge updates. None of these base models is instruction tuned.Sequential_Helium_6B: Kyutai model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | kyutai/Sequential_Helium_6Bsequential_2020sequential_2021sequential_2022sequential_2023sequential_2024shuffle_eq2020shuffle_eq2024shuffle_eq2025 |
| License | CC BY-SA 4.0.Sequential_Helium_6B: Kyutai model card |
| Main checkpoint | 2.5 trillion training tokens; source dates from 2018 to the 2025 cutoff.Sequential_Helium_6B: Kyutai model card |
Related models
- Helium 1 2B
Kyutai ยท Helium
A 2B base language model for the 24 official EU languages, built by combining selected trained variants.
Launch date not confirmedResearch releaseDownloadable