Nous ResearchYaRN
YaRN SOLAR 10.7B 64K
This release adapts Upstage SOLAR-10.7B-v1.0 to a 64K context target.
Research release
Training and model behavior
The card describes continued pretraining on two billion long-context tokens. It publishes length-dependent perplexity results and the position-scaling configuration needed to use the extension.YaRN SOLAR 10.7B 64K model card
Working with this model
Practical guidance. Retain the exact scaling setup when comparing the 32K and 64K variants. Plan memory for the full sequence, including the output, when serving long documents.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | January 2024 |
| Access | Downloadable |
| Model identifiers | NousResearch/Yarn-Solar-10b-64k |
| Context window | 65,536 tokens Context specified by the model card. YaRN SOLAR 10.7B 64K model card |
| Base model | upstage/SOLAR-10.7B-v1.0YaRN SOLAR 10.7B 64K model card |
| Repository license label | apache-2.0YaRN SOLAR 10.7B 64K model card |
Related models
A Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadable