Nous ResearchYaRN
YaRN Llama 2 70B 32K
This release adds YaRN training to extend Llama 2 70B to a 32K context target.
Research release
Training and model behavior
The model received 400 additional training steps. Its card credits LAION compute support and the JUWELS supercomputer. Inference needs the release's position-scaling implementation.YaRN Llama 2 70B 32K model card
Working with this model
Practical guidance. Evaluate retrieval and completion quality with the same length settings used by your application. Keep this continued-pretraining model separate from instruction-tuned Llama releases.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | 2023 |
| Access | Downloadable |
| Model identifiers | NousResearch/Yarn-Llama-2-70b-32k |
| Context window | 32,768 tokens Context specified by the model card. YaRN Llama 2 70B 32K model card |
| Base model | Meta Llama 2 70BYaRN Llama 2 70B 32K model card |
| Repository license label | apache-2.0YaRN Llama 2 70B 32K model card |
Related models
A Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadable