Nous ResearchYaRN
YaRN Mistral 7B 64K
This release extends Mistral-7B-v0.1 to a 64K context target using YaRN.
Research release
Training and model behavior
The 64K model received 1,000 additional training steps. The card documents long-context perplexity measurements and comparisons on shorter tasks. The runtime must apply the released YaRN configuration.YaRN Mistral 7B 64K model card
Working with this model
Practical guidance. Use an instruction-tuned model if your task requires dependable chat behavior. This base extension is useful for continued training or controlled long-context research.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | 2023 |
| Access | Downloadable |
| Model identifiers | NousResearch/Yarn-Mistral-7b-64k |
| Context window | 65,536 tokens Context specified by the model card. YaRN Mistral 7B 64K model card |
| Base model | mistralai/Mistral-7B-v0.1YaRN Mistral 7B 64K model card |
| Repository license label | apache-2.0YaRN Mistral 7B 64K model card |
Related models
A Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadable