Nous ResearchYaRN
YaRN Mistral 7B 128K
This release extends Mistral-7B-v0.1 to a 128K context target using YaRN.
Research release
Training and model behavior
The 128K model received 1,500 additional training steps. The card documents long-context perplexity measurements and comparisons on shorter tasks. The runtime must apply the released YaRN configuration.YaRN Mistral 7B 128K model card
Working with this model
Practical guidance. Use an instruction-tuned model if your task requires dependable chat behavior. This base extension is useful for continued training or controlled long-context research.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | |
| Access | Downloadable |
| Model identifiers | NousResearch/Yarn-Mistral-7b-128k |
| Context window | 131,072 tokens Context specified by the model card. YaRN Mistral 7B 128K model card |
| Base model | mistralai/Mistral-7B-v0.1YaRN Mistral 7B 128K model card |
| Repository license label | apache-2.0YaRN Mistral 7B 128K model card |
Related models
A Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadableA Llama 2 model with continued training for longer context.
Launch date 2023Research releaseDownloadable