Hermes RWKV v5 7B
This adaptation starts from a 7B RWKV v5 model pretrained on more than one trillion multilingual tokens. It adds OpenHermes 2.5 chat training. The base architecture belongs to the RWKV project; this entry describes the trained adaptation by EleutherAI.
Research release
Weight format does not create another model
The project also publishes a Transformers-format version under Hermes-RWKV-v5-7B-HF. That is a distribution of this adaptation, not another seven-billion-parameter model. The original and converted files must use their matching loader.Hermes-RWKV-v5-7B model cardHermes-RWKV-v5-7B model configuration
Use the released chat markers
These adaptations replace the usual ChatML start and end markers with ASCII control characters 0x16 and 0x17. The card describes this choice for RWKV tokenizer compatibility. Literal ChatML tokens are not interchangeable with the published format.Hermes-RWKV-v5-7B model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | February 2024 |
| Access | Downloadable |
| Model identifiers | EleutherAI/Hermes-RWKV-v5-7BEleutherAI/Hermes-RWKV-v5-7B-HF |
| Base architecture | RWKV v5 EagleHermes-RWKV-v5-7B model cardRWKV language-model project |
| Nominal parameters | 7BHermes-RWKV-v5-7B model card |
| License | Apache 2.0Hermes-RWKV-v5-7B model card |
Related models
- Hermes RWKV v5 3B
EleutherAI ยท Hermes RWKV
A three-billion-parameter recurrent chat adaptation of RWKV v5.
Launch date not confirmedResearch releaseDownloadable