Hermes RWKV v5 3B
This is a 3B RWKV v5 Eagle adaptation trained on OpenHermes 2.5. The base learned from multilingual text. The published chat dataset is English, so the multilingual base alone does not establish equivalent instruction performance across languages.
Research release
A custom Transformers implementation
The configuration selects Rwkv5ForCausalLM with 32 layers and a hidden width of 2,560. Its repository includes custom model code. Inspect that implementation before allowing a loader to execute remote repository code.Hermes-RWKV-v5-3B-HF model cardHermes-RWKV-v5-3B-HF model configuration
Use the released chat markers
These adaptations replace the usual ChatML start and end markers with ASCII control characters 0x16 and 0x17. The card describes this choice for RWKV tokenizer compatibility. Literal ChatML tokens are not interchangeable with the published format.Hermes-RWKV-v5-3B-HF model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | EleutherAI/Hermes-RWKV-v5-3B-HF |
| Base architecture | RWKV v5 EagleHermes-RWKV-v5-3B-HF model cardRWKV language-model project |
| Nominal parameters | 3BHermes-RWKV-v5-3B-HF model card |
| License | Apache 2.0Hermes-RWKV-v5-3B-HF model card |
Related models
- Hermes RWKV v5 7B
EleutherAI ยท Hermes RWKV
A seven-billion-parameter RWKV chat model with recurrent inference.
Launch date February 2024Research releaseDownloadable