Llama 3.1 8B TFree-HAT BASE
This TFree-HAT research checkpoint starts from the Llama 3.1 8B base backbone and continues pretraining in English and German.
Research release
Model design
It replaces Llama's tokenizer with a hierarchical architecture that encodes and decodes characters around a word-level backbone. The Base checkpoint precedes the supervised and preference-trained variants.Llama 3.1 8B TFree-HAT BASE model card
Practical use
Practical guidance. Use Base for tokenizer and adaptation research. Compare bytes or words processed as well as sequence positions. A shorter internal sequence is not sufficient evidence of lower end-to-end latency with a different inference implementation.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | Aleph-Alpha/llama-3_1-8b-tfree-hat-base |
| License | Open Aleph License for non-commercial research and education.Llama 3.1 8B TFree-HAT BASE model card |
Related models
- Llama 3.1 8B TFree-HAT DPO
Aleph Alpha · TFree-HAT
English-and-German TFree-HAT instruction model further trained with direct preference optimization.
Launch date not confirmedResearch releaseDownloadable - Llama 3.1 8B TFree-HAT SFT
Aleph Alpha · TFree-HAT
Tokenizer-free Llama 3.1 8B adaptation with supervised instruction training in English and German.
Launch date not confirmedResearch releaseDownloadable