Falcon3 Mamba 7B Instruct
Falcon3 Mamba 7B Instruct continues the original Falcon Mamba model with additional training. It uses a pure Mamba1 architecture rather than the attention-and-Mamba hybrid introduced by H1.
Training and release background
The December 2024 release adds 1.5 trillion training tokens and instruction tuning on 1.2 million samples. It has 64 decoder blocks and a width of 4,096. The card states a 32K context and names TII Falcon-LLM License 2.0.Falcon3 Mamba 7B Instruct model card
Compare recurrent and hybrid behavior
Practical guidance. Use this model to compare a pure state-space design with H1 on the same document task. Measure factual recall as well as throughput. A recurrent architecture's memory behavior does not establish reliable recall of every detail in an arbitrarily long input.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | December 2024 |
| Access | Downloadable |
| Model identifiers | tiiuae/Falcon3-Mamba-7B-Instruct |
| License | TII Falcon-LLM License 2.0Falcon3 Mamba 7B Instruct model card |
| Release status | Earlier release; published weight access remains availableFalcon3 Mamba 7B Instruct model card |
Related models
Falcon H1 7B Instruct is a multilingual hybrid model with a published 262,144-position configuration.
Launch date not confirmedDownloadableFalcon Mamba 7B Instruct is the earlier pure state-space Falcon model.
Launch date not confirmedDownloadable