Jamba v0.1
Jamba v0.1 is the original base model published in March 2024. Its mixture-of-experts network combines attention and state-space layers, using 12 billion active parameters from 52 billion total.
Why attention and Mamba were combined
AI21 used attention to address recall limitations in a pure state-space model, while retaining Mamba's efficient sequence processing. The release established the Joint Attention and Mamba architecture used by later Jamba models.AI21 introduces the Jamba architecture
This checkpoint is not an instruction assistant
The model card warns that this is a pretrained base release without instruction tuning or moderation. It is intended as a starting point for further training.Jamba-v0.1 official model card
Practical guidance. Do not interpret fluent text completion as reliable chat behavior. Applications that need conversation, tool calls or controlled answers need an appropriate adapted model.
Mamba blocks need separate treatment
The card's 8-bit loading example leaves Mamba blocks outside the quantized modules to avoid quality loss. This makes the recommended loading path different from simply quantizing every linear layer.Jamba-v0.1 official model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | |
| Access | Downloadable |
| Model identifiers | Jambaai21labs/Jamba-v0.1 |
| Context window | 262,144 tokens Maximum position count in the published weight configuration; deployed limits depend on the serving setup. Jamba-v0.1 configuration |
| Parameters | 52 billion total; 12 billion activeJamba-v0.1 official model card |
| Weight license | Apache 2.0Jamba-v0.1 official model card |
Related models
- Jamba Tiny Dev
AI21 Labs · Jamba research
319-million-parameter trained development checkpoint for work on the Jamba architecture.
Launch date Research releaseDownloadable - Jamba Mini 1.5
AI21 Labs · Jamba
Original instruction-tuned Mini model with tool calls, JSON output and long text input.
Launch date Deprecated API; check access detailsDownloadable · API - Jamba-Instruct
AI21 Labs · Jamba
Historical hosted instruction version of the original Jamba architecture.
Launch date Access not confirmed