ALIA 40B
ALIA 40B is BSC's larger multilingual base model. It provides the foundation for the centre's later 40B instruction and tool-calling releases.
Model and training
BSC reports 9.37 trillion pretraining tokens across 35 European languages and code. The decoder has 48 layers, an 8,192-wide hidden state and 64 attention heads. Final training includes a high-quality data stage and context extension. The rotary-position method draws on Llama 3.1's technique; this does not make the model a Meta-weight derivative.ALIA 40B: official model card
Practical use
Practical guidance. Use this base model for continued pretraining and controlled research. It requires a different serving budget from the smaller Salamandra releases. Do not assume that a base completion model can follow the same system prompt as its instructed descendants. Compare downstream tuning results against an available instructed baseline.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | ALIA-40b |
| Context window | 32,768 tokens Maximum sequence length documented for these weights; the prompt and generated text share the window. ALIA 40B: official model card |
| License | Apache License 2.0.ALIA 40B: official model card |
| Model size | 40,433,885,184 parameters.ALIA 40B: official model card |
Related models
ALIA 40B Instruct 2601 is a long-context conversational release from BSC's 40B language model family.
Launch date not confirmedDownloadableALIA 40B Instruct 2606 combines long-context instruction following with BSC's helpfulness and safety post-training.
Launch date not confirmedDownloadable