Nous ResearchOLMo Bitnet
OLMo Bitnet 1B
This 1B model tests the 1.58-bit training method using OLMo software and Dolma data.
Research release
Training and model behavior
Nous trained the proof of concept on the first sixty billion tokens of Dolma. A separate run used standard FP16 weights with the same hyperparameters for comparison. The release includes example inference code.OLMo Bitnet 1B model card
Working with this model
Practical guidance. Use this artifact to study the training method and its implementation. Do not assume a generic quantized-model loader reproduces the training scheme.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | March 2024 |
| Access | Downloadable |
| Model identifiers | NousResearch/OLMo-Bitnet-1B |
| Repository license label | apache-2.0OLMo Bitnet 1B model card |
| Parameter scale | 1BOLMo Bitnet 1B model card |