INTELLECT-1
INTELLECT-1 was pretrained from scratch on one trillion tokens of English text and code. Its release demonstrates language-model training with compute from independent contributors.
What the release provides
Training used up to 14 concurrent nodes across three continents. Thirty contributors supplied compute. The framework supports workers joining or leaving, and reduces communication with local training steps and compressed synchronization.INTELLECT-1 official model card
How to use it well
Practical guidance. Use this base model for continued training or text completion. For a chat application, start with the instruction-trained release and compare its behavior on the same prompts.
Include the start-of-sequence token
The card requires a beginning-of-sequence token for each sample. Leaving it out can reduce performance.INTELLECT-1 official model card
Precision and training checkpoints
The organization also provides an FP32 package and intermediate step checkpoints. These are formats or training snapshots of INTELLECT-1, rather than additional model families.INTELLECT-1 full-precision packagePrime Intellect official model organization
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | |
| Access | Downloadable |
| Context window | 8,192 tokens Configured position limit. Serving limits depend on the runtime and settings. INTELLECT-1 model configuration |
| Repository license | Apache 2.0INTELLECT-1 official model card |
| Parameters | 10 billionINTELLECT-1 official model card |
Related models
- INTELLECT-1-Instruct
Prime Intellect ยท INTELLECT
The instruction-trained version of the 10-billion-parameter INTELLECT-1 model.
Launch date not confirmedDownloadable