Pythia 410M Deduped
Pythia 410M Deduped is the deduplicated-Pile counterpart of Pythia 410M. It preserves the architecture while changing the training data.
Research release
A mid-sized model for data-removal studies
The deduplicated checkpoint has 24 layers and a hidden width of 1,024. Comparing it with the standard-data model keeps those architecture choices fixed. The comparison concerns the training corpus; it is not an instruction-tuning comparison.pythia-410m-deduped model cardpythia-410m model configuration
Weights and training revisions
The release includes 154 saved training steps. The data order is shared with other sizes in the same suite. The main branch provides the final model; a step revision selects an earlier checkpoint. This base model has not been trained as a conversational assistant.pythia-410m-deduped model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | |
| Access | Downloadable |
| Model identifiers | EleutherAI/pythia-410m-deduped |
| Context window | 2,048 tokens Maximum position count in the released model configuration. pythia-410m model configuration |
| Nominal parameters | 410M including embeddingspythia-410m-deduped model card |
| Network | 24 layers; hidden width 1,024; 16 attention headspythia-410m model configuration |
| Training data | Deduplicated Pilepythia-410m-deduped model card |
| License | Apache 2.0pythia-410m-deduped model card |
Related models
- Pythia 410M
EleutherAI ยท Pythia
A 24-layer research model with a 1,024-wide hidden state.
Launch date Research releaseDownloadable