EleutherAI

From a volunteer group to a research institute

Connor Leahy, Sid Black and Leo Gao founded the group in July 2020. It began on Discord with an effort to reproduce large language models openly. The Pile dataset followed at the end of that year. GPT-Neo and GPT-J arrived in 2021, followed by GPT-NeoX in 2022. The group incorporated as a nonprofit in early 2023.

Stella Biderman leads the institute as executive director. The staff page also names Quentin Anthony as head of high-performance computing, Nora Belrose as head of interpretability, and Aviya Skowron as head of policy and ethics.

A nonprofit supported by research funding

The EleutherAI Institute is a United States 501(c)(3) organization. Donations, grants and donated computing resources support its research. Its funding policy says it retains control of its research agenda. It is not a listed company, so stock-market capitalization does not describe its size.

Access to the training process

The institute publishes more than final model weights. Training software, data descriptions and intermediate checkpoints let other researchers examine how model behavior changes. Its GPT-NeoX software is also used in independently led projects. Use of that software does not make those projects EleutherAI models.

Pythia makes this approach visible through matched training runs. The original eight sizes here describe the corrected releases, rather than the earlier v0 runs. Those v0 weights and the later PolyPythias random-seed experiments remain research variants. Intermediate steps and seed variations are grouped with the relevant model sizes, rather than counted as separate products.

Language, mathematics and model behavior

The Polyglot team developed Korean models with data from TUNiB and computing support from Stability AI. Its contribution to Japanese StableLM focused mainly on data collection. Those Stability AI releases are not reassigned to EleutherAI here.

Llemma combines mathematical text and code in continued training of Code Llama models. It is a collaboration with academic researchers. The institute also studies what models learn, how researchers can interpret that knowledge, and whether training-data choices affect the safety of open weights.

Comma is a joint Common Pile project, published under the Common Pile account. Its models study training with public-domain and openly licensed text. The articles identify that collaboration rather than presenting the models as an EleutherAI-only release.

Model releases and research variants

Practical guidance. The entries distinguish models trained from scratch from adaptations of Code Llama, Mamba and RWKV. Training snapshots, tensor formats and optimizer files stay with their parent release. The Deep Ignorance and Quirky Models articles group their controlled research variants. A downloadable research model does not imply that EleutherAI operates a hosted API.