Llama-Prometheus
Llama-Prometheus is the English evaluator in AI4Bharat's CIA suite. It fine-tunes Llama-3.1-8B-Instruct on Feedback-Collection using the Prometheus 2 training setup.
The training method does not make it the Prometheus 2 checkpoint
The release uses a different base model while following the cited evaluation-training approach. Its output contains feedback and a separate score marker.Llama-Prometheus official model card
Keep the rubric stable across candidates
Practical guidance. A score cannot be compared fairly when the reference answer or score definitions change between runs. Inspect feedback when a numerical ranking conflicts with a human review.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | llama-prometheus |
| Prompt inputs | Instruction, candidate answer, reference answer and a detailed 1–5 scoring rubric.Llama-Prometheus official model card |
| Terms | MIT declared in the card; Llama 3.1 upstream terms also apply.Llama-Prometheus official model card |
Related models
- Hercule Bengali
AI4Bharat · Hercule
Fully fine-tuned Llama-3.1-8B-Instruct evaluator for Bengali answers against English references.
Launch date not confirmedDownloadable - Hercule Hindi
AI4Bharat · Hercule
Fully fine-tuned Hercule evaluator for Hindi answers against English references.
Launch date not confirmedDownloadable