MERaLiON SpeechEncoder v1
SpeechEncoder v1 is a 630M-class speech representation model. It was trained from scratch with self-supervised learning. Its main focus is English, including Singapore speech, rather than direct text generation.
A speech encoder trained from scratch
The release uses BEST-RQ learning to predict quantized targets from masked audio features. The card reports 170,000 hours of English, including 10,000 hours from Singapore, and 30,000 hours across 113 other languages. Its training is distinct from the Whisper-derived AudioLLM line.MERaLiON SpeechEncoder v1 official model card
From acoustic features to a task model
The examples extract hidden features from 16 kHz audio. A separate example adds a CTC head and trains a recognizer. Loading the base encoder does not, by itself, reproduce a trained transcription system.MERaLiON SpeechEncoder v1 official model card
An older downloadable foundation
The v1 public license lists MIT for this encoder, subject to its added notice, redistribution and other conditions. No separate current hosted endpoint is established for this checkpoint.MERaLiON Public Licence v1, 2024
Match the encoder to your data
Practical guidance. Use this model to measure how Singapore-focused pretraining affects a downstream task. Compare both feature extraction and supervised adaptation. An encoder score after training a task head does not describe a ready-to-use transcription service.MERaLiON SpeechEncoder technical report
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | MERaLiON-SpeechEncoder-v1 |
| Architecture | 24 Conformer layers; 1,024 hidden dimensions.MERaLiON SpeechEncoder v1 published configuration |
| Training corpus | 200,000 hours of unlabelled speech.MERaLiON SpeechEncoder technical report |
| Output | Latent acoustic features for downstream models.MERaLiON SpeechEncoder v1 official model card |
Related models
- MERaLiON AudioLLM Whisper SEA-LION
MERaLiON / A*STAR IAIC · MERaLiON AudioLLM
A regional speech-text model that adapts a speech encoder and text model for audio questions and transcription.
Launch date not confirmedDownloadable - MERaLiON SpeechEncoder 2
MERaLiON / A*STAR IAIC · MERaLiON SpeechEncoder
A 630M-class multilingual speech encoder trained from scratch to provide acoustic features for downstream tasks.
Launch date not confirmedDownloadable