Hy4 Preview
Hy4 Preview is a text model with a 770B mixture-of-experts backbone and 49B active parameters per token. A separate native multi-token prediction layer adds 10B total parameters.
Preview
Model design and access
The 78-layer backbone uses 256 routed experts per MoE layer, selecting eight plus a shared expert. Gated sparse attention reuses index information between layers. The MTP layer supports speculative decoding. BF16 and FP8 repositories are formats of the same release.Hy4 Preview official model card
Practical use
Practical guidance. Budget for all expert weights even though only part of the network runs per token. For local serving, assess total memory, cache growth and communication between GPUs. Use the separate Hy4 API when your workflow requires the newly documented image input.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Preview |
| Access | API, Downloadable, Provider product |
| Model identifiers | hy4-preview |
| License | Apache 2.0Hy4 Preview official model card |
| Context | 1M tokensHy4 Preview official model card |
| Access | Weights, TokenHub API and Tencent applicationsTencent Hy4 preview release and access |
Related models
A hosted general-purpose model for text, image understanding, tool calls and long tasks.
Launch date not confirmedAPIHy3 is a 295B mixture-of-experts language model with 21B active parameters and a separate 3.8B multi-token prediction layer.
Launch date not confirmedDownloadable · API