SuperCLUE released programming-specific evaluation data for Tencent's Hy3 language model, pitting it against popular models like DeepSeek-V4-Pro to compare their coding capabilities. Hy3 adopts an MoE architecture, with a total parameter scale of 295B and only 21B activated parameters. It supports a context length of up to 256K and is claimed to be the strongest language model in the Huan Yuan series. The results show that this model, with a much smaller parameter size than its competitors, delivered an unexpectedly strong performance in programming scenarios.
Comprehensive comparison across four dimensions, Hy3 balances performance and cost
This evaluation was specifically designed for real-world coding scenarios commonly encountered by Chinese programmers. Each programming task requires dozens of back-and-forth communications, gradually completing code analysis, modification, and verification, more closely resembling actual development and debugging processes. The evaluation compares models based on four dimensions: usage cost, runtime speed, communication rounds, and token consumption, helping developers choose suitable models based on their budget and efficiency needs.
In terms of coding ability scores, the high-performance version of Hy3 scored 47.37, tying with DeepSeek-V4-Pro. It's worth noting that many competing models have parameters several times larger than Hy3 but failed to create a gap, indicating extensive optimizations in its model architecture and training methods. Cost-wise, Hy3's advantage is even more pronounced—completing a single programming question costs an average of just 0.43 yuan, making it affordable for long-term, frequent use and significantly reducing the financial burden of commercial deployment.
Less than 400 seconds per question, 40 rounds of conversation complete the entire task
In terms of runtime speed, Hy3 takes less than 400 seconds on average per question, ranking among the top in the evaluation, making it suitable for real-time coding and online debugging scenarios requiring quick responses. Communication efficiency is also impressive, as the average number of conversation rounds needed to complete an entire task is over 40, without the need for repeated instruction adjustments to fix code, resulting in faster convergence. At the same time, each task consumes only about 1.16 million tokens, requiring less cloud computing power and placing less pressure on platform operations.
For programmers who frequently use AI-assisted development, Hy3 has found a rare balance between performance and cost. When parameter size is no longer the sole measure of capability, "small in size but big in power" models are proving through actual data that optimized architectures and training strategies can also compete with flagship-level opponents on the programming track. With Hy3 now fully open-sourced under the Apache 2.0 license, this cost-effectiveness battle may have just begun.
