On November 27, the DeepSeek team released a massive 236B parameter model called DeepSeek-Math-V2 on Hugging Face, using a MoE architecture with only 21B active parameters, and expanding the context length to 128K tokens. The official Apache 2.0 weights were also released, with no commercial restrictions, which immediately overwhelmed the server bandwidth.
Overview of mathematical performance (zero-shot CoT):
- Achieved 75.7% on the MATH benchmark, nearly matching GPT-4o (76.6%);
- Solved 4 out of 30 problems in AIME2024, more than Gemini 1.5 Pro and Claude-3-Opus;
- Achieved 53.7% on Math Odyssey, also ranking among the top tier.
The core secret of the model is the "self-validation" dual engine: the Generator first produces a draft, and the Verifier checks line by line, sending errors back for rewriting, with up to 16 iterations, suppressing hallucinations through majority voting and meta-verification. The training corpus reached 100 billion tokens, including papers, competition questions, and synthetic data, and introduced GRPO reinforcement learning to align with human preferences.
Thanks to the mixed code-math corpus, DeepSeek-Math-V2 is also powerful in programming: 90.2% on HumanEval, 76.2% on MBPP, and SWEBench saw an open-source model break the 10% barrier for the first time, directly competing with GPT-4-Turbo and Claude 3 Opus.
The model is now available on Hugging Face, requiring only 80GB of GPU memory for multi-GPU inference; community reproductions are rapidly underway. If you want to give AI a "math gold medal" brain, all you need is one line of `transformers` loading—domestic open source once again cracks the moat of closed-source giants down to microscopic cracks.
