Product
Xiaohongshu and Peking University Open-Source UltraEP: A Real-Time Load Balancing Solution for Large-Scale MoE Training and Inference
Xiaohongshu and Peking University propose UltraEP, which for the first time introduces real-time load balancing based on precise routing information into production systems, dynamically replicating hot experts in each microbatch and each layer. On models like Qwen3-235B, training throughput averages 94.6% of ideal performance, a 42% improvement over Megatron-LM; inference prefill throughput is 1.56x higher than SGLang.
Read the original (opens in a new tab)
News stream data aggregated by AI HOT