AI Chips News and AI Updates
Follow AI Chips developments across AI companies, labs, and open-source projects.
Latest AI Chips news, research, benchmarks, product releases, and industry adoption updates.
Latest Articles
- Creating Power Flexibility in Datacenters via Query Routing — A query-routing framework lets LLM datacenters cut power draw on demand without deferring requests, reaching up to 16% power flexibility in a simulated 10,000-H100 site, with 90% of the target cut landing within about 10 seconds.
- Twice as Fast, Until It Isn't: Quantized Serving of Mid-Size Open LLMs — An audit of public benchmarks for serving 9B-20B open LLMs finds 4-bit quantization roughly doubles small-batch speed but the gain disappears at large batches, and many reported accuracy losses sit within noise.
- HybridSpec: Throughput-Aware Speculative Decoding for Hybrid GDN-MoE Models — HybridSpec picks speculative-decoding verification length on the GPU for hybrid Gated DeltaNet and MoE models, running 1.75x-1.88x faster than autoregressive decoding and about 1.3x faster than DFlash.
- Look Before You Launch: Learned Throughput Prediction for Hardware Selection in a Production Training Fleet — LEAP, a production service that predicts training-job throughput across 20+ accelerator types, now ranks 95% of eligible GPU-hours and saves an estimated 3% of training compute.
- FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM Training — FP4Quant compresses inter-GPU communication in model-parallel LLM training to 4-bit, reporting up to 1.56x end-to-end throughput over BF16 on H100s with Qwen-7B, with a trade-off in model quality.
- ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration — ROCm InferaSim predicts LLM serving performance from a few short kernel benchmarks, replacing roughly 25,000 GPU-hours of trial-and-error tuning with single-digit GPU-hours and errors under 10%.
- Formalizing PTX in Lean to Prove GPU Kernel Correctness — Researchers formalized NVIDIA's PTX in Lean and used it to prove correctness of simple WGMMA GEMM, FlashAttention and a three-kernel MoE pipeline, bringing formal proofs to deep-learning GPU kernels.
- RoboScope Kit: Unified Interfaces for Fast and Scalable Embodied-AI Evaluation — RoboScope Kit wraps robot policies and benchmarks behind shared interfaces and smarter scheduling, cutting 500-episode RoboTwin evaluation from about six hours to 14.4 minutes on four GPUs with identical outcomes.
- GC-MoE: Overlapping Expert-Parallel Token Dispatch with FP8/FP4 Grouped GEMM via CUDA Green Contexts — GC-MoE uses CUDA Green Contexts to split GPUs into separate communication and compute partitions for MoE inference, speeding DeepSeek-V4.1-Flash decode by about 1.19x on 8 H20s and beating DeepGEMM Mega MoE.
- Cortex Training: Serverless Fine-Tuning for Open-Weight LLMs — Snowflake put Cortex Training in public preview, a serverless service for fine-tuning open-weight LLMs with SFT, RL and custom training loops, claiming up to 61% lower cost than competing serverless offerings.