Meta Expands Its Custom Silicon Strategy From Compute Into Networking

Details

Meta's MTIA 300 is the company's first custom accelerator designed specifically for training recommendation and ranking models — workloads where embedding tables can contain more than 99% of parameters and frequent AllReduce, AllToAll, and AllGather operations make the network as critical as raw compute. Unlike LLM training where floating-point throughput dominates, Meta built MTIA 300 around communication-compute co-design.\n\nThe architecture integrates two network chiplets into the accelerator package, each with six custom 800 Gbps RDMA NICs, providing 1.2 TB/s of total I/O bandwidth without traversing a PCIe bus. The same 12 NICs handle both scale-up traffic within a rack and scale-out traffic between racks, giving Meta the flexibility to reallocate bandwidth without chip redesigns.\n\nA key design choice is 16 dedicated message engines that handle collective communication independently from the main compute grid, including near-memory hardware for reduction operations. Meta reports that this lets large matrix operations and collective communication run concurrently with less than 0.5% compute degradation, compared to more than 20% on conventional GPUs. The co-designed HCCL library compiles collective operations into subgraphs the message engines execute autonomously — the host CPU is no longer in the critical path. In production, HCCL reaches up to 940 GB/s of communication bandwidth within a rack, and on a 150B-parameter recommendation model across 40 accelerators, MTIA 300 reduced total communication time by 3.9x.