LFM2.5 Q4\_0 Checkpoints from Quantization-Aware Distillation

Details

Liquid AI has released quantization-aware distillation (QAD) Q4_0 GGUF checkpoints for the LFM2.5 family: 230M, 350M, 1.2B-Instruct, and 2.6B. Standard post-training quantization (PTQ) to Q4_0 typically sacrifices noticeable quality, forcing developers toward larger formats like Q4_K_M or Q5_K_M. QAD addresses this by distilling a high-precision BF16 teacher into the quantized student during training rather than just truncating weights afterward.\n\nBenchmark results span reasoning (GPQA Diamond), instruction-following (IFEval, IFBench, Multi-IF), tool use (BFCLv4), and math (GSM8K, AIME25). The QAD checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their BF16 baselines respectively. On real hardware — MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5 — the 230M and 350M models match Q5_K_M quality at 4-33% higher decode throughput. The 1.2B and 2.6B models match Q4_K_M quality at 3-14% higher throughput.\n\nThe checkpoints are also benchmarked against Unsloth's UD-Q4_K_XL where applicable, matching that strong external baseline. All four models are now available on Hugging Face and work with llama.cpp or any GGUF Q4_0 runtime. For developers targeting edge deployment with a fixed memory budget, these offer the best accuracy-throughput tradeoff in the LFM2.5 line.