Anatomy of a Quantized Agent: VRAM Stability and Forecasting in Code-Synthesis Agentic Workloads

August 18, 2026 | Source: arXiv AI

Tags: VRAM, quantization, LLM agents, code synthesis, inference optimization, Qwen2.5-Coder

Across 1,920 trajectories on a single H100, simple closed-form analytical models predict peak VRAM for 4-bit quantized LLM agents as accurately as learned models (MAPE 2.2-4.4%), while code synthesis success varies from 5.7% (Phi-4-mini) to 62.0% (Qwen2.5-Coder-14B) regardless of VRAM.

Details

Planning VRAM budgets for deployed LLM agents typically involves guesswork or conservative overprovisioning. This paper provides a rigorous measurement study of peak VRAM behavior in quantized (Q4_K_M) agentic code-synthesis workloads, using a LangGraph-based CUDA kernel synthesis agent across four LLM backbones on a single H100. The main finding: closed-form analytical models — decomposing VRAM into weight storage, KV-cache, and activation terms — achieve competitive forecasting accuracy when given just two empirical constants (loaded weight VRAM and a fixed activation overhead). On three of four backbones, this simple model matches or beats learned regression baselines (MAPE 2.2-4.4% vs 3.4-6.5%). More practically significant: VRAM variance across trajectories is remarkably low (CV 0.3-9.4%), meaning complex predictive models add little value in highly quantized regimes. Code synthesis success, however, varies dramatically by backbone capability — from 5.7% for Phi-4-mini to 62.0% for Qwen2.5-Coder-14B — confirming that model selection matters far more than memory budgeting for agentic code synthesis.