Optimizing cloud economics with linear elastic caching

Details

The core insight of Google's linear elastic caching paper, published at CIDR (Conference on Innovative Data Systems Research), is reframing cache eviction as a cost optimization problem rather than a resource-allocation problem. Traditional caching allocates a fixed memory budget and runs eviction policies like LRU when that budget fills—resulting in either performance degradation (too small) or wasted spend (too large). Linear elastic caching treats memory as a variable cost that integrates over time. Each cached data item faces an ongoing rental cost (memory occupied × time held) against a potential buy cost (the I/O and latency penalty of a cache miss if eviction triggers re-fetching). The system uses lightweight machine learning to predict re-access probability and make per-item eviction decisions that minimize total cost of ownership. The approach is particularly relevant to serverless environments, where some providers charge up to $3 per day per GiB. The ski-rental framing is well-established computer science theory—the contribution here is the production implementation at Google Cloud scale and the application to dynamic cache sizing. Engineers report meaningful TCO reduction without performance degradation, though specific figures are not quoted in the extracted content. The technique addresses a real engineering problem: cache sizing in production is currently an art of static estimation, and systems routinely over-allocate to avoid miss-rate penalties.