These startups are chasing the next big thing in LLMs

Details

Transformers' dense attention mechanism requires comparing every token with every other token — a process that scales quadratically with sequence length. A 10,000-word document may require 50 million multiplications. At production scale, this has OpenAI spending an estimated $50 billion on compute in 2026, with the IEA projecting global data center electricity consumption to double by 2030.\n\nA cohort of startups is now building what MIT Technology Review calls LLMs+: architectures that replace or augment dense attention with approaches that handle longer context more cheaply. Subquadratic, whose name states the bet directly, is one example. These companies are betting that recent LLM advances — reasoning models, large context windows — are workarounds on top of fundamental transformer limitations, not extensions of its core strengths.\n\nThe strategic opening is structural: incumbents like OpenAI, Anthropic, and Google have the most infrastructure and talent invested in transformers, giving challengers less to lose from architectural bets that would obsolete existing investments. The article is behind a paywall; specific alternative architectures covered by each startup are not fully captured in the available extract.