Kog is going deeper to squeeze more inference out of GPUs

August 14, 2026 | Source: TechCrunch AI

Tags: Kog, GPU-inference, inference-optimization, LLM-speed, Laneformer, Cerebras

French startup Kog hit 3,000 tokens/second on standard H200/MI300X GPUs in a software-only demo, generated 200 business leads from a Hacker News front page, and is now racing to scale from a 2B-param proof-of-concept to full-size LLMs.

Details

Kog, a solo-founded French startup, is betting that conventional datacenter GPUs (AMD MI300X, NVIDIA H200) are dramatically underutilized for AI inference and can be unlocked through deep software optimization rather than purpose-built chips. Its Kog Inference Engine (KIE) demo hit 3,000 per-request tokens/second using a 2B-parameter open-source model (Laneformer 2B, now open-sourced), earning front-page Hacker News attention in May and 200 tangible business leads. CEO Gaël Delalleau says software engineering workflows are the primary target — specifically practitioners who wait hours for results in tools like Claude Code. Anthropic already charges a premium for Claude's Fast Mode, validating that inference speed has real willingness-to-pay. Kog is positioning KIE as the infrastructure layer that brings that speed to enterprises' existing GPU fleets without hardware replacement. The critical challenge: scaling KIE from the 2B-param demo to large LLMs. Delalleau is confident newer GPUs' growing memory bandwidth makes this feasible — and that the misconception that GPUs are poorly suited for agentic workflows is exactly the gap Kog is exploiting. Comparable French startup ZML takes a hardware-agnostic approach; Kog goes deeper at the kernel level, more akin to Stanford's Hazy Research lab.