Meet Needle 2: An Open 45M-Parameter Tool-Calling Model That Ships as a 14MB Binary and Runs a Full Session in 28MB of RAM

Details

Cactus Compute has released Needle 2, a 45M-parameter open model for tool calling and structured extraction. The entire model ships as a single 14MB binary using 28MB of RAM during a full session. Weights are trained at CQ2-bit precision using Cactus Quants and are sealed inside the company's own C++ engine -- no runtime to install and no additional download at inference time. Reported decode throughput is 500 tokens/sec on a Raspberry Pi 5, 400-1,500 tokens/sec on Meta Quest 3S and Apple Vision Pro, and 300-700 tokens/sec on sub-$200 phones. The model uses a "Simple Attention Network" architecture -- 27 layers, 512 hidden width, with a Hadamard MLP replacing the standard FFN, GQA attention, and engram key-value memory from hashed n-gram tables. At 70 MFLOPs per token, it is roughly 8x more compute-efficient than FunctionGemma 270M (540 MFLOPs) and 85x more efficient than Apple FM (~6,000 MFLOPs). The design premise is intentionally narrow: mapping a sentence to a typed function signature requires no world knowledge, which is why 45M parameters is sufficient. Cactus already has a production deployment through Pebble, which runs Needle locally in the Index 01 app for offline voice actions. Prebuilt binaries ship for macOS, Linux (x86-64, ARM64, ARMv7, RISC-V, MIPS32el), Windows, Android, iOS/watchOS/tvOS, and WebAssembly. The model is described as open, though the specific license is not stated in the article. Target verticals include wearables, service robotics, retail kiosks, automotive in-cabin control, and regulated settings where audio cannot leave the device.