Google wants video AI to understand what people need as they work
| Source: THE DECODER
Tags: Google, StreamVibe, StreamVibe-VL, Qwen3.5-9B, Gemini 2.5 Flash, Video Agents, Reinforcement Learning
A Google-led team introduced StreamVibe, a video agent meant to speak up only when a user needs help, and its benchmark shows no tested model reliably picks the right moment in longer videos.
Details
Researchers from Google, Stanford, Oxford and several U.S. universities built StreamVibe, a video assistant that watches tasks like cooking or furniture assembly and decides on its own when to offer help, including for vague requests such as "help me if I run into difficulties." They assembled a benchmark of about 4,100 examples from more than 183 hours of everyday and software recordings. Gemini 2.5 Flash proposed when and what to say, and four human evaluators had to unanimously approve each example. The test covers explicit instructions, vague goals and requests that change midway. The team trained StreamVibe-VL, reportedly based on Alibaba's Qwen3.5-9B, to decide every second whether to stay quiet or speak, then used reinforcement learning to make it less interruptive. A separate memory system tracks pending requests. The headline result is sobering: in longer videos, errors compound and response quality drops sharply for all tested models, so proactive video assistants remain an open problem.