Putting sign language AI into users’ hands

Details

Google DeepMind's sign-language-to-text (SL2T) model is the first sign language AI to ship inside a mainstream consumer product. It powers two Pixel 11 features: sign-to-text dictation in Gboard (replacing typing across any app — web search, messaging, Gemini queries) and Live Transcribe (enabling signed responses in conversations). The initial deployment supports American Sign Language to English, with more languages planned.\n\nDeepMind frames SL2T as solving two distinct technical challenges that separate it from speech recognition. First, sign languages are independent natural languages with their own grammars and lexicons — they require true machine translation, not a sequential sign-to-word mapping. Second, the model must process continuous visual input across multiple channels: hand shapes, movement trajectories, and non-manual markers like facial expressions and mouth movements, all of which carry grammatical meaning in ASL. This is a fundamentally harder perception problem than sequential audio transcription.\n\nThe model is described as massively multilingual despite the ASL-first rollout, suggesting the architecture was designed for multi-language coverage from the start. DeepMind notes that sign language AI progress has been slow due to both technical complexity and widespread misconceptions about sign language structure. Roughly 70 million people globally use one of 200+ sign languages as their primary language — a population that prior AI speech advances have not served.