Google has unveiled Sign-Language-to-Text, or SL2T, an AI model that converts sign language into written text. The model powers new features in Gboard and Live Transcribe on the Pixel 11 series. Users can sign into the phone’s camera instead of typing for web searches, text messages, emails, documents or Gemini queries. In face-to-face settings, Live Transcribe lets users sign responses rather than type them. The system currently supports American Sign Language to English. Google trained SL2T on more than 100,000 hours of data covering over 50 sign languages, with ASL making up about a quarter of the data. An on-device MediaPipe Holistic model tracks body landmarks and sends only geometric coordinates to the server, so raw video is not uploaded. The model translates the landmark sequence directly into text without intermediate gloss annotations. On the FLEURS-ASL benchmark it scores 70 BLEURT in a zero-shot setting, which Google describes as the highest result to date.
Google has unveiled Sign-Language-to-Text, or SL2T, an AI model that converts sign language into written text. The model powers new features in Gboard and Live Transcribe on the Pixel 11 series. Users can sign into the phone’s camera instead of typing for web searches, text messages, emails, documents or Gemini queries. In face-to-face settings, Live Transcribe lets users sign responses rather than type them. The system currently supports American Sign Language to English. Google trained SL2T on more than 100,000 hours of data covering over 50 sign languages, with ASL making up about a quarter of the data. An on-device MediaPipe Holistic model tracks body landmarks and sends only geometric coordinates to the server, so raw video is not uploaded. The model translates the landmark sequence directly into text without intermediate gloss annotations. On the FLEURS-ASL benchmark it scores 70 BLEURT in a zero-shot setting, which Google describes as the highest result to date.