Google Weaves Gemini Into Input: Pointer, Speech, Gboard

Google announced Rambler, a Gemini-powered AI dictation feature integrated into Gboard that can transcribe speech, remove filler words, handle on-the-fly corrections, and recognize code-switching within a single sentence without losing context. Rambler runs with a mix of on-device and cloud processing; Google says it does not store raw audio and will clearly notify users when the feature is active. Initially launching this summer on Samsung Galaxy and Google Pixel phones, the capability will later roll out to other Android devices. The addition aims to make voice input more natural and useful across apps while addressing privacy concerns through engineering investments. Key players: Google, Gemini, Gboard.

Google researchers showcased an experimental AI-enabled mouse pointer powered by Gemini that understands both what the user points at and why it matters, aiming to eliminate “AI detours.” The prototype captures visual and semantic context across apps, letting users point and speak shorthand commands like “Fix this” or “Show me directions,” and converting pixels into actionable entities (places, dates, objects). Four interaction principles — maintain the flow, show and tell, embrace shorthand gestures, and turn pixels into entities — guide the design to let AI meet users in-place across documents, images, maps, and code. This could reshape UI patterns and streamline human–AI collaboration across desktop workflows.