Google today announced several new AI features that it is building into the Android 15 operating system, giving us some insight into what Apple's iPhone operating system will be competing with later this year.
According to Google, Android has been designed with AI at its core, and Gemini is a foundational part of the Android experience that works at the system level. There is an AI powered search bar that can be used for answering queries and looking up information, and the Circle to Search feature can be used for solving math and physics problems or getting homework help as of today.
Gemini serves as the AI assistant on Android instead of Google Assistant, and the Gemini AI will be able to be brought up as an overlay on any app that's in use. Gemini is context aware and can anticipate what the smartphone user is doing and provide context in the moment. It can create images for texts and social media, or answer questions about a video that's being watched. It can parse PDFs, websites, and other content, providing summaries and translating.
Google is using on-device AI for some features in order to keep sensitive data private, and this is also Apple's plan. Multiple rumors have suggested that many of Apple's first AI features will run on-device rather than through a cloud server.
Gemini Nano, Google's on-device AI, currently does things like generate smart replies in messaging apps and summarize voice memos. Gemini Nano with Multimodality is coming to Pixel phones later this year, enabling new on-device AI features like TalkBack, which can help blind and low-vision users better interpret the world around them. Google is also developing a call screening feature that actively listens to calls and lets users know if it sounds like a scam.
Google today also announced Gemini 1.5 Flash, a lightweight version of its Gemini Pro AI model. Gemini Flash is more efficient, less expensive, and has lower latency, which makes it ideal for large scale rollouts. Google says that Gemini 1.5 Flash is able to deliver "impressive quality" for its size, excelling at summarization, chat applications, image and video captioning, and data extraction from long documents.
Gemini 1.5 Pro has also been improved and it is able to follow increasingly complex and nuanced instructions, including parsing documents up to 1,500 pages long or an hour of video content. Gemini can be interacted with in Google Messages, and Gemini Advanced subscribers will be able to experience a new mobile conversational experience that makes speaking with Gemini more intuitive. Customized versions of Gemini called Gems are also available to subscribers, for personalized assistants that can help with workouts, provide recipes, and more, with specific personalities.
Project Astra, another Google effort, focuses on AI agents that can be helpful in everyday life. Google demoed prototype agents able to identify objects, provide context on information from the camera, and recall a timeline of events for finding a lost item.
Google plans to provide more information on new features coming to Android 15 later this week.
