Roundup: All the Google AI-powered software features launching with Pixel 3

Abner Li | Oct 10 2018 - 1:20 pm PT

As we noted last night, the Google Pixel 3 and Pixel 3 XL deliver delightful hardware that is noticeably more refined than last year. Beyond the physical devices, a key part of Made by Google is the “AI + software” experience, with the Pixel 3 debuting a handful of new Google AI-powered features at launch.

Gmail Smart Compose

Announced at I/O 2018, Smart Compose takes Smart Reply a step further by writing out entire emails for you. Limited to desktop, and recently just becoming available for G Suite enterprise users, Smart Compose is now coming to mobile.

Available first on the Google Pixel 3, users will be able to perform a left-to-right swipe in order to populate a Smart Compose suggestion. Like on the web, text appears in fainter text and updates as users write out their message.

Smart Compose will be available on more mobile devices starting next year, with it already available globally in English. In the coming months, it will support Spanish, French, Italian and Portuguese.

Google Lens

With Lens Suggestions in Google Camera, Google’s visual search feature becomes more automated, and doesn’t require users to manually activate. Hovering over an address, number, or email address will provide relevant actions. Meanwhile, the full Lens experience can now be launched directly by long pressing in the camera app.

Elsewhere, with Lens already available from the system Share menu, Google is making Lens accessible from the Recent apps screen. Long pressing on an image will bring up a toolbar to launch Lens or share.

Playground

Framed as allowing users to add “stories” to their pictures and videos, AR Stickers has been rebranded to Playground. Announced last year alongside ARCore, Google redesigned the interface and is adding more sticker packs as well as licensed characters.

The lighter UI is somewhat reminiscent to Snapchat with a capture button flanked by shortcuts to open the stickers panel and add speech bubbles. Playground also uses AI to recommend the right stickers for the moment with suggestions appearing right above the shutter.

Last year, Google featured Stranger Things and Star Wars. It is now adding Avengers characters from Marvel. In an upgrade from just interacting with each other, these stickers can now respond to your actions and facial expressions. For example, if you perform a thumbs up, Iron Man will follow suit. Additionally, Google is adding a new Childish Gambino pack that is “only on Pixel.”

Flip to Shhh

Announced at I/O 2018 as part of Digital Wellbeing, users will be able to lay their Pixel flat on a surface to automatically mute sounds and notifications.

Google Duplex

Announced to much criticism at I/O, Google Duplex is in fact real and launching first on Pixel phones. Starting next month, user will be able to book a table at a restaurant as part of city-by-city availability in New York, Atlanta, Phoenix, and San Francisco. Initially, food-based reservations appear to be the only capability of Duplex.

Call Screen

Another standout feature on the Pixel 3 is “Call Screen” in the Google Phone app. When users receive an incoming call, they will have the option to tap the “Screen call” button to have Google Assistant pick up on their behalf.

The Pixel will note how the caller is currently talking to a “screening service from Google” and asks for a name and reason for calling. You will see a transcript of both sides of the conversation and offered suggestion chips to dictate how Assistant replies: Who is this, I’ll call you back, Report as spam, and Tell me more.

These suggestions will update in real-time according to the call, with the Report as spam option telling the caller to remove your number from a dialing list. This functionality is powered entirely with on-device machine learning and launching on the Pixel 3, before coming to older Pixel devices.