• by Colin Kirkland , June 19, 2024

Snap is testing a real-time, on-device image diffusion model that can generate enhanced augmented reality (AR) experiences on mobile devices. The social-media company unveiled an early version of the model, along with other AR-focused generative-AI tools, at the Augmented World Expo. 

Notably, the model is small enough to run on a smartphone and fast enough to re-render text-prompted image frames in real-time, according to Snap co-founder and CTO Bobby Murphy. 

Snap has made a business off of fusing AR experiences into its popular social media platform Snapchat. With the rising interest in generative-AI image creation, which Snap has taken advantage of through various Lens features, it makes sense that the company would want to embed the technology into its suite of AR tools.

Murphy explained during his presentation on Tuesday that Snap teams have been working to speed up generative-AI image diffusion models, which must run much faster to make an impact on augmented reality experiences opposed to image creation. 

“This and future real time on device generative ML (machine learning) models speak to an exciting new direction for augmented reality, and is giving us space to reconsider how we imagine rendering and creating AR experiences altogether,” Murphy said.

Snapchat’s user base will begin seeing Lenses with the new generative model over the coming months, while Snap says it wants to bring the feature to creators by 2025. 

In addition, Murphy showcased the launch of Lens Studio 5.0 for developers, who will gain access to a new list of generative-AI tools meant to accelerate the creation of AR effects, like selfie Lenses with realistic machine learning face effects, cutting out weeks, even months of work. 

Creators should also be able to generate custom stylization effects and apply them to a real-time transformation over a user’s face, body and surroundings, generate a 3D asset in minutes to include in Lenses, as well as fantastical characters, face masks, texture and materials via a text or image prompt using Snap’s Face Mesh technology. 

To help AR creators make use of these new tools, Snap included a an AI assistant that answers questions in the latest version of Lens Studio.