Roblox Introduces ‘Real-Time Dreaming’ for AI World Models

Less than a week after Google unveiled Project Genie, an AI-driven tool that allows users to create interactive 3D experiences from simple prompts, Roblox is revealing its own vision for utilizing AI world models and prompts to empower creators to generate and modify experiences in real time, a concept they refer to as “real-time dreaming.”

Tech giants like Google, Meta, and xAI have shown a growing interest in the potential of AI to assist users in crafting interactive experiences. However, many creators are pushing back against AI tools, with a significant number of game developers expressing concerns that generative AI could harm the gaming industry. Some have opted out of using AI development tools altogether, and companies providing these tools, including Google, have faced copyright infringement lawsuits.

In a virtual briefing, Roblox’s SVP of engineering, Anupam Singh, showcased a glimpse of real-time dreaming through a pre-recorded video. The demo featured an AI-generated Viking-themed world that dynamically responded to prompts, adding elements like a tsunami wave and a boat for the Viking character to board. You can catch a glimpse of this demo in a video featuring Roblox CEO Dave Baszucki.

Roblox’s real-time dreaming is still in the “research stage,” according to Karun Channa, senior director of product, and there’s no set timeline for when these tools will be available. I didn’t get a chance to try real-time dreaming myself, so I can’t speak to its current effectiveness. My experience with Project Genie last week wasn’t particularly impressive, and while Roblox’s demo showcased real-time changes—something Project Genie lacks—I remain skeptical that the ability to modify experiences on the fly will significantly enhance their enjoyment.

“The next frontier of creation on Roblox is the ongoing AI-driven evolution of our platform, enabling creators to generate immersive environments, iterate, debug, and collaborate with their teams using natural language prompts,” Singh stated in a blog post. “If someone can dream it, they should be able to bring it to life.”

However, Roblox doesn’t foresee these AI world model tools completely replacing traditional game development. Singh emphasized that creating a game requires a creative mindset, and “we don’t see a model replacing that creative part.” A similar sentiment was echoed by an executive from Take-Two during an earnings call.

Additionally, Roblox is launching “4D creation” tools that developers can integrate into their games, allowing players to prompt AI to generate interactive objects. These tools, which were previously in closed beta, are now entering open beta, enabling users to create items they can drive, fly, or shoot.

To see what developers can achieve with these tools, check out an experience called Wish Master, which has already implemented them. To use the new “4D” tools, you’ll need to select them from a model picker menu, reminiscent of a traditional AI chatbot.

After a brief period of novelty, I found generating objects in the Wish Master experience to be somewhat lacking in depth. The environment is primarily an expansive open space for players to create and explore, leaving little to do with the objects once they’re made.