This week, a new generative AI tool from Google helped me create a bad knockoff of a 3D Nintendo world.
See my version something like this super mario 64: :
I didn’t like it Metroid Prime 4: Beyondbut it’s better than my version metroid prime Experience:
Or how about me? the Legend of Zelda: : breath of the wildComplete with a paraglider (and, briefly, a second link):
All of this was made possible thanks to Project Genie, an experimental research prototype that Google gave me access to this week, although I don’t think I’ve been using it exactly the way Google wanted.
Google DeepMind is putting a lot of effort into building its AI “world” models that can generate virtual interactive spaces with text or images as prompts. The company announced its impressive-looking Genie 3 model last year, but at the time it was only available as a “limited research preview.” Project Genie, which starts today for Google AI Ultra customers in the US, will be the first opportunity for more people to really try out what Genie 3 is capable of.
Google is releasing Project Genie now partly because it wants to see how people use it. “This is really a chance for us to learn about new use cases that we hadn’t thought about,” explains Diego Rivas, product manager at Google DeepMind. The Verge. The company is already excited about how Genie can help visualize scenes for film productions or interactive educational media. If you want, you can take a photo of your children’s favorite toy and use it to inspire the genie-generated world. Genie could also potentially help robots navigate in the real world. But Project Genie is not yet “an end-to-end product that we expect people to use every day,” stressed Shlomi Fruchter, Google DeepMind research director.
With Project Genie, you choose from a set of worlds designed by Google or define prompts for the environments and characters you want to create in your world. After a short wait, the genie first creates a thumbnail, then you can have it create the world. You can explore each generated world for 60 seconds, and each has a resolution of around 720p and a frame rate of around 24fps. When you’re in one, you can (usually) move your character with the WASD keys, jump or move up with a tap of your space bar, and rotate the camera with the arrow keys.
One of Google’s worlds, called “Rollerball,” features a blue sphere in a white, snowy globe, and as you move around, the sphere leaves a trail of paint behind it. As a “game”, Project Genie wasn’t great. There was nothing to do except wander around; There were no objectives or goals. There was no sound. There was disappointing input lag that was worse than what I sometimes experience when cloud gaming. (Some of this may be due to the generally poor Wi-Fi I get in my office.)
During the 60-second experience, the Genie sometimes forgot to show the streak of paint where I had previously rolled. Sometimes, the ball would randomly stop spilling paint. So I began to distrust Genie’s ability to remember what I had already seen with my own eyes.
Another world designed by Google, “Backyard Racetrack”, was a little more fun because it had an actual track to follow. My racing lines were terrible – the input lag didn’t help – but I had fun making turns and trying to stay on the road. However, near the end of the experience, a portion of the track unexpectedly turned into grass, ruining the immersion. And the wheel rims looked really lively.
I had a lot of fun pushing the boundaries of Project Genie to create 3D, AI-generated games with recognizable characters, like me with Super Mario, metroid primeAnd the Legend of Zelda-Themed world. Although they made me laugh, the world doesn’t have a score or anything to strive for, so there’s nothing to do except run or jump. Even if there were specific things to do, the input lag made the world basically unplayable. (Again, this could be a Wi-Fi issue, but I experienced lag even when I was close to my router.)
I couldn’t create everything I wanted. Project Genie will not create the world whose scenario I described Kingdom Hearts – This was my prompt if you’re curious:
Environment
It’s a world filled with Disney characters with a steampunk vibe. Donald and Goofy are your companions. Jack Skellington and Claude Strife are present.
Character
You are a courageous, anime teen with spiky brown hair who wields a key-like blade.
When I removed the characters’ specific names and wrote descriptions in their place, Project Genie produced a thumbnail preview of the world of those characters, who were dead ringers for Sora (the series’ protagonist), Donald, Goofy, Jack Skellington, and Cloud. But when I tried to generate the actual experience, Project Genie blocked me.
I asked why I was able to create worlds with Nintendo characters. “Project Genie is an experimental research prototype designed to follow user-provided signals,” says Rivas. “As with all experiments, we are closely monitoring and listening to users’ feedback.” Rivas also notes that the Genie 3 model was “trained primarily on publicly available data from the Web.” (This probably partly explains why Link deployed his paraglider in my testing, which surprised me. At a high level, the Genie model is constantly trying to predict the next frame, and I’m sure it will many video of people jumping breath of the wild And then sliding forward, from which the model probably learned.) Shortly before publishing this article, Project Genie stopped allowing me to create worlds based on it. super mario 64 Due to “the interests of third-party content providers”.
Considering that Google restricts the ability to generate interactive worlds based on known gaming franchises – I can’t imagine Nintendo would be happy with what I was able to generate! -Project Genie otherwise isn’t that good at this point. The input lag and 60-second limit make them very poor interactive experiences. Sometimes, I can’t control my character at all, only the camera. After the strangeness of the paint splatters and the road turning into grass, I had a general realization that I couldn’t trust the world to stand still from moment to moment.
Project Genie is better than some of the AI-generated worlds I tried last year, but it’s still far worse than a real handmade video game or interactive experience. Fruchter described a possible future where the lines between different types of media are blurred thanks to technology like Genie, but I think there’s still a long way to get there.
Maybe my standards are too high. Project Genie is, after all, an experimental research prototype. And maybe I’ll feel differently as technology improves. But I can’t imagine that people will want to spend an extended period of time jumping into this type of AI-generated world any time soon. With world models, I don’t think we need to worry about the genie coming out of the bottle just yet.
