Groundbreaking dynamic interactive experience
Genie 3's interactive system realizes three major technological innovations: first, an autoregressive rendering pipeline of 24 frames per second, which maintains screen coherence through a spatial attention mechanism; second, a behavioral response prediction module, which understands commands such as 'turn 30 degrees left' and calculates the subsequent 300 frames; and finally, a scene memory matrix, which uses a neural architecture similar to the hippocampus of the brain. Finally, the scene memory matrix, which uses a neural architecture similar to the hippocampus of the brain, ensures that the scene objects remain in their original state when the user returns after 5 minutes away from the original area (tested to an accuracy of 92%). Compared with Runway and other video generation tools, its interaction delay is controlled within 80ms, reaching game-level experience standards, which is a hardware breakthrough realized by the dedicated tensor processing chip TPUv5.
This answer comes from the articleGenie 3: Generating virtual worlds that can be interacted with in real timeThe
