Microsoft Xbox Unveils an Ambitious AI Model for ‘Gameplay Ideation’

Yesterday, Microsoft Xbox took a significant step by unveiling Muse, a fascinating generative AI model created to spark creativity in game design. This was accompanied by an insightful article on Nature.com, a detailed blog post, and a YouTube video for an engaging visual experience. You might wonder what “gameplay ideation” involves. According to Microsoft, it refers to creating “game visuals, controller actions, or both,” though in practice, its capabilities are quite limited and certainly don’t replace the traditional game development process.

Despite these limitations, there’s some intrigue in the data. To train Muse, they utilized a substantial number of H100 GPUs, requiring approximately one million updates just to stretch one second of real gameplay into an extra nine seconds of dynamic, engine-accurate simulated gameplay. Most of this training data was sourced from existing multiplayer game sessions.

Instead of simulating the game on a standard PC, Microsoft had to rely on a cluster of 100 Nvidia H100 GPUs. This method is far costlier and more power-intensive, yet it only manages to produce a modest resolution of 300×180 pixels for those additional nine seconds of extrapolated gameplay.

One of the most captivating aspects of Muse was its ability to duplicate props and enemies within the game’s environment and mimic their functionality. But it raises the question: why invest so heavily in hardware, energy consumption, and AI training when developers could simply use existing tools to spawn elements in games?

Even though Muse impressively managed to preserve object permanence and replicate the original game’s behavior, the outcome seems rather extravagant compared to conventional game development approaches, which are already quite efficient.

The potential future advancements of Muse might offer more compelling capabilities, yet, for now, it joins a long list of projects aimed at simulating gameplay purely through AI. While it’s encouraging to see some level of engine accuracy and object permanence retained, this method is hardly a practical approach for developing, testing, or playing video games. After thoroughly examining the material, it’s puzzling why anyone would choose this route, considering its sub-optimal nature.