Google DeepMind’s Newest AI Agent Demonstrates Ability to Play 3D Games
Artificial Intelligence (AI) models that specialize in one game and always play to win are now old news. Google Deepmind has taken another shot at creating a new model designed to play a number of 3D games like a human. Not only that, it also understands and acts according to the player’s instructions. Google DeepMind recently introduced its AI-based virtual gaming partner SIMA, which plays just like professional gamers by responding to instructions and completing tasks accordingly.
The ability of an AI agent is not limited to linear games where you have to follow a certain path or story; it is also available in open world settings.
Google DeepMind: Virtual gaming companion
SIMA stands for “Scalable, Instructable, Multiworld Agent” and has been trained on several video games and essentially understands natural language commands related to the 3D game world and image recognition to perform tasks efficiently.
According to Google’s blog post, the company has partnered with several game developers to train SIMA for several video games, including Hello Games’ No Man’s Sky, Tuxedo Labs, Valheim’s Teardown, and Goat Simulator 2, among others. Furthermore, it has translated the capabilities of advanced AI models into useful, real-world actions through a language interface.
“SIMA is an artificial intelligence agent that can perceive and understand different environments and take action to achieve a guided goal. It includes a model designed to accurately map imagery and a video model to predict what will happen next on the screen,” it said, adding that the AI agent does not need access to the game’s source code or custom APIs, but only requires on-screen images and simple, natural language instructions from the user.
“SIMA uses keyboard and mouse outputs to control the games central character to carry out these instructions. This simple interface is what people use, meaning SIMA can potentially interact with any virtual environment,” the post adds.
SIMA is currently assessed on 600 basic skills, including navigation, object interaction and menu use. Speaking about the same at a media event, DeepMind researcher Tim Harley shared that the AI agent is trained to do what the game tells it to do. “We want our future agents to perform tasks that require high-level strategic planning and multitasking,” he added.
