Do you know? Video games can record a 3D environment alongside the actions a player takes inside it. A British startup believes those paired visual and control signals could help train AI systems designed to navigate the physical world.
What is changing?
Worldmodeldata is building a data-brokering business around video game information. The startup aims to package controller inputs and other data collected by game studios into training datasets for so-called world models.
World models are a class of AI intended to represent aspects of the real world and predict how actions affect an environment. Researchers cited by WIRED say these systems need both visual information and action data to learn real-world physics and movement.
That need has created a data problem. Unlike large language models, which can be trained on vast amounts of text, world-model developers do not have an equivalent internet-scale supply of cause-and-effect data involving physical actions.
Who is involved?
- Worldmodeldata: The British startup is advised by Yann LeCun and is positioning itself as an intermediary between game studios and AI laboratories.
- Rhea Loucas: The company’s CEO argues that the variety and scale of video game experiences could provide useful training material.
- University of Surrey: Associate professor Xiatuan Zhu says world models require cause-and-consequence data that is scarce online.
- Nvidia: The company uses a custom engine designed to replicate real-world physics for its world-model work and is more cautious about using game inputs for precise manipulation.
- Other companies: General Intuition and Niantic are already collecting video game data from their own platforms to build models, according to the report.
How could game data help?
Video game environments can provide visual representations of 3D spaces paired with player actions. That combination may offer large quantities of varied examples, including unusual or unexpected situations that are difficult to capture through manually supervised robotics experiments.
Worldmodeldata says it has licensed almost 1 million hours of data from studios behind various popular video games. Loucas declined to identify those studios. The company also hopes to create ways for individual players to be compensated in the future.
| Potential advantage | Reported limitation |
|---|---|
| Large quantities of recorded gameplay | Game physics can be coarse or eccentric |
| Visual scenes paired with player actions | Inputs may not capture fine motor control |
| Varied environments and edge cases | Game shortcuts can imitate realism without modeling physical detail |
Why researchers remain cautious
The central idea has not yet been fully tested. The report says some researchers expect world-model performance to improve as training datasets grow, but the usefulness of game data for physical-world tasks remains disputed.
Ming-Yu Liu, who leads world-model development at Nvidia, says game-trained models are unlikely to perform well on tasks requiring fine-grained motor control, such as carefully manipulating objects. Video games may make an action look realistic while omitting details such as the pressure applied by individual fingers.
Zhu similarly describes games as approximate simulators with some physical grounding, rather than precise representations of the real world. Nvidia’s view is that game data may be better suited to generating hyperrealistic video or 3D environments than to controlling physical objects.
What happens next?
Worldmodeldata plans to continue licensing and organizing game data for AI laboratories. Loucas believes video game data could eventually make up most of the training material for world models, followed by task-specific data from real-world environments.
That outcome is not established. Researchers and investors cited in the report say multiple approaches remain under consideration, and it is still unclear which path will work best.
FAQ
What are world models?
They are AI models focused on representing environments and learning relationships between observations, actions and consequences.
Why use video game data?
Games can generate visual information about 3D spaces together with the actions taken by players, and they are available in large quantities.
Can game data directly teach robots to handle objects?
Researchers quoted in the report are skeptical. They say game physics often omit the detailed physical interactions needed for precise manipulation.
Has the approach been proven?
No. The report says the hypothesis has not yet been fully tested, and the field has not settled on the best way to train world models.
Bottom Line
Video game data could give world-model developers a large source of visual and action-based training examples. But game environments are simplified simulations, and whether they can reliably prepare AI for the precision and unpredictability of the physical world remains an open question.
Source
This report is based on information published by WIRED.
