AI that "imagines" unseen space to reason — "Astra," a spatial-reasoning agent coupled with a world simulator

This work (Astra) lets an AI reason about space it cannot directly see by imagining it.

While Vision-Language Models (VLMs) show strong visual reasoning, their spatial reasoning is largely constrained to observed images and text-oriented chain-of-thought. With only limited egocentric observations, they struggle to infer unobserved layouts, maintain cross-view consistency, and reason from alternative viewpoints.

The authors study this as thinking with imagination, where a VLM actively acquires imagined visual evidence by interacting with a world simulator during reasoning. Astra couples Astra-VL, an RL-trained VLM policy, with Astra-WM, a Bagel-based world simulator that generates novel-view observations from context images and natural-language camera motions. To provide reliable imagined evidence, Astra-WM is trained with view-consistency tuning to improve pose and content consistency across views. In the RL stage, a world-simulator-in-the-loop two-phase curriculum stabilizes tool-use exploration and advances the model's ability to invoke the simulator only when imagined observations improve over direct answering.

Experiments show both components are necessary: Astra-WM improves simulator-augmented Gemini-3-Flash on MMSI-Bench from 45.1 to 49.5, while Astra-VL improves the Qwen3-VL backbone from 29.8 to 38.8 on MMSI-Bench and from 36.8 to 42.7 on MindCube. Imagined observations can provide useful spatial evidence, but effective world-model-augmented reasoning requires learning when, where, and how to imagine.