MindTopo reveals VLMs’ spatial reasoning abilities

Details

MindTopo evaluates topological reasoning across five categories inspired by cognitive science research on spatial understanding: continuity (paths remaining unbroken), separation (distinct vs. merged structures), order (arrangement along a path), enclosure (inside/outside relationships), and knots (truly knotted vs. tangled loops). The benchmark tests both static recognition and dynamic planning — can the model not only identify a topological property but maintain and manipulate it through a sequence of actions? Current multimodal models show a substantial gap: they perform much better on static recognition than on interactive planning tasks. Failures emerge during planning rather than perception — models lose track of structural relationships as scenes change or propose physically impossible actions that violate topological constraints. A model might correctly identify that an animal is inside a fence in a static image but then propose an exit action that passes through the fence. The findings are directly relevant for robotics and physical AI systems, where reliable operation depends on understanding what stays connected, enclosed, or knotted as the world changes. Standard spatial benchmarks focus on Euclidean properties (distance, direction, size) — MindTopo fills a gap that has been absent from AI evaluation.