Measuring whether models can reason about crashes from dashcam video — AUTOPILOT-VQA (CVPR 2026)

This work (AUTOPILOT-VQA) provides a benchmark for whether autonomous-driving AI can correctly reason about situations that lead to crashes.

1What autonomous-driving benchmarks were missing

Recent advances in Vision-Language Models (VLMs), LLMs, and multimodal LLMs have improved autonomous-driving tasks such as scene understanding, decision making, trajectory prediction, and visual question answering. Yet evaluating whether these models can reliably reason about safety-critical incidents remains a difficult, open challenge.

2An incident-centric VQA benchmark

To close this gap, the authors present AUTOPILOT-VQA, an incident-centric VQA benchmark. Targeting dashcam video understanding, it evaluates different systems through structured questions designed around real-world driving incidents and near-incidents (near-misses).

The safety-relevant categories it covers are broad — weather and lighting conditions, traffic environment, road layout, road-surface state, signage, involved entities, accident occurrence, impact location, and avoidability-related reasoning.

3Requiring grounded answers

By requiring models to answer grounded questions about both contextual scene properties and event-level incident details, AUTOPILOT-VQA moves beyond mere object recognition toward temporally grounded, safety-aware reasoning. The dataset is released as part of the AUTOPILOT CVPR 2026 competition, providing a standardized benchmark for assessing the reliability of autonomous-driving systems across diverse scenarios.