Meet 𝚆𝚑𝚎𝚛𝚎-𝙾𝙿𝙳: a simple strategy to generate synthetic data and tasks for MLLM on-policy self-distillation to improve visual perception. Counting is a super useful skill to have and giving the teacher hints about where to look makes this work.

1/n 🚀 New paper 𝚆𝚑𝚎𝚛𝚎-𝙾𝙿𝙳 In on-policy self-distillation, the student learns from a privileged version of itself. In recent methods, privilege often comes from a better view of the image. What if, instead of a better view, the teacher knew where to look? 👀

We were in Barcelona for the Summer School on Autonomous Driving! 🚗 The event explored how AI can support safer, smarter mobility in complex real-world environments. It was great to see so much curiosity, technical exchange and energy in Barcelona. 🤝

So, can a car learn to drive by playing against itself, cameras only? In simulation, yes. You don't need to assume perception is solved for self-play to work in driving. Train the way you deploy: from pixels. What would you build with a simulator that renders 2M images/s?

How Alberti performs ✅ Approaches its privileged counterpart in-domain in most metrics ✅ Zero-shot transfer to Waymo Open Motion Dataset layouts re-rendered in Pictura ✅ On those unseen layouts it actually BEATS the privileged agents

With Pictura's throughput we trained Alberti: to our knowledge the first large-scale driving self-play policy learned directly from perspective images, no privileged observations, just plain PPO. Training run: 50B agent steps. Roughly 35 million km of driving.

So we made it cheap. Meet Pictura: a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every single step. Up to 500K agent-steps per second. That's 2M images per second. On ONE H100.

Our fix removes it for real: no teacher, just self-play directly from camera views. Why has nobody done this at scale? Because rendering every agent's camera view at every step is brutally expensive.

The usual patch hides the shortcut rather than removing it. Distillation: train a "privileged" policy with perfect information, then teach a camera-only student to imitate it. Problem: the student copies decisions its own view cannot justify, reacting to cars it cannot see.

Can a car learn to drive by playing against itself, using only its cameras? Self-play driving agents have relied on a shortcut: perfect knowledge of every vehicle's position and speed, even the ones they can't see. A real car never gets that. We take the shortcut away. 🧵