Results (UNIC protocol: ImageNet cls, transfer, segmentation, depth): ✅ IDeaL > Dead Leaves in every setting, every task ✅ Students beat the weakest teacher on 3/4 tasks — even with just 1K samples ✅ At 1K images, IDeaL matches or BEATS a 1K ImageNet subset on classification

Image decorrelation (L_ID): same trick at the image level. Final-layer CLS embeddings across the batch → minimize pairwise similarity → every generated image is globally distinct, as perceived by each teacher. No labels, no dataset statistics, no privileged info. Just the frozen teachers.

How do we optimize? Two decorrelation losses, one idea: push representations apart. Patch decorrelation (L_PD): per layer, compute pairwise cosine similarity between patch representations, push off-diagonals to 0. Diverse patches = self-attention has work to do.

Our key idea: the teachers themselves know what makes an image informative. So we treat pixels as learnable parameters and optimize Dead Leaves through the frozen teachers. The result: IDeaL samples that don't look real, but are maximally informative.

Multi-teacher distillation (e.g. UNIC) combines complementary teachers (e.g. DINO, iBOT, DeiT-3, dBOT) into one student. But it assumes access to the teachers' training data. What if that data is private, licensed, or just gone? We asked: how far can we get without it?

🧐 Can you distill knowledge from 4 vision teachers into one student using ZERO real images? 🥳 Turns out: YES, and surprisingly well. 📣 Our ECCV 2026 paper "IDeaL" closes most of the gap with real-image distillation, using optimized structured noise.