The teacher was looking at the future — aligning what teacher and student can see when distilling a streaming video model
To speed up a model that generates video front to back while responding to controls, a few-step student is distilled from a heavier teacher. The teacher, though, had been scoring complete clips. This work fixes the mismatch in which a score handed to the student could rest on frames and instructions that had not yet appeared when it made its move.
Paper overview (our summary)
- Field (arXiv category)cs.CV
- AuthorsHmrishav Bandyopadhyay, Xuanchi Ren, Zijian Huang, et al. (10)
- Submitted2026-08-13
- arXiv ID2608.13391v1
Key points
- The setting is distillation of an interactively controlled video model that generates front to back, into a few-step student for speed.
- A teacher that scored whole clips could hand down a judgment resting on frames and instructions that had not yet appeared when the student made its move.
- The proposal replaces that with a causal teacher blind to the future and initializes the student from the same teacher, giving one formulation from training to inference.
- Scoring is matched to the prefix the student actually rolled out, with perturbation applied to the crude histories of early training.
- Top results are reported on benchmarks covering short sequences and long ones alike, with better obedience to camera instructions that shift over time.
1The constraint of building front to back
Two things are needed at once when a video generation model is to be steered interactively: no waiting, and controls that take effect partway through. In this setting the video is not produced all at once but built from the front. That imposes an obvious constraint. The frame being made now can depend on the past already made and on the control just given. It cannot consult a future that does not exist yet.
2Only the teaching side could see ahead
Distillation for speed copies the behavior of a heavy teacher into a light student. What had been overlooked is that teacher and student were not seeing the same range. The student can only look backward, while the teacher assigns a score after viewing the whole. In effect the student was being marked against a standard it could never reach. Naming that mismatch is the center of this work.
3Score against the context the student is actually in
- 1Make the teacher causalA teacher that does not see future frames or controls
- 2Initialize from the same teacherPut the student's starting point in the same formulation
- 3Score under the real prefixEvaluate each target under the history the student actually generated
- 4Damp the early instabilityPerturb the unreliable histories produced early in training
The second device matches the context used for scoring to the past the student actually built, rather than an idealized one. Student output is not perfect, so evaluating the next step on top of that imperfect history is what keeps the training aligned with how the model is really used. Early in training that history is itself crude, which is why perturbation is applied there to keep things stable.
4Finding the mismatch is the method
The results place it at the top among methods that build front to back, on benchmarks covering short sequences and long ones alike, with markedly better obedience to instructions that shift over time. What deserves attention is that the gain comes less from adding a new component than from making the assumptions of teacher and student agree.
Distillation for speed is already a widely used move, and a mismatch in premises was still sitting inside it.
Why it matters
Distillation for speed is widely used, and mismatched premises can survive inside it. When the teaching side and the learning side have access to different information, the supervising signal becomes a standard that cannot be reproduced. Aligning not only what serves as teacher but when the teacher is allowed to look is a lesson that generalizes across efforts to make models lighter.
FAQ
What is distillation?
Why is a teacher that sees the future a problem?
Is this about adding a new component?
Sources (primary)
Source: arXiv (descriptive metadata is CC0 public domain). Summaries are our own; see arXiv for the original text and PDF.
- arXiv abstract page (original, official)
- PDF (arXiv)
- arXiv ID: 2608.13391