What does this equation mean?

The strategy carries two structural limitations that follow directly from how it works, not from any one paper’s shortcoming. First, it requires access to the large model itself, or at minimum to its outputs at sufficient fidelity to train against — a dependency none of the other three strategies share. If the frontier model is API-only, gated, or simply unavailable to the team doing the compressing, the achievable fidelity of “dark knowledge” transfer is bounded by whatever access is actually granted. Second, and more fundamentally, a compressed model is trained to match its teacher, not trained against the underlying task from scratch — whatever the teacher gets systematically wrong, declines to answer, or was never taught, the student is optimized to reproduce rather than to independently discover a better answer. This is a structural consequence of the objective in the equation above, not an empirical finding reported by any single paper cited here: the loss function rewards matching $z_t$ , and has no term that would push the student past it.