Byeongho Heo, Jaehui Hwang, Sangdoo Yun +1 more
When you train a smaller model to imitate a bigger teacher, the usual approach is to have the student copy the teacher's full output distribution token by token. This paper asks whether there is a cleaner signal to copy. Their idea, the delta signal, is the difference between the teacher and the base model it started from before it was tuned for reasoning. That difference isolates exactly what the reasoning tuning added, so the student picks up the reasoning changes instead of everything the teacher happens to output.
They call the method On-Policy Delta Distillation, and across math, science, and code benchmarks it beats standard on-policy distillation, letting reasoning models reach strong scores after only a short bout of post-training.
This is my summary of the abstract, so check the paper for the exact setup and where the delta signal helps most.
On-policy distillation is an alternative post-training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token-level supervision from a teacher model. Although on-policy distillation has been studied and applied across various settings, its fundamental design remains underexplored. In this paper, we introduce a new distillation reward, termed the delta signal, instead of directly imitating the teacher's output…
PRISA: Proactive Infrastructure LiDAR Framework for Intersection Safety Assessment
arXiv (cs.CV) · July 17, 2026CLIFE: Camera-LiDAR Fusion Framework for Edge-Deployable Roadside VRU Perception
arXiv (cs.CV) · July 17, 2026VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds
arXiv (cs.LG) · July 17, 2026Learning Standard Model structure from LHC data with Riemannian flow matching