Új tréningmódszert mutatott be az Nvidia és a Perplexity

Perplexity , along with NVIDIA team, has introduced JOLT, a new post-training method for language models. JOLT stands for Joint On-policy Learning and Teaching . It trains a single model in 2 roles. The teacher sees privileged information; the student does not. The key change is that the teacher is trained too, so its feedback stays usable. On MATH, JOLT matched GRPO’s accuracy with 6.5x fewer processed training tokens. For teams running RL post-training, that is a direct compute question. TL;DR What it is: a post-training method. Size: tested on Qwen3-0.6B, Qwen3-4B-Instruct, Qwen3-8B, Qwen3-30B-A3B-Instruct and DeepSeek-V2-Lite. Runs on: 1 shared model plays both roles, so no separate teacher model. Training hardware: Not disclosed. Performance: JOLT+ posts the highest mean score in every setting tested against GRPO, OPSD and π-Distill. Best: AppWorld state-test pass rate of 72.09% with JOLT+, vs 62.20% for GRPO (+9.9 points). Worst: JOLT without student reward scores 51.92% on AppWorld, below GRPO. What is Perplexity JOLT? JOLT is a self-distillation method from Perplexity and NVIDIA that trains one model as teacher and student. It builds on On-Policy Self-Distillation (OPSD) . Why does RL post-training need a better teacher? Outcome-reward RL, such as GRPO , gives one reward per attempt. That signal is sparse. It does not say which tokens helped. On long agentic tasks, successes are rare, so learning is slow. On-Policy Distillation fixes sparsity by having a stronger teacher score each student token. OPSD removes the separate teacher. The same model becomes the teacher when given privileged context, such as a verified solution. The research team argues this is not enough. A teacher holding the answer can take shortcuts the student cannot follow. Its guidance may sit beyond the student’s reach. In the paper, OPSD’s teacher-student disagreement grows during MATH training. How does JOLT work? JOLT trains both roles of one shared model at the same time: Teacher: gets privileged information. It learns from outcome rewards, plus a KL penalty that keeps it close to the student. Student: sees only the task. It learns from the teacher’s token-level distillation signal on its own attempts. On-policy: each role learns from trajectories generated under its own context. JOLT+: adds the student’s own outcome reward to the student update. The research stands in the center on a ‘reward-consistent’ teacher. Its distillation update should point the same way as the student’s reward gradient. The ideal teacher favors high-advantage actions while staying near the student’s policy. How does JOLT perform on benchmarks? The research team tested 5 settings, each with a different base model and type of privileged information: GSM8K: DeepSeek-V2-Lite; teacher gets the verified answer. MATH: Qwen3-0.6B; teacher gets worked reference solutions. LiveCodeBench v6: Qwen3-8B on 131 problems; teacher gets a passing sibling solution or execution feedback. AppWorld: Qwen3-4B-Instruct; teacher gets summaries of past attempts. Terminal-Bench 2.1: Qwen3-30B-A3B-Instruct trained on TMax ; teacher gets private verifier files. JOLT+ had the highest mean score in every setting. Against GRPO, the strongest baseline, it gained 1.5 points on GSM8K. It also gained 4.2 on MATH, 3.0 on LCBv6 and 9.9 on AppWorld. On Terminal-Bench 2.1 , JOLT+ reached 10.96% pass@1 and 34.83% pass@32. GRPO scored 8.25% and 30.34%. Even plain JOLT, with no student reward, beat GRPO on MATH: 72.14% versus 68.36%. How much training compute does JOLT save? On MATH, the research team fit accuracy curves against training-token budgets. At 63.44% accuracy, GRPO’s checkpoint used 8.40M processed training tokens. JOLT’s fitted curve needed 1.30M, a 6.5x reduction. Counting completion tokens only, the gap widens to about 13x. Across 55% to 65% accuracy targets, savings range from 4.3x to 7.4x on processed tokens. JOLT also leve