These distilled models function an fascinating benchmark, exhibiting how far pure supervised effective-tuning (SFT) can take a model without reinforcement studying. As we will see, the distilled models are noticeably weaker than DeepSeek-R1, however they are surprisingly strong relative to DeepSeek-R1-Zero, regardless of being orders of magnitude smaller. 3. Supervised tremendous-tuning (SFT) plus RL, which led to DeepSeek-R1, DeepSeek’s flagship reasoning model. Actually, the SFT knowledge used for this distillation course of is identical dataset that was used to practice DeepSeek-R1, as described within the earlier part. Interestingly, the results recommend that distillation is far more practical than pure RL for smaller fashions. The results of this experiment are summarized in the table beneath, where QwQ-32B-Preview serves as a reference reasoning model based mostly on Qwen 2.5 32B developed by the Qwen workforce (I think the coaching details have been by no means disclosed). See the results for yourself. You possibly can see numerous anchor positions and the way surrounding elements dynamically adjust.