These distilled fashions serve as an attention-grabbing benchmark, showing how far pure supervised wonderful-tuning (SFT) can take a mannequin with out reinforcement studying. As we are able to see, the distilled fashions are noticeably weaker than DeepSeek-R1, but they are surprisingly sturdy relative to DeepSeek-R1-Zero, regardless of being orders of magnitude smaller. 3. Supervised effective-tuning (SFT) plus RL, which led to DeepSeek-R1, DeepSeek’s flagship reasoning model. In actual fact, the SFT data used for this distillation course of is identical dataset that was used to train DeepSeek-R1, as described within the previous part. Interestingly, the results suggest that distillation is much more effective than pure RL for smaller models. The results of this experiment are summarized within the desk beneath, the place QwQ-32B-Preview serves as a reference reasoning mannequin based on Qwen 2.5 32B developed by the Qwen staff (I believe the coaching details had been by no means disclosed). See the results for your self. You'll be able to see varied anchor positions and how surrounding parts dynamically regulate.