These distilled fashions function an fascinating benchmark, exhibiting how far pure supervised high quality-tuning (SFT) can take a model with out reinforcement learning. As we are able to see, the distilled fashions are noticeably weaker than DeepSeek-R1, but they are surprisingly strong relative to DeepSeek-R1-Zero, regardless of being orders of magnitude smaller. 3. Supervised high quality-tuning (SFT) plus RL, which led to DeepSeek-R1, DeepSeek’s flagship reasoning mannequin. In truth, the SFT information used for this distillation process is identical dataset that was used to prepare DeepSeek-R1, as described within the previous part. Interestingly, the results counsel that distillation is much more practical than pure RL for smaller fashions. The outcomes of this experiment are summarized in the desk under, where QwQ-32B-Preview serves as a reference reasoning mannequin based on Qwen 2.5 32B developed by the Qwen team (I think the coaching details had been by no means disclosed). See the outcomes for yourself. You'll be able to see varied anchor positions and the way surrounding parts dynamically adjust.