Initially, DeepSeek created their first mannequin with architecture similar to different open models like LLaMA, aiming to outperform benchmarks.