The post-training phase consists of two key stages — supervised fine-tuning and reinforcement learning — with a total cost of only 5K H800 GPU hours.
- Supervised fine-tuning (SFT) uses high-quality human-annotated conversational data to align the base model, enabling it to better follow human instructions and preferences;
- Reinforcement learning (RL) employs an innovative Group Relative Policy Optimization (GRPO) method, which optimizes the model policy by comparing groups of responses based on relative rankings rather than absolute scores. This approach is more stable and better captures the nuances of human preferences;
- Evaluation results show that DeepSeek-V3-Chat performs comparably to leading closed-source models such as GPT-4o and Claude-3.5-Sonnet on standard benchmarks, and receives high ratings in open-ended evaluations across helpfulness, safety, factual accuracy, and creativity.