Post-training data is critical to determining the usefulness and behavior of Llama 3 models. Its construction revolves around preference data and supervised fine-tuning data. Preference data is generated by deploying multiple different models after each post-training round, sampling two responses for the same user prompt from different models to increase diversity, with annotators providing a four-level preference strength rating and an editing step that allows annotators to further improve the chosen response, resulting in some data containing three explicitly ranked responses: "edited > chosen > rejected." Compared to Llama 2, the average length of both prompts and responses has increased, indicating that the training tasks are more complex. Supervised fine-tuning data is a mixture of rejection-sampled data (generated by using reward models to sample from human-annotated prompts and then filtering), synthetic data, and other sources, subjected to rigorous quality control and cleaning to cover areas such as knowledge-based QA, precise instruction following, reasoning, coding, multilingual tasks, and tool use.