ORPO (Odds Ratio Preference Optimization) is an innovative single-stage preference alignment method that achieves reference-model-free preference optimization by directly integrating an odds ratio penalty term into the supervised fine-tuning (SFT) negative log-likelihood loss. While traditional SFT indiscriminately increases the generation probability of both preferred and rejected responses, ORPO introduces the odds ratio loss LOR=−logσ(log(odds(yw∣x)/odds(yl∣x))) to dynamically penalize rejected responses, thereby simultaneously accomplishing domain adaptation and preference alignment in a single training stage. Experiments demonstrate ORPO's strong performance across multiple benchmarks: using only the UltraFeedback dataset for a single training round, Mistral-ORPO-β (7B) achieves 12.20% on AlpacaEval 2.0, 66.19% on IFEval instruction-level loose accuracy, and 7.32 on MT-Bench, outperforming larger models including Zephyr-β and Llama-2-Chat (13B). ORPO not only simplifies the training pipeline (no SFT warm-up phase, no reference model) but also significantly reduces computational resource requirements (approximately 50% training time reduction vs. DPO), offering a new paradigm for efficient model alignment in resource-constrained settings.