The evaluation framework for the Gemma 2 base model is divided into two main parts: academic capability benchmarks and ethics and safety assessments.
On academic benchmarks, the model was systematically tested on a range of widely recognized tasks, including MMLU (multitask language understanding), GSM8K (mathematical reasoning), and HumanEval (code generation). The results show a clear positive correlation between performance and model scale: the 27B model achieves 75.2 on MMLU and 74.0 on GSM8K, setting a new standard for models of its size; the 9B model performs impressively, scoring 71.3 on MMLU and surpassing many open-source models in its class; even the 2B lightweight model demonstrates practical baseline capability. The evaluation also highlights language understanding, knowledge QA, mathematics, and code generation as the model's areas of strength.
On ethics and safety, the model passed rigorous internal red-teaming and met safety policy thresholds across preventing harmful content generation, reducing bias, and controlling memorization risks. Test results were also provided on external safety benchmarks such as BBQ and TruthfulQA. Overall, the evaluation confirms that the Gemma 2 base model achieves an excellent balance between parameter count and performance, offering developers a powerful and efficient foundation model choice.