This evaluation framework is designed to comprehensively measure the general capabilities of base language models (without alignment fine-tuning), covering seven mainstream benchmarks:
- MMLU (5-shot) and C-Eval (5-shot) for evaluating comprehensive knowledge and reasoning ability
- GSM8K (8-shot) and MATH (4-shot) for testing mathematical problem-solving
- HumanEval (0-shot) and MBPP (0-shot) for measuring code generation capability
- BBH (3-shot) for assessing complex reasoning
Experimental results show that all Qwen-1 model variants (1.8B, 7B, 14B) perform strongly across all downstream tasks. Notably, Qwen-14B surpasses the much larger LLaMA2-70B on three tasks, while Qwen-7B significantly outperforms LLaMA2-13B. Even the smallest Qwen-1.8B can compete with larger models on specific tasks, highlighting its exceptional parameter efficiency. Furthermore, by combining techniques such as NTK-aware interpolation, LogN-Scaling, and hierarchical window assignment, the model effectively maintains performance in long-context settings exceeding 8192 tokens. The overall evaluation robustly validates that Qwen base models possess balanced and powerful core capabilities across different parameter scales.