Qwen-VL's training pipeline adopts a three-stage progressive strategy to systematically build the model's multimodal capabilities.
Stage 1 pretraining focuses on basic vision-language alignment, using 1.4 billion cleaned web image-text pairs (including both Chinese and English), freezing the language model while optimizing only the visual encoder and adapter, and training for 50,000 steps at 224×224 resolution to endow the model with initial visual understanding.
Stage 2 multi-task pretraining introduces fine-grained annotation data, increases resolution to 448×448 to reduce information loss, unlocks the entire model for end-to-end training, and jointly learns 7 tasks (such as image captioning, visual question answering, grounding, and OCR). Interleaved image-text data and grounding annotations enhance the model's adaptability to complex scenarios including text, charts, and referential understanding.
Stage 3 supervised fine-tuning uses 350K instruction samples to optimize conversational ability, constructing dialogue examples that incorporate grounding and multi-image understanding. The visual encoder is frozen while the language model and adapter are fine-tuned, ultimately yielding the interactive model Qwen-VL-Chat.