The core idea of LLaVA is to systematically introduce the successful "instruction tuning" paradigm from pure language models into the vision-language multimodal domain for the first time. The method cleverly addresses the key bottleneck of scarce multimodal instruction data: it uses powerful text-only GPT-4 as a "teacher" that "imagines" image content by reading textual descriptions (such as captions and bounding boxes), thereby automatically generating 158K high-quality vision-language instruction samples covering dialogue, detailed description, and complex reasoning. Based on this data, LLaVA adopts a simple "vision encoder (CLIP-ViT) + linear projection layer + language model (Vicuna)" architecture and, through a two-stage training process (feature alignment pre-training followed by end-to-end instruction tuning), successfully unlocks the model's general-purpose multimodal conversational ability.