ViLT's pretraining model adopts a minimalist design. Its core innovation is completely discarding the computationally intensive visual embedders (e.g., convolutional neural networks or object detectors) that traditional vision-language models rely on, and instead adopting a lightweight, symmetric approach to text embedding: images are split into patches and embedded as vector sequences via linear projection, then concatenated with text tokens and fed into a unified single-stream Transformer (initialized from ViT) for deep cross-modal interaction. The model is pretrained with Image-Text Matching and Masked Language Modeling (using whole word masking to strengthen multimodal learning) objectives, combined with image augmentation techniques. The result is competitive performance across a variety of downstream tasks while achieving orders-of-magnitude efficiency gains, demonstrating that effective cross-modal representations can be learned without a complex visual encoder.