The model architecture consists of two core components: an image encoder and a text encoder, which jointly learn to map images and text into a shared multimodal embedding space. The image encoder can use either a ResNet or Vision Transformer (ViT) architecture to extract image features — for example, ResNet variants produce feature maps from input images, while ViT splits images into patch sequences processed through Transformer layers. The text encoder is based on a Transformer model and encodes text descriptions (such as natural language sentences) into vector representations. During training, the model is optimized via a contrastive learning objective (e.g., InfoNCE loss), maximizing the similarity of matched image-text pairs in the embedding space while minimizing the similarity of mismatched pairs. This enables zero-shot transfer without requiring task-specific labels. This design allows CLIP to directly perform image classification or retrieval based on text prompts, enhancing its generalization ability.