GPT-1 adopts a two-stage training approach: first, generative language model pre-training on a large-scale unlabeled text corpus (such as BooksCorpus), followed by supervised fine-tuning for specific tasks. During fine-tuning, the model uses task-specific input transformations to convert structured inputs (such as sentence pairs, question-answer triples) into a single token sequence, thereby minimizing architectural modifications.