GPTQ (Generative Pre-trained Transformer Quantization) is an efficient post-training quantization method for large-scale generative pre-trained Transformer models. The method leverages approximate second-order (Hessian) information and introduces three key technical breakthroughs — arbitrary order quantization, lazy batch updates, and Cholesky reformulation — to reduce algorithmic complexity from cubic to a level that can handle models with hundreds of billions of parameters. GPTQ can quantize models with 175 billion parameters (such as OPT-175B and BLOOM-176B) to 3–4 bit weights in approximately 4 GPU hours, with minimal perplexity degradation (e.g., OPT-175B loses only 0.03 perplexity under 4-bit quantization). Compared to prior post-training methods that could only achieve 8-bit quantization, GPTQ more than doubles the compression ratio, enabling — for the first time — a 175B-parameter model to run generative inference on a single NVIDIA A100 GPU, delivering significant end-to-end inference speedups (approximately 3.25x on A100, 4.5x on A6000). The method also maintains reasonable accuracy under extreme quantization (2-bit or even ternary), providing a practical solution for deploying ultra-large language models in resource-constrained environments.