Everyone talks about speed. Nobody talks about the quality cliff at INT4.
I have spent the last eight months running quantized models in production — customer-facing APIs, internal tools, batch inference pipelines. I have watched GPTQ turn a coherent legal summarizer into a rambling mess. I have seen AWQ rescue a model that GPTQ destroyed. And I have learned that the benchmarks people cite online have almost nothing to do with what you will see when you actually use these models.
This is not a theoretical overview. This is what happens when you quantize real weights, serve real traffic, and measure real output quality.
What Quantization Actually Does
If you already know what quantization is, skip this section. If you do not, here is the one-paragraph version.
Neural network weights are stored as floating-point numbers. A full-precision model uses 32 bits per weight (FP32). Most production models today ship at 16 bits per weight (FP16 or BF16). Quantization compresses those weights into lower-precision formats — 8-bit integers (INT8), 4-bit integers (INT4), or custom formats like FP8. The goal is simple: smaller models use less VRAM, load faster, and run more tokens per second. The catch is that you are literally throwing away information. Some of that information matters.
The part nobody tells you: not all weights are equally important. A quantization method that treats every layer the same will destroy the layers that matter most. That is why two different 4-bit quantization methods can produce wildly different results on the same model.
FP16: The "Correct" Answer
If you have the VRAM, run FP16. Full stop.
FP16 is not technically "lossless" — you are still down from FP32 — but for modern transformer models, the difference between FP32 and FP16 is negligible in practice. I have never seen a case where FP16 output was meaningfully worse than FP32 output on the same model. The perplexity gap is typically under 0.1%, and human evaluators cannot tell the difference.
The problem is cost. A 70B parameter model at FP16 needs about 140GB of VRAM just for the weights. Add KV cache, activations, and overhead, and you are looking at 160-180GB for reasonable batch sizes. That is four RTX 4090s in tensor parallelism, or two A100 80GBs, or one H100 if you are lucky. Most people do not have that hardware sitting around.
So we quantize. And that is where the trouble starts.
INT8: When It Works, When It Does Not
INT8 is the conservative choice. You cut your VRAM usage in half compared to FP16, and the quality loss is usually small enough that you will not notice it in casual use.
Here is where INT8 shines: dense factual retrieval, classification tasks, straightforward summarization of well-structured documents. I run INT8 versions of Qwen2.5-72B and Llama-3.3-70B for internal document search, and the hit rate on correct answers is within 2% of FP16. For a tool that saves me 80GB of VRAM, that is an easy trade.
Here is where INT8 falls apart: reasoning, coding, and anything requiring precise symbolic manipulation. I first noticed this when I quantized DeepSeek-Coder-V2 to INT8. The model could still write basic Python, but it started making subtle errors in type annotations, misplacing closing brackets in nested structures, and hallucinating function signatures that did not exist in the context. These were not dramatic failures — the code looked plausible — which made them dangerous. A model that generates broken code confidently is worse than a model that admits it does not know.
Another INT8 weakness: long-context coherence. At 16K+ tokens, INT8 models start losing thread. Characters in roleplay scenarios forget their motivations. Legal summaries start contradicting earlier sections. The error accumulates because the compressed attention patterns are slightly noisier, and over thousands of tokens that noise compounds.
My rule: INT8 is fine for retrieval and classification. I do not use it for generation tasks longer than 2K tokens, and I never use it for code.
GPTQ: The Trade-Off Explained
GPTQ was the first widely adopted post-training quantization method that made 4-bit models actually usable. It uses layer-wise quantization with approximate second-order information to minimize the error introduced by compressing each layer. In theory, this should preserve quality better than naive rounding. In practice, it depends heavily on the model architecture.
Let me be specific about what goes wrong.
I tested GPTQ on Mistral-Large-Instruct-2407 at 4-bit. The model is excellent at FP16 — coherent, nuanced, good at following complex instructions. The GPTQ version, using the standard 128-group-size configuration from TheBloke's repository, was a different animal. It could still answer simple questions correctly. But when I gave it a multi-step reasoning prompt — "Analyze these three contracts, identify conflicting clauses, and suggest resolution language" — the output degraded noticeably.
Specific failures I logged:
- Repetition loops: The model would get stuck repeating the same sentence with minor variations. This happened in about 8% of long-generation prompts.
- Loss of negation: The model would drop "not" from instructions. I asked it to "list the provisions that do not apply to subcontractors," and it listed the ones that do apply. This is a known GPTQ failure mode on certain architectures.
- Numerical drift: In math problems, the model would make arithmetic errors that FP16 never made. Not complex math — simple addition in word problems.
The frustrating part is that GPTQ works well on some models. Llama-3.1-70B-GPTQ is surprisingly solid. The architecture is robust to quantization, and the attention patterns seem to degrade gracefully. But Mistral models, Qwen models, and anything with Mixture-of-Experts layers tend to suffer more. You cannot assume GPTQ will be "good enough" just because the perplexity numbers look acceptable.
Perplexity, by the way, is a terrible proxy for real quality. A model can have a perplexity increase of only 0.3 and still be unusable for tasks that require precise instruction following. The benchmark numbers lie. You have to test on your actual workload.
AWQ: What Makes It Different
AWQ (Activation-aware Weight Quantization) takes a different approach. Instead of treating all weights equally, it protects the weights that correspond to the most important activation channels. The insight is simple: not every neuron matters equally. Some channels carry critical semantic information, while others are effectively noise. AWQ identifies the salient channels and quantizes the rest more aggressively.
In my testing, AWQ consistently outperforms GPTQ on the same bit width. The gap is not always dramatic, but it is reliable. Where GPTQ turns Mistral-Large into a rambling mess, AWQ keeps it coherent. Where GPTQ causes Qwen2.5-72B to drop negations, AWQ preserves them.
The trade-off is speed. AWQ models are generally slower to inference than GPTQ models on the same hardware. The activation-aware protection requires more complex dequantization logic, and not all inference engines optimize for it equally. On vLLM with CUDA graph capture, the gap is usually 10-15%. On llama.cpp with CPU offload, it can be 25% or more. You are paying for quality with latency.
Another AWQ advantage: it handles MoE models better. I tested Mixtral-8x22B in both GPTQ and AWQ at 4-bit. The GPTQ version had noticeable quality degradation in expert routing — the model would sometimes activate the wrong expert for a given token, leading to bizarre context switches mid-sentence. The AWQ version maintained coherent expert selection. If you are running MoE models, AWQ is almost always the better choice.
The downside: AWQ model files are less widely available. TheBloke and other quantizers have standardized on GPTQ because it is older and better supported. Finding a high-quality AWQ quant of a niche model can be difficult. You may need to quantize it yourself, which requires significant GPU memory and time.
The Quality Comparison Table
Here are my qualitative ratings based on eight months of production use. These are not benchmark scores — they are my assessments of real-world usability across different task types. Scale is 1-10, where 10 is indistinguishable from FP16.
| Format | Factual QA | Summarization | Reasoning | Coding | Long Context | Creative Writing |
|---|---|---|---|---|---|---|
| FP16 | 10 | 10 | 10 | 10 | 10 | 10 |
| INT8 | 9 | 8 | 7 | 6 | 6 | 7 |
| GPTQ 4-bit | 8 | 7 | 5 | 5 | 5 | 6 |
| AWQ 4-bit | 9 | 8 | 7 | 7 | 7 | 7 |
A few notes on these numbers. Factual QA is the most forgiving task — even 4-bit models usually get the facts right if the knowledge is in their training data. Summarization starts showing cracks at 4-bit because the model must maintain coherence across long passages while compressing information. Reasoning and coding are where the damage is most visible: GPTQ 4-bit is genuinely unreliable for production code generation, in my experience. Long context suffers across all quantized formats because attention pattern errors compound over distance. Creative writing is surprisingly resilient, probably because there is no objective "correct" answer and minor coherence issues can pass as stylistic choices.
Speed Comparison
Quality is only half the story. The other half is throughput. Here are my measured numbers on a single RTX 4090, running vLLM 0.8.0, with a 2K context prompt and 512 output tokens. Model is Qwen2.5-72B-Instruct where it fits, otherwise the largest variant that loads.
| Format | Tokens/sec | VRAM (GB) | Batch Size 1 | Batch Size 4 |
|---|---|---|---|---|
| FP16 | 28 | 145 | Does not fit | Does not fit |
| INT8 | 42 | 78 | Fits | Does not fit |
| GPTQ 4-bit | 68 | 42 | Fits | Fits |
| AWQ 4-bit | 58 | 42 | Fits | Fits |
The pattern is clear: you trade quality for speed and memory. FP16 is the quality ceiling but requires hardware most people do not have. INT8 is the middle ground — decent quality, moderate speed, still heavy on VRAM. The 4-bit methods unlock batching and larger models on consumer hardware, but the quality cost is real and task-dependent.
One detail that matters: GPTQ is faster than AWQ at the same bit width. If your workload is latency-sensitive and quality requirements are loose, GPTQ wins on speed. If you need reliable output and can tolerate slightly lower throughput, AWQ is worth the cost.
Which Tasks Suffer Most
After running hundreds of prompts across dozens of model-format combinations, here is my ranking of task sensitivity to quantization, from most to least affected:
1. Multi-step reasoning with symbolic logic. This is where quantization hurts most. The model must maintain precise state across multiple inference steps, and small weight errors propagate. I have seen GPTQ 4-bit models fail on simple chain-of-thought prompts that FP16 handles flawlessly.
2. Code generation with complex types. Type systems are unforgiving. A slightly wrong weight in the attention pattern can cause the model to hallucinate a method signature or misplace a generic parameter. These errors are subtle and dangerous because the code looks correct until you compile it.
3. Long-document analysis. Anything over 8K tokens starts showing degradation. The model loses track of earlier sections, contradicts itself, or fixates on recent context. This affects both summarization and question-answering over long texts.
4. Instruction following with negation. "Do not include X" becomes "Include X" with alarming frequency on GPTQ models. AWQ is better but not perfect. This is a known failure mode caused by asymmetric quantization of certain attention heads.
5. Factual recall. Surprisingly resilient. If the knowledge is in the training data, even 4-bit models usually retrieve it correctly. The exception is rare facts that require precise activation of infrequently used neurons — those get lost more easily.
6. Creative writing. Least affected. Minor coherence issues can actually improve creative output by adding unpredictability. I would not publish a novel from a GPTQ model, but for brainstorming and first drafts, 4-bit is fine.
Model-Specific Notes
Not all models quantize equally. Here are my findings on specific architectures:
Llama 3.x: Robust to quantization. Both GPTQ and AWQ perform well. The dense transformer architecture seems to distribute information broadly enough that losing precision in some weights does not destroy specific capabilities. My go-to for 4-bit deployment.
Qwen 2.5: Mixed. The 72B model handles INT8 well but suffers in GPTQ 4-bit on coding tasks. AWQ rescues it. The 32B model is more forgiving — I have run it in GPTQ 4-bit for general chat without major issues.
Mistral Large / Codestral: Sensitive. These models seem to rely on precise attention patterns more than Llama, and quantization hits them harder. I avoid GPTQ on Mistral entirely. AWQ 4-bit is acceptable for non-critical tasks.
DeepSeek-V3 / R1: MoE architecture makes quantization tricky. Expert routing decisions are sensitive to weight precision. GPTQ causes noticeable expert mis-selection. AWQ is better but still not as clean as dense models. I run these at INT8 when possible.
Phi-4: Small enough that you rarely need 4-bit, but when I tested it, GPTQ was surprisingly bad for the model size. I suspect the architecture has narrow bottlenecks that quantization destroys.
What I Actually Use in Production
Here is my current setup, as of April 2026. No hedging, no "it depends on your use case." This is what runs on my servers.
For customer-facing APIs where correctness matters: FP16 when hardware allows. On my A100 80GB nodes, I run Llama-3.3-70B at FP16 with tensor parallelism. On RTX 4090 nodes, I run the same model at INT8. I do not serve 4-bit models to paying customers for tasks involving reasoning or code.
For internal tools and retrieval: INT8 across the board. Document search, classification, simple extraction — these tasks are forgiving, and INT8 saves me enough VRAM to run larger models or higher batch sizes. Qwen2.5-72B-INT8 is my workhorse for internal RAG pipelines.
AWQ 4-bit. When I am processing thousands of documents overnight, speed and memory matter more than perfection. AWQ gives me the best quality-per-gigabyte ratio. I validate outputs with spot checks and have a fallback to INT8 for any job that produces suspicious results.
For personal experiments and prototyping: Whatever fits. I have run GPTQ 3-bit models just to see what happens. They are terrible, but they load on a single 24GB card, and sometimes that is all I need to test a prompt template.
For coding assistants: FP16 or INT8 only. I do not use 4-bit models for code generation in any production context. The risk of subtle bugs is too high. My Copilot-like internal tool runs DeepSeek-Coder-V2 at INT8, and I am considering moving it to FP16 as hardware becomes available.
For MoE models: INT8 minimum. I will not run Mixtral or DeepSeek-V3 at 4-bit for anything important. The expert routing degradation is real and unpredictable. If I need the capacity of an MoE model, I budget for the VRAM to run it at INT8 or better.
The Bottom Line
Quantization is not free. The community has done an impressive job making 4-bit models usable, but "usable" is not the same as "good." The benchmarks you see online — perplexity on WikiText, accuracy on multiple-choice tasks — do not capture the failures that matter in production.
My advice: start with the highest precision your hardware allows. Drop to INT8 when you need to fit a larger model or run higher batch sizes. Use 4-bit only for tasks where errors are cheap to fix, or where a human will review the output. And test on your actual data, not on generic benchmarks. The worst quantization failures are the ones that look almost right.
I have a 4-bit GPTQ model running right now, actually. It is handling a low-stakes content tagging job, labeling support tickets by category. It gets about 94% of them right, and the 6% it gets wrong are caught by a simple validation rule. That is a good use of 4-bit — fast, cheap, and failure-tolerant.
But I would not let it write code. Not even with AWQ.