Quantization - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

xbill for Google Developer Experts

Aug 19

Deploying a QAT Checkpoint Your Serving Stack Can't Load: Gemma 4 E2B in Pure JAX on One TPU

#tpu #jax #llm #quantization

10 min read

Aomi Qaza

Aug 15

KV Cache INT4 Quantization for 1M+ Token Context Windows

#aiengineering #quantization

3 min read

Pneumetron

Jul 17

Unsloth Releases Qwen3.6-27B-NVFP4: Enhanced Throughput and Agentic Coding for Developers

#aiml #largelanguagemodels #quantization #qwen

3 min read

Pneumetron

Jul 15

Bonsai-27B: A 1-Bit LLM for On-Device Inference with Llama.cpp and MLX

#llm #quantization #1bit #gguf

3 min read

JoTeq the First

Jul 14

Quantizing MedGemma to INT4 (GPTQ/W4A16): Everything That Broke Along the Way

#machinelearning #llm #quantization #opensource

6 min read

Aug 13

LLM Inference Optimization: From Quantization to Speculative Decoding

#llm #inference #quantization #optimization

2 min read

AI Explore

Jul 7

Self-Hosting a Model Means Self-Hosting Its Evaluation Too

#ai #llm #quantization #mlops

5 min read

Jun 11

Gemma 4 QAT on a 1080 Ti: What 'Quantization-Aware' Actually Buys — and Fitting the 12B on 8 GB at 16k

#llm #machinelearning #gemma #quantization

5 min read

Jun 11

Quantization formats compared: GGUF vs GPTQ vs AWQ vs NF4

#llm #quantization #mlops #tutorial

7 min read

soy

Jun 10

INT8 Q/DQ Calibration on Blackwell: 1.8 the TRT 10 + FP16 Baseline

#tensorrt #quantization #gpu #machinelearning

7 min read

Kunal

Jul 6

LLM Quantization Levels Compared: Q4_K_M vs Q8_0 vs FP16 [2026]

#localllm #quantization #gguf #ollama

15 min read

Jun 11

How to Pick a GGUF Quant Level for Your VRAM Budget

#localllm #gguf #quantization #gpu

4 min read

Alan West

May 27

Why your quantized LLM loses its MTP heads and how to keep them

#machinelearning #llm #python #quantization

5 min read

MxGuru

May 20

The Best Result This Week Was a Failed Prediction — Phase-3a Doesn't Transfer

#quantization #hsaq #methodology #granite

1 min read