New arXiv work presents the first unified evaluation of weight-only quantization strategies for LLM inference, measuring energy use across Llama 2 7B, Phi 3.5 Mini Instruct, OLMo 1B, Qwen 2.5 7B, and Qwen 2.5 14B on…
#GreenAI #LLMs #Quantization #AIInfrastructure
https://arxiv.org/abs/2610.17531
