Local AI 301: Quantization

描述

Learn about quantization in this livestream and get started with quantization using llama.cpp and vLLM. Program: - Introduction to Quantization, Merve Noyan, Hugging Face - Calibration based quantization, Colin Kealty (aka bartowski), RedHat - Quantization Aware Distillation, Maxime Labonne, Aditya Tadimeti, LiquidAI - Fast Microscaling Formats on Any GPU with vLLM, Michael Goin, RedHat