Where the Blocks Live: Tier-Aware KV-Cache Routing for vLLM at Scale

Name: Where the Blocks Live: Tier-Aware KV-Cache Routing for vLLM at Scale
Uploaded: 2026-10-20T04:55:44.000Z
Description: When GPU memory fills up, vLLM offloads the KV cache to CPU or storage. llm-d tracks where every block lives, and sends each request to its cache. At PyTorch Conference North America, Bongwoo Bak from Moreh will present a poster on "Where the Blocks Live." You'll see when CPU is enough, and when shared storage pays off. Join us in San Jose, October 20-21: https://hubs.la/Q04v4SL60

When GPU memory fills up, vLLM offloads the KV cache to CPU or storage. llm-d tracks where every block lives, and sends each request to its cache. At PyTorch Conference North America, Bongwoo Bak from Moreh will present a poster on "Where the Blocks Live." You'll see when CPU is enough, and when shared storage pays off. Join us in San Jose, October 20-21: https://hubs.la/Q04v4SL60

20 October 2026 · Source: YouTube — PyTorch Official

Watch on YouTube →

More in this topic

ВидеоКак ИИ компании превращают рынок в $300 млрдYouTube — Armeta Kazakhstan13 hours ago

ВидеоDeepSeek Harness (DSH) EXPLAINED | Everything Is a PluginYouTube — Hugging Face14 hours ago

ВидеоCould AI Have Predicted The November 5 Stock Market Crash?YouTube — All About AI21 hours ago

ВидеоThis controls games characters AND robotsSharing the latest news and advancements in AI Discover More: 🛠️ Explore AI Tools & News: https://futuretools.io/ 📰 Weekly Newsletter: https://futuretools.io/newsletter Socials: ❌ Twiter/X: https://x.com/mreflow 🖼️ Instagram: https://instagram.com/mr.eflow 🧵 Threads: https://www.threads.net/@mr.eflow 🟦 LinkedIn: https://www.linkedin.com/in/matt-wolfe-30841712/ 👍 Facebook: https://www.facebook.com/mattrwolfe Let’s work together! - Brand, sponsorship & business inquiries: business@mattwolfe.YouTube — Matt Wolfeyesterday