VibeServe explores using coding agents to automatically generate bespoke LLM serving stacks, matching vLLM and SGLang on Llama-3.1-8B and reaching 2.3x the goodput of tuned SGLang on a 397B MoE model across four…

#AIInfrastructure #LLMServing #GPUs #Semiconductors
https://arxiv.org/abs/2605.06068

New arXiv work introduces RaReCache, a framework that lets a larger LLM decode from a smaller model's KV cache by selectively recomputing only the information-dense tokens that hurt transfer accuracy. A practical step for…

#KVCache #LLMServing #OpenSourceAI
https://arxiv.org/abs/2610.11358