ByteDance Seed traced DeepSeek's wobble to "token phase".

Same fact, one phase over:
retrieval accuracy can swing 40 percentage points.

Chunked KV compression is what made long context affordable.

The bill just moved from memory to variance. #LongContext

KV² is a self-refining KV-cache compression method that uses a lightweight proxy scorer to find informative tokens, then only reprocesses that subset for eviction scoring, outperforming baselines as cache budgets tighten.

#KVCache #LLMInference #LongContext
https://arxiv.org/abs/2610.03198

New SMat-Attention paper introduces structured causal masks with tunable VC-dimension for long-context tasks, bridging softmax and linear attention with chunkwise algorithms. No OOP for true long-range modeling performance yet.

#LongContext #Attention #AIResearch
https://arxiv.org/abs/2609.36062

📝 Neuer Blog-Beitrag: RAG vs. Long Context 2026: Was die Daten wirklich sagen

RAG ist 1.250x günstiger pro Query – aber Long Context liefert bessere Ergebnisse bei globalem Dokumentverständnis. Die 2026er Daten zeigen: Hybride Architekturen mit intelligentem Routing…

#RAG #LongContext #KI

AI "attention sinks" mean models fake long-context memory, and new fixes may not work. A new paper asks the uncomfortable question. #AI #LLM #MachineLearning #AttentionSinks #LongContext

https://freegardner.com/synapse/ai-attention-sinks-remain-unsolved-at-long-context.html

In 1973, 8 healthy people faked one voice, got locked in psych wards, and Rosenhan’s famous study later looked shaky.

On Being Sane in Insane P…
https://users.sussex.ac.uk/~grahamh/RM1web/Classic%20papers/Rosenhan1973%20(Science%20179).pdf

#Research #Science #LongContext #Hallucination #Psychology