AI "attention sinks" mean models fake long-context memory, and new fixes may not work. A new paper asks the uncomfortable question. #AI #LLM #MachineLearning #AttentionSinks #LongContext

https://freegardner.com/synapse/ai-attention-sinks-remain-unsolved-at-long-context.html