🔁 Context matters too. False alarms were more likely when a sentence shared a content word with, or was semantically similar to, a previously seen sentence. The closest semantic distractor shared a content word only 6.7% of the time. (5/6)

✨ Among six semantic-distinctiveness measures, SBERT-based distinctiveness had the highest predictive power. It predicted recognition accuracy beyond word-level features and was also associated with faster responses when participants endorsed a sentence as familiar. (4/6)

🧩 Constituent words matter: average word memorability significantly predicts sentence memorability. But sentence memory is not reducible to the words alone. Semantic distinctiveness explains additional variance beyond word memorability and word frequency. (3/6)

The study tested 2,500 six-word target sentences with 443 participants. Memorability varied substantially across sentences and was moderately consistent across participants, suggesting that intrinsic sentence properties affect how likely a sentence is to be remembered. (2/6)

🧠Why are some sentences easier to remember than others? In our latest WiAIR podcast, we spoke with Dr. Greta Tuckute about her paper, “A distinctive meaning makes a sentence memorable”, on what predicts sentence recognition memory. (1/6)

📊 Frontal and temporal language regions differ: MFG shows no significant gain beyond surprisal, while temporal fROIs show a significant gain from feature-based models. Within the Matryoshka hierarchy, the most general 0.4% of features are most predictive. (5/6🧵)

🧠 The method identifies a coherent 32-voxel “Ghost” subpopulation associated with people-centered content, including relationships/emotions, actions involving people, and pronouns. The cluster is concentrated in a subset of participants. (4/6🧵)

🔍 Sparse SAE features predict voxel responses nearly as well as dense LM residual-stream features. Hard-to-Process voxels show no significant gain beyond surprisal, while Abstract and Concrete voxels benefit significantly from LM-derived features. (3/6🧵)

The study introduces Augmented Sparse Encoding Models, combining sparse autoencoder (SAE) features with surprisal. It tests them on 7T fMRI data from 8 participants listening to 200 linguistically diverse sentences. (2/6🧵)

Can sparse, interpretable LM features help us understand how the brain responds to language? 🧠 In our latest WiAIR podcast, we spoke with Dr. Greta Tuckute about her co-authored paper, “Interpreting Brain Responses to Language with Sparse Features from Language Models.” (1/6🧵)

🔄 Early in training, next-word prediction and behavioral alignment correlate with brain alignment. These links weaken later, as models surpass human next-word prediction proficiency. With feature size controlled, model size is not a reliable predictor of brain alignment. (5/6🧵)

📚 Brain alignment tracks formal linguistic competence, including grammar, syntax, and compositional rules, more closely than functional competence involving world knowledge and reasoning. Functional competence keeps improving after brain alignment saturates. (4/6🧵)

🧠 Brain alignment rises sharply early in training, peaks around 2 to 8B tokens, then largely saturates or declines. The trajectory is similar across very different model sizes. (3/6🧵)

The study evaluates 8 Pythia model sizes across 34 training checkpoints spanning ~300B tokens, asking how brain alignment relates to linguistic competence throughout training. (2/6🧵)