Maarten Sap

@maartensap.bsky.social

Working on #NLProc for social good. Currently at LTI at CMU. 🏳‍🌈

@akhilayerukola.bsky.socialReposted by @maartensap.bsky.social

Did you know gifting four koi 🐟 is offensive in Japan (4 = "shi" = death 💀), or that chopsticks 🥢upright in rice is inappropriate in China (like funeral incense 🕯️)? Top VLMs don't!

Our #COLM2026 paper introduces 🌏NormViz to measure visual norm understanding in 16 countries.

Best model: 25.3% 🚨 🧵👇

Overview of NORMVIZ-BENCH for visual norm understanding, consisting of 3,272 contrastive image pairs across 16 countries that differ only in the culturally relevant behavior. (a) A contrastive pair example from Japan; group accuracy requires a model to correctly classify both images in a pair. (b) Images are sourced via Text-to-Image generation, inpainting, and Retrieval. (c) Benchmark statistics and pair type distribution.
@carolynrose.bsky.socialReposted by @maartensap.bsky.social

I am grateful to the ACL executive committee for nominating me for the position of ACL VP Elect.The ballot is now open for ACL members to vote. We are up against unprecedented challenges in the world and in our community, but if we join forces, we can move forward in synergy, strength, and success.

@maartensap.bsky.social

Is the safety-capability tradeoff for LLMs real? Or could it be an artefact of the benchmarks we use to measure safety?? We did some explorations with psychometrics-inspired multi-dimensional IRT models and created BenchMIRT to explore these questions! See 🧵