Let's make NLP work for dialects! Submit to VarDial :-)

This would not be possible without my amazing coauthors: @nverma1.bsky.social , @zouhar.bsky.social , Philipp Koehn, and David Yarowksy, @jhuclsp.bsky.social. Working with you has been its own reward.

Our paper pushes for stating what we want from multilinguality, for being precise in defining our famous curse, for thinking about theoretical solutions to this curse, and for questioning the gap between theory and practice.

We then look the empirical curse of multilinguality for embedding space structure on small scale models for 100 languages for 8 training and evaluation configurations and find some non-cursed settings.

The above scaling is mild. We show that it is possible to fit 7000 languages in 30 extra dimensions while maintaining perfect multilinguality. Thus, we argue there is no *theoretical* curse of multilinguality for embedding space structure.

Then we show that any non-degenerate space encoding L languages has a *necessary* dimensionality at least Omega(log L). This proves our theorem.

We do it in two parts. First, we look at the minimum *sufficient* dimensionality of a perfect multilingual space with L languages. We show a construction with orthogonal language offsets whose minimum sufficient dimensionality is O(log L).

Then we prove this theorem.

Multilinguality aims for two things: shared high-quality monolingual semantics, and cross-lingual alignment. We define these as our conditions for perfect multilinguality.

First, we need to define the goal: perfect multilinguality for embedding spaces. Then, we'll see what perfect multilinguality costs in theory as we scale up language coverage. If the capacity cost is high, we have a theoretical curse. If not, no theoretical curse!

We all know about the curse of multilinguality. We know that empirical performance degrades as you add languages to a model. But *in theory*, does it have to? Let’s talk about the theoretical curse of multilinguality for embedding space structure.

Colourful idioms, false friends, garden paths, cultural experience, signboards, menus, dialectal slang, archaic language, minority scripts, self-referential logic, social conventions, audio dialogue, daily life - find it in Last Translation Benchmark. Join us for the next one :-)