New arXiv work proposes the Superposed Specialisation Hypothesis for Mixture of Experts models, arguing experts specialise in fine-grained features rather than single domains, and introduces RouterInterp for…
#AI #MachineLearning #MixtureOfExperts #Interpretability
https://arxiv.org/abs/2610.11775
