Only later I realized that distance fields are composable, meaning I can compute the fields for individual letters, then just take the min value of overlapping fields to get the final fields for the whole canvas. This avoids recalculating the fields on every movement.
