DeBLAS: Accelerate LLM Pretraining by Length-based Sequence Scheduling

Geng Yu, Jianing Zhu, Jiangchao Yao, Bo Han, Gang Niu, Masashi Sugiyama

Action editor: Andriy Mnih

https://openreview.net/forum?id=s0MTAeYvww

#pretraining #batches #training