Generating high-resolution video with a diffusion transformer means attending over very long sequences that stretch across both space and time, and self-attention is where the cost concentrates. Sparse attention cuts that bill without retraining by having each head look at only the parts that matter.

The saving creates a scheduling problem once the work is split across GPUs. Adaptive routing means different heads end up attending to different amounts, so the per-head workloads stop being equal, and in synchronous parallel execution everybody waits for the slowest. The authors call it a rank-level straggler problem, which is exactly right: the cost is now set by whichever GPU drew the heaviest share.

So the fix has to be runtime load balancing rather than a smarter sparsity pattern. Once work is data-dependent you cannot divide it evenly in advance, and any static split will leave some ranks idle while one finishes.