Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity.

Alexander Tyurin, Marta Pozzi, Ivan Ilin, Peter Richtárik

January, 2024 ML