ML

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem.
Malinovskii Vladimir, Andrei Panferov, Ivan Ilin, Han Guo, Peter Richtárik, Dan Alistarh
PDF
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression.
Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, Peter Richtárik
PDF
Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity.
Alexander Tyurin, Marta Pozzi, Ivan Ilin, Peter Richtárik
PDF
Kimad: Adaptive Gradient Compression with Bandwidth Awareness. In Proceedings of the 4th International Workshop on Distributed Machine Learning (pp. 35-48).
Jihao Xin, Ivan Ilin, Shunkang Zhang, Marco Canini, Peter Richtárik
PDF