Coding for Large-scale Distributed Machine Learning

This project aims at investigating network and error-control coding for large-scale distributed machine learning (DML) to improve system reliability, efficiency and security.

With the development of computing and communication technologies and emerging data-driven applications, the volume of data for various learning systems increases explosively along with the number of involving computing nodes.

However, large-scale DML faces severe challenges of slow-response nodes, high communication loads and complexity and security, which largely have not been addressed yet.

In the project, for slow-response nodes, we will apply structured random codes (i.e., BATS codes), which can improve error-correction and meanwhile reduce communication loads with low complexity.

For federated learning (FL), we will investigate reliable and secure codes for uplinks and communication-efficient downlink. We will develop reliable and secure ADMM with coding. A two-step coding scheme is planned to address the nonlinear primal-dual optimization of ADMM. To our knowledge, coding for FL and ADMM has seldom been studied.

We will first analyze theoretical performance and then optimize related coding schemes. More practical learning networks will be built for experiments with real data after computer simulations.

The project staff consists of two Ph.D students and the applicant and will be hosted by Division of Information Science and Engineering, EECS School, KTH. The project is planned to be 4-years.