Randomness in Dynamical Systems and Machine Learning

Theoretical and empirical advances in modern machine learning (ML) have been informed by state-of-the-art developments in multiple disciplines.

Central to these developments is the use of modeling tools and perspectives from dynamical systems, leading to principled approaches and cutting-edge advances in understanding and designing learning models.

However, the roles and effects (particularly those constructive ones) of randomness on the learning behavior of modern deep networks are largely unexplored.

Since randomness appears in various forms during the learning process for these networks, solid understanding of its effects is pivotal to advance the field.

The main objective of this project is to investigate the roles of randomness in learning with modern deep networks through the lens of dynamical systems via a series of interconnected studies.

Random elements, particularly in the form of stochastic noise, are capable of inducing various non-trivial effects on the system dynamics.

Therefore, we expect to discover various interesting noise-induced phenomena and study their implications for ML systems.

Our approach is to employ dynamical system frameworks and methods to (a) explore the impacts of noise on the learning behavior of several deep network models in various directions and settings; and (b) exploit the findings in (a) to construct efficient and reliable learning models for physical applications.

This project will be carried out at Nordita over the next four years.