On the Deep Learning theory

Research in Machine Learning has brought a paradigm shift in most fields. A deeper understanding of Machine Learning through Applied Mathematics would lead someone to realize how far it could improve our daily life. From Medicine, that has outperformed human diagnosis in medical imaging, to autonomous vehicles that show great promises on making the road safer, or even in complex games such as GO which demonstrated neural networks were capable to be creative.

With such improvement at stake, no doubt that Machine Learning is at the heart of what I consider to be currently one of the most exciting areas of research.

As a first part of my doctoral programme, I would be really interested in providing theoretical results that could lead us to a better understanding of Deep Learning performances, such as it has been done with the study of the impact of the initialization and the choice of activations functions in the training of a neural network in the paper S. Hayou, A.

Doucet, J. Rousseau, International Conference on Machine Learning (ICLR), 2019. I really appreciate this idea of thinking in an original and in-depth way in order to improve a model rather than simply rely on a larger amount of data, that perhaps will make one's model perform well without really knowing why.

As a doctoral student, I would like to focus on how mathematics can justify the performances achieved by Machine Learning and more specifically Deep Learning techniques. I am perfectly convinced that Neural Networks architectures can be designed with respect to a mathematical model and thus should not be seen as a black box with tuning of hyperparameters (such as the depth) to achieve the best accuracy possible.

Indeed, with the fast technological advance, some engineers nowadays tend to rely on the huge amount of computational resources which is now available. However, to build such a model, basically one has to think long and hard about it before designing it. In my view, providing Mathematical proofs to empirical results is clearly the most captivating part of the work done in Machine Learning so far.

Finally, even if based on theoretical evidence, I would like my work to found applications in fields such as health care. With the pandemic these last two years, one can realize how impactful in our daily life can artificial intelligence be through automatic diagnosis at a large scale that could end up being faster and perhaps more accurate than medical expertise.

To do so, I will compare my methods to state-of-the-art methods on some datasets that I hope to have access to. Therefore, my research work includes a programming component that would essentially avoid huge computations time. This project falls within the EPSRC Artificial Intelligence Technologies and Numerical Analysis research areas. It will be supervised by Professor Jared Tanner