Man, Machine, and Mathematics: A mathe-physical search for differential equation solutions via deep learning

Well posed mathematical problems are seldom solvable in an explicit, conventional sense. A notable example of this phenomenon is within the field of differential equations, where an exact solution often exists but is analytically impossible to find. The development of approximation methods to estimate such well-parametrized but not explicitly knowable solutions is an important endeavour within the field.

This project presents a blueprint for building such approximations via deep learning. We build our methods on assumptions satisfied by a large class of differential equations, such as Frechet differentiability of the equation operators and compact domains of interest. We aim to demonstrate that neural network solution models are almost always capable of being found under such assumptions.

We also hope to present explicit results on the errors expected from these models, alongside techniques for quantifying and minimising those errors. Finally, we aim to provide strict guarantees on the model sizes, architectures, optimization run-times, etc., needed to search for models that are accurate up to pre-specified tolerances.

The project will hope to advance the field of neural network-based differential equation solving by adding two novel facets to it. First, by fusing ideas from numerical methods and error analysis into deep learning of parametrized objects, we hope to move the field away from its reliance on surrogate markers, like loss functions, as a means of error analysis. In turn, that will also lead us to methods of efficient error correction.

Second, we aim to provide a rigorous set of a priori-decidable strategies for efficient model building by leveraging the emerging advances in computing infrastructure and the algorithms that can tap into them.

Indeed, the project has already seen its first successes in the modelling of dynamical systems' differential equations (Physical Review E 105, 065305) and in sparsifying deep networks (Redman et al., ICML 2022). Given the generality of the results developed in the latter work (it concerns all deep networks, not just differential equation solution models), we believe the project will lead to significant results beyond its originally planned scope.

Success in these objectives will require the creation of new techniques within the fields of random dynamical systems, stochastic computational methods, deep learning methods for equation solving, and optimization/complexity analysis, amongst others, each of which plays a central role in the CDT's research and training missions.