Logo: to the web site of Uppsala University

At the extremes, two antithetical approaches to describing natural processes exist. Theoretical models can be derived from first principles, allowing for clear interpretability; on the downside, this approach may be infeasible or inefficient for complex systems. Alternatively, methods from statistical machine learning can be employed to learn black box models from large amounts of data, while providing little or no understanding of their inner workings.

Both approaches have different desirable properties and weaknesses. It is natural to ask how they may be combined to create better models. This is the question that the field of physics-informed machine learning is concerned with, and which we will consider in this thesis. More precisely, we investigate ways of integrating additional prior knowledge into machine learning models.

In Paper I, we consider multitask Gaussian processes and devise a way to include so-called sum constraints into the model, where a nonlinear sum of the outputs is required to equal a known value. In Paper II, we consider the task of determining unknown parameters from data when solving partial differential equations (PDEs) with physics-informed neural networks. Given the prior knowledge that the measurement noise is homogeneous but otherwise unknown, we demonstrate that it is possible to learn the solution and parameters of the PDE jointly with the noise distribution. In Paper III, we consider generative adversarial networks, which may produce realistic-looking samples but fail to reproduce their true distribution. In our work, we mitigate this issue by matching the true and generated distributions of statistics extracted from the data.

Conventionally, the models used in science and technology were obtained through the careful study of nature. Scientists proposed models based on their observations, and the models were understood in detail. In the new paradigm of machine learning (ML), on the other hand, models are instead trained on large amounts of data. They are typically of a black-box nature and come without guarantees on their accuracy and range of validity. 

It is natural to ask if both approaches can be combined beneficially. Incorporating domain knowledge from the sciences may make ML models more accurate and robust. Conversely, models and techniques from ML may constitute useful tools for improving scientific models. These questions are the subject of this dissertation, which therefore belongs to the broad area of physics-informed machine learning.        

The first main contribution of the thesis concerns various problems encountered in regression tasks. Physics-informed neural networks (PINNs) are extended to suit situations with noisy or incomplete data. Firstly, a method to learn the noise distribution in the case of homogeneous measurement noise is developed. Secondly, repulsive ensembles are employed to enable Bayesian uncertainty quantification in PINNs. Furthermore, multitask Gaussian processes (GPs) are considered, and a method to incorporate nonlinear sum constraints is presented.        

The second group of contributions deals with the use of generative models in physical applications. A modification of generative adversarial networks (GANs) is developed, where the distribution of certain high-level statistics chosen from domain knowledge is matched between true and generated data. A modularized architecture tailored to the generation of in-ice radio signals is devised for use as a component in an optimization pipeline. This architecture enables the generation of physically consistent samples as well as the differentiability of the samples in an efficient manner.