Logo: to the web site of Uppsala University

Real-time imaging can be used to monitor, analyze and control medical treatments. In this thesis, we want to explain the spatiotemporal motion and thus enable more advanced procedures, especially real-time adaptation in radiation therapy. The motion occurring between image acquisitions can be quantified by image registration, which generates a mapping between the images. The contribution of the thesis consists of three papers, where we have used different approaches to estimate the motion between images.

In Paper I, we combine a state-of-the-art method in real-time tracking with a learned sparse-to-dense interpolation scheme. For this, we track an arbitrary number of regions in a sequence of medical images. We estimated a sparse displacement field, based on the tracking positions and used the interpolation network to achieve its dense representation.

Paper II was a contribution to a challenge in learnable image registration where we finished at 2nd place. Here we train a deep learning method to estimate the dense displacement field between two images. For this, we used a network architecture inspired by both conventional medical image registration methods and optical flow in computer vision.

For Paper III, we estimate the dynamics of spatiotemporal images by training a generative network. We use nonlinear dimensional reduction techniques and assume a linear dynamic in a low-dimensional latent space. In comparison with conventional image registration methods, we provide a method more suitable for real-world scenarios, with the possibility of imputation and extrapolation. Although the problem is challenging and several questions are left unanswered we believe a combination of conventional, learnable, and dynamic modeling of the motion is the way forward.

Motion is a fundamental aspect of human life. Even during low-intensity activities, we move. The lungs absorb oxygen when inhaling and desorb carbon dioxide when exhaling. The heart pumps oxygenated blood to the body's organs. Wave-like contractions help us process food. All such events cause motion within the body. Being able to describe motion offers benefits in medical health, e.g., analysis of organ functions and guidance during ongoing treatments. The motion can be captured by acquiring medical images in real-time. However, in several cases, the resolution of the medical images is limited by the acquisition time, and the images suffer from low temporal and spatial resolution. One such example appears in radiotherapy, e.g., by acquiring 2D cine-MRIs for monitoring ongoing treatment sessions. An accurate estimation of the entire 3D motion provides a more realistic estimate of the actual delivery outcome and is a necessary feature for more advanced procedures, like real-time beam adaptation.

In this thesis, we develop methods to estimate the motion from temporally and spatially sparse medical image sequences. We start by extracting knowledge from optimization-based medical image registration methods and showing how deep learning can reduce execution time. Then, we model the motion dynamics as a sequence of deformable image registrations. Due to the high dimensionality of the medical image, we model the dynamics in a lower dimensional space. For this, we apply dimension reduction techniques like principal component analysis and variational auto-encoders. The dynamic is then modeled using state-space representations and diffusion probabilistic models to solve the two inference problems of forecasting and simulating the state processes.

The main contribution lies in the five presented scientific articles, where we deal with the problem of temporally and spatially sparse sequences separately and then combine them into a uniform solution. The proposed methods are evaluated on medical images of several modalities, such as MRI, CT, and ultrasound, and finally demonstrated on the use case in the radiotherapy domain, where more accurate motion estimates could spare healthy tissues from being exposed to radiation dose.