Logo: to the web site of Uppsala University

Microscopy is an integral technique in biology to study the fundamental components of life visually. Digital microscopy and automation have enabled biologists to conduct faster and larger-scale experiments with a sharp increase in the data generated. Microscopy images contain rich but sparse information, as typically, only small regions in the images are relevant for further study. Image analysis is a crucial tool for biologists in the objective interpretation and extraction of quantitative measurements from microscopy data. Recently, deep learning techniques have shown superior performance in various image analysis tasks. The models learn feature representations from the data by optimizing for a task. However, the techniques require a significant amount of annotated data to perform well. Domain experts are required to annotate microscopy data, making it expensive and time-consuming. The models offer no insight into their prediction, and the learned features are not directly interpretable. This poses challenges to the reliable utilization of the technique in high-trust applications such as drug discovery or disease detection. High data variability in microscopy and poor generalization performance of deep learning models further increase the difficulty in general usage of the technique. 

The work in this thesis presents frameworks and methods to solve the practical challenges of applying deep learning in microscopy. The application-specific evaluation approaches were presented to validate the approaches, aiming to increase trust in the system. The major contributions of this work are as follows. Papers I and III present human-in-the-loop frameworks for quick adaption of deep learning to new data and for improving models' performance based on human input in visual explanations provided by the model, respectively. Paper II proposes a template-matching approach to improve user interactions in the framework proposed in Paper I. Papers III and IV present architectural modifications in the deep learning models proposed for better visual explanation and image-to-image translation, respectively. Papers IV and V present biologically relevant evaluations of approaches, i.e., analysis of the deep learning models in relation to the biological task.

This thesis is aimed towards better utilization and adaptation of the DL methods and techniques to the microscopy data. We show that the annotation burden for the user can be significantly reduced by intuitive annotation frameworks and using contemporary deep-learning paradigms. We further propose architectural modifications in the models to adapt to the requirements and demonstrate the utility of application-specific analysis in microscopy.

The rise of Big Data has catalyzed numerous advanced data-driven methods, while simultaneously posing significant challenges in data management. This thesis aims to address two fundamental aspects of data management–storage management and information extraction–by leveraging machine learning (ML) techniques. In particular, we focus on two research topics: Storage Hierarchy, which explores hierarchical storage management (HSM) in multi-tiered storage systems; and Information Hierarchy, which targets the extraction of intrinsic data hierarchies from raw data.

We begin by introducing the key stages of data life cycle and their associated challenges in the Big Data era, alongside a review of machine learning foundations and their potentials for addressing these challenges. Subsequently, we present the Storage Hierarchy project, which is detailed across Paper I, II, and III. In these works, we develop automated, adaptive, and efficient HSM approaches using reinforcement learning (RL). In Paper I we introduce the HSM-RL framework for managing file-level data migration in hierarchical storage system (HSS). It leverages RL to optimize file placement and temporal difference learning for real-time adaptability. Paper II extends this work to complex real–world scenarios using scientific datasets, exploring the framework’s flexibility, scalability, and effectiveness. Moving to finer granularity, Paper III presents ReStore, an RL-based page-level data migration approach that incorporates the unique characteristics of modern Solid-State Drives (SSDs), such as read/write asymmetry and parallelism.

The Information Hierarchy project focuses on autonomous extraction of implicit data hierarchies from raw, unlabeled data. Presented in Paper IV, we propose InfoHier, a framework that integrates self-supervised learning (SSL) with hierarchical clustering (HC) to uncover latent data representations and hierarchical structures. By jointly training SSL and HC through a dynamic balancing loss, InfoHier ensure that the HC results align with the intrinsic data hierarchy. This method facilitates meaningful and structured information extraction and retrieval. 

Collectively, the Storage Hierarchy and Information Hierarchy projects advance intelligent data management by enabling efficient storage solutions and autonomous information extraction. These contributions pave the foundation for next generation data management systems, addressing the challenges of Big Data with adaptive and scalable solutions.