Logo: to the web site of Uppsala University

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning (ML) are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap encompasses key aspects of how ML is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of ML for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Prostate cancer is a heterogeneous disease showing variability both among individuals and within a patient. While most cases are indolent, aggressive tumors require early intervention. Accurately predicting tumor behavior is challenging, contributing to overdiagnosis but also undertreatment. Current imaging methods may miss the most malignant areas, leading to biopsies often capturing non-malignant prostate tissue even if cancer is present elsewhere in the organ. This non-malignant tissue, however, holds potential as a source for novel diagnostic and prognostic markers. Our clinical dataset comprises men with raised prostate-specific antigen but whose initial prostate needle biopsies only contained benign tissue. Half of the paired patients remained cancer-free for over eight years, while the others were diagnosed with prostate cancer within 30 months of follow-up. We share these initial benign biopsies to enable the exploration of morphological changes in non-malignant tissue and the potential for improved diagnostic accuracy in the early identification of patients with prostate cancer.

Antibiotic resistance is a severe danger to human health, and combination therapy with several antibiotics has emerged as a viable treatment option for multi-resistant strains. CombiANT is a recently developed agar plate-based assay where three reservoirs on the bottom of the plate create a diffusion landscape of three antibiotics that allows testing of the efficiency of antibiotic combinations. This test, however, requires manually assigning nine reference points to each plate, which can be prone to errors, especially when plates need to be graded in large batches and by different users. In this study, an automated deep learning-based image processing method is presented that can accurately segment bacterial growth and measure distances between key points on the CombiANT assay at sub-millimeter precision. The software was tested on 100 plates using photos captured by three different users with their mobile phone cameras, comparing the automated analysis with the human scoring. The result indicates significant agreement between the users and the software ([Formula: see text] mm mean absolute error) and remains consistent when applied to different photos of the same assay despite varying photo qualities and lighting conditions. The speed and robustness of the automated analysis could streamline clinical workflows and make it easier to tailor treatment to specific infections. It could also aid large-scale antibiotic research by quickly processing hundreds of experiments in batch, obtaining better data, and ultimately supporting the development of better treatment strategies. The software can easily be integrated into a potential smartphone application, making it accessible in resource-limited environments. Integrating deep learning-based smartphone image analysis with simple agar-based tests like CombiANT could unlock powerful tools for combating antibiotic resistance.

Spatial transcriptomics has transformed our understanding of tissue architecture by preserving the spatial context of gene expression patterns. Simultaneously, advances in imaging AI have enabled extraction of morphological features describing the tissue. This review introduces a framework for categorizing methods that combine spatial transcriptomics with tissue morphology, focusing on either translating or integrating morphological features into spatial transcriptomics. Translation involves using morphology to predict gene expression, creating super-resolution maps or inferring genetic information from H&E-stained samples. Integration enriches spatial transcriptomics by identifying morphological features that complement gene expression. We also explore learning strategies and future directions for this emerging field.

Approximately 50 million people suffer from sepsis yearly, and 13 million die from it. For every hour a patient with septic shock is untreated, their survival rate decreases by 8%. Therefore, rapid detection and antibiotic susceptibility profiling of bacterial agents in the blood of sepsis patients are crucial for determining appropriate treatment. Here, we introduce a method to isolate bacteria from whole blood with high separation efficiency through Smart centrifugation, followed by microfluidic trapping and subsequent detection using deep learning applied to microscopy images. We detected, within 2 h, E. coli, K. pneumoniae, or E. faecalis from spiked samples of healthy human donor blood at clinically relevant concentrations as low as 9, 7 and 32 colony-forming units per ml of blood, respectively. However, the detection of S. aureus remains a challenge. This rapid isolation and detection represents a significant advancement towards culture-free detection of bloodstream infections.

Diagnostic needle biopsies that miss clinically significant prostate cancer (PCa) often sample benign tissue near hidden cancers. Such benign samples might still display subtle morphological signs of cancer elsewhere in the prostate. This study examined if artificial intelligence (AI) could detect these morphological clues in benign biopsies from men with elevated prostate-specific antigen (PSA) levels to predict subsequent diagnosis of clinically significant PCa within 30 months. We analysed biopsies from 232 men initially diagnosed as benign, matched for age, diagnosis year, and PSA levels-half were later diagnosed with PCa, while the rest remained cancer-free for at least eight years. The AI model accurately predicted future PCa diagnosis from initial benign biopsies (AUC = 0.82), highlighting patterns such as changes in stromal collagen and altered glandular epithelial cells. This demonstrates that AI analysis of routine haematoxylin-eosin biopsy sections can detect subtle signs indicating clinically significant PCa before it becomes histologically apparent. Such morphological patterns shed light on the broader tissue alterations induced by prostate cancer, even in benign tissue, potentially enhancing early detection and clinical decision-making.

The Xenium In Situ platform is a new spatial transcriptomics product commercialized by 10x Genomics, capable of mapping hundreds of genes in situ at subcellular resolution. Given the multitude of commercially available spatial transcriptomics technologies, recommendations in choice of platform and analysis guidelines are increasingly important. Herein, we explore 25 Xenium datasets generated from multiple tissues and species, comparing scalability, resolution, data quality, capacities and limitations with eight other spatially resolved transcriptomics technologies and commercial platforms. In addition, we benchmark the performance of multiple open-source computational tools, when applied to Xenium datasets, in tasks including preprocessing, cell segmentation, selection of spatially variable features and domain identification. This study serves as an independent analysis of the performance of Xenium, and provides best practices and recommendations for analysis of such datasets.

Time series of compositional data are a common format for many high-throughput studies of biological molecules, e.g., analyzing the response to a treatment or with the aim of predicting an outcome. However, data from some time points may be missing, which reduces the size of the complete dataset. We propose a method for binary classification that includes imputation for missing values, dimensionality reduction, and logarithmic transformation of compositional data. Imputation approaches entail models that incorporate artificial data alongside true measurements, thereby supplementing the dataset. In the application part, we consider two case studies with longitudinal data and associated target labels, aiming to improve prediction accuracy. We predict infants’ food allergies from their gut microbiome with a balanced accuracy of 0.72. We forecast postpartum depression based on gut microbiome data collected during pregnancy, with a balanced accuracy of 0.62. Features extracted from the microbiome time series, specifically ratios of bacterial abundance, are statistically significant indicators of depression.

For effective treatment of bacterial infections, it is essential to identify the species causing the infection as early as possible. Current methods typically require hours of overnight culturing of a bacterial sample and a larger quantity of cells to function effectively. This study uses one-hour phase-contrast time-lapses of single-cell bacterial growth collected from microfluidic chip traps, also known as a “mother machine”. These time-lapses are then used to train deep artificial neural networks (Convolutional Neural Networks and Vision Transformers) to identify the species. We have previously demonstrated this approach on four different species, which is now extended to seven common pathogens causing human infections: Pseudomonas aeruginosa, Escherichia coli, Klebsiella pneumoniae, Acinetobacter baumannii, Enterococcus faecalis, Proteus mirabilis, and Staphylococcus aureus. Furthermore, we expand upon our previous work by evaluating real-time performance as additional frames are captured during testing, and investigating the role of training set size, data quality, and data augmentation as well as the contribution of texture and morphology to performance. The experiments suggest that spatiotemporal features can be learned from video data of bacterial cell divisions, with both texture and morphology contributing to classifier decision. The method could be used simultaneously with phenotypic antibiotic susceptibility testing (AST) in the microfluidic chip. The best models attained an average precision of 93.5% and a recall of 94.7% (0.997 AUC) on a trap basis in a separate, unseen experiment with mixed species after around one hour. However, in a real-world scenario, one can assume many traps will contain the actual species causing the infection. Still, several challenges remain, such as isolating bacteria directly from blood and validating the method on diverse clinical isolates. This proof of principle study brings us closer to real-time diagnostics that could transform the initial treatment of acute infections.

Automatic segmentation of histopathology whole -slide images (WSI) usually involves supervised training of deep learning models with pixel -level labels to classify each pixel of the WSI into tissue regions such as benign or cancerous. However, fully supervised segmentation requires large-scale data manually annotated by experts, which can be expensive and time-consuming to obtain. Non -fully supervised methods, ranging from semi -supervised to unsupervised, have been proposed to address this issue and have been successful in WSI segmentation tasks. But these methods have mainly been focused on technical advancements in algorithmic performance rather than on the development of practical tools that could be used by pathologists or researchers in real -world scenarios. In contrast, we present DEPICTER (Deep rEPresentatIon ClusTERing), an interactive segmentation tool for histopathology annotation that produces a patch -wise dense segmentation map at WSI level. The interactive nature of DEPICTER leverages self- and semi -supervised learning approaches to allow the user to participate in the segmentation producing reliable results while reducing the workload. DEPICTER consists of three steps: first, a pretrained model is used to compute embeddings from image patches. Next, the user selects a number of benign and cancerous patches from the multi -resolution image. Finally, guided by the deep representations, label propagation is achieved using our novel seeded iterative clustering method or by directly interacting with the embedding space via feature space gating. We report both real-time interaction results with three pathologists and evaluate the performance on three public cancer classification dataset benchmarks through simulations. The code and demos of DEPICTER are publicly available at https://github.com/eduardchelebian/depicter.