Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Deep learning approaches for image cytometry: assessing cellular morphological responses to drug perturbations
Uppsala University, Disciplinary Domain of Medicine and Pharmacy, Faculty of Pharmacy, Department of Pharmaceutical Biosciences. (Pharmaceutical Bioinformatics)ORCID iD: 0000-0003-4046-9017
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Description
Abstract [en]

Image cytometry is the analysis of cell properties from microscopy image data and is used ubiquitously in basic cell biology, medical diagnosis and drug development. In recent years deep learning has shown impressive results for many image cytometry tasks, including image processing, segmentation, classification and detection. Deep learning enables a more data-driven and end-to-end approach than was previously possible with conventional methods. This thesis investigates deep learning-based approaches for assessing cellular morphological responses to drug perturbations. In paper I we demonstrated the benefit of combining convolutional neural networks and transfer learning for predicting mechanism of action and nucleus translocation. In paper II we showed, using convolutional and recurrent neural networks applied to time-lapse microscopy data, that it is possible to predict if mRNA delivery via nanoparticles has been effective based on cell morphology changes at time points prior to the protein production evidence of successful delivery. In paper III we used convolutional neural networks, adversarial training and privileged information to faithfully generate fluorescence imaging channels of adipocyte cells from their corresponding z-stack of brightfield images. Our models were both faithful at the fluorescence image level and at the level of the features extracted from these images, features that are commonly used for downstream analysis, including the design of effective drug therapies. In paper IV we showed that convolutional neural networks trained on brightfield image data provide similar, and in some cases superior, performance to models trained on fluorescence image data for predicting mechanism of action, due to the brightfield images possessing additional information not available in the fluorescence images. In paper V we applied deep learning models to brightfield time-lapse image data to explore the evolution of cellular morphological changes after drug administration for a diverse set of compounds, compounds that are often used as positive controls in image-based assays.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2023. , p. 55
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Pharmacy, ISSN 1651-6192 ; 336
Keywords [en]
Deep Learning, Microscopy, Image Analysis
National Category
Medical and Health Sciences
Research subject
Pharmaceutical Science
Identifiers
URN: urn:nbn:se:uu:diva-508615ISBN: 978-91-513-1864-6 (print)OAI: oai:DiVA.org:uu-508615DiVA, id: diva2:1785791
Public defence
2023-09-22, B21, Uppsala biomedicinska centrum (BMC), Husargatan 3, Uppsala, 13:00 (English)
Opponent
Supervisors
Available from: 2023-09-04 Created: 2023-08-04 Last updated: 2023-09-04
List of papers
1. Transfer Learning with Deep Convolutional Neural Networks for Classifying Cellular Morphological Changes
Open this publication in new window or tab >>Transfer Learning with Deep Convolutional Neural Networks for Classifying Cellular Morphological Changes
2019 (English)In: SLAS discovery : advancing life sciences R & D, ISSN 2472-5552, Vol. 24, no 4, p. 466-475Article in journal (Refereed) Published
Abstract [en]

The quantification and identification of cellular phenotypes from high-content microscopy images has proven to be very useful for understanding biological activity in response to different drug treatments. The traditional approach has been to use classical image analysis to quantify changes in cell morphology, which requires several nontrivial and independent analysis steps. Recently, convolutional neural networks have emerged as a compelling alternative, offering good predictive performance and the possibility to replace traditional workflows with a single network architecture. In this study, we applied the pretrained deep convolutional neural networks ResNet50, InceptionV3, and InceptionResnetV2 to predict cell mechanisms of action in response to chemical perturbations for two cell profiling datasets from the Broad Bioimage Benchmark Collection. These networks were pretrained on ImageNet, enabling much quicker model training. We obtain higher predictive accuracy than previously reported, between 95% and 97%. The ability to quickly and accurately distinguish between different cell morphologies from a scarce amount of labeled data illustrates the combined benefit of transfer learning and deep convolutional neural networks for interrogating cell-based images.

Keywords
cell phenotypes, deep learning, high-content imaging, machine learning, transfer learning
National Category
Bioinformatics and Systems Biology
Identifiers
urn:nbn:se:uu:diva-375566 (URN)10.1177/2472555218818756 (DOI)000461840200004 ()30641024 (PubMedID)
Funder
Swedish Foundation for Strategic Research Swedish National Infrastructure for Computing (SNIC)
Available from: 2019-01-31 Created: 2019-01-31 Last updated: 2023-08-04Bibliographically approved
2. Deep-learning models for lipid nanoparticle-based drug delivery
Open this publication in new window or tab >>Deep-learning models for lipid nanoparticle-based drug delivery
Show others...
2021 (English)In: Nanomedicine, ISSN 1743-5889, E-ISSN 1748-6963, Vol. 16, no 13, p. 1097-1110Article in journal (Refereed) Published
Abstract [en]

Background: Early prediction of time-lapse microscopy experiments enables intelligent data management and decision-making. Aim: Using time-lapse data of HepG2 cells exposed to lipid nanoparticles loaded with mRNA for expression of GFP, the authors hypothesized that it is possible to predict in advance whether a cell will express GFP. Methods: The first modeling approach used a convolutional neural network extracting per-cell features at early time points. These features were then combined and explored using either a long short-term memory network (approach 2) or time series feature extraction and gradient boosting machines (approach 3). Results: Accounting for the temporal dynamics significantly improved performance. Conclusion: The results highlight the benefit of accounting for temporal dynamics when studying drug delivery using high-content imaging.

Place, publisher, year, edition, pages
Future MedicineFuture Medicine Ltd, 2021
Keywords
artificial neural networks, high-content imaging, machine learning, predictive modeling, time-lapse microscopy
National Category
Pharmaceutical Sciences
Research subject
Pharmaceutical Science
Identifiers
urn:nbn:se:uu:diva-454055 (URN)10.2217/nnm-2020-0461 (DOI)000647143100001 ()33949890 (PubMedID)
Funder
Swedish Foundation for Strategic Research , BD15-0008SB16-0046Science for Life Laboratory, SciLifeLabeSSENCE - An eScience Collaboration
Available from: 2021-09-26 Created: 2021-09-26 Last updated: 2024-01-15Bibliographically approved
3. Learning to see colours: Biologically relevant virtual staining for adipocyte cell images
Open this publication in new window or tab >>Learning to see colours: Biologically relevant virtual staining for adipocyte cell images
Show others...
2021 (English)In: PLOS ONE, E-ISSN 1932-6203, Vol. 16, no 10Article in journal (Refereed) Published
Abstract [en]

Fluorescence microscopy, which visualizes cellular components with fluorescent stains, is an invaluable method in image cytometry. From these images various cellular features can be extracted. Together these features form phenotypes that can be used to determine effective drug therapies, such as those based on nanomedicines. Unfortunately, fluorescence microscopy is time-consuming, expensive, labour intensive, and toxic to the cells. Bright-field images lack these downsides but also lack the clear contrast of the cellular components and hence are difficult to use for downstream analysis. Generating the fluorescence images directly from bright-field images using virtual staining (also known as “label-free prediction” and “in-silico labeling”) can get the best of both worlds, but can be very challenging to do for poorly visible cellular structures in the bright-field images. To tackle this problem deep learning models were explored to learn the mapping between bright-field and fluorescence images for adipocyte cell images. The models were tailored for each imaging channel, paying particular attention to the various challenges in each case, and those with the highest fidelity in extracted cell-level features were selected. The solutions included utilizing privileged information for the nuclear channel, and using image gradient information and adversarial training for the lipids channel. The former resulted in better morphological and count features and the latter resulted in more faithfully captured defects in the lipids, which are key features required for downstream analysis of these channels.

Place, publisher, year, edition, pages
Public Library of Science (PLoS)Public Library of Science (PLoS), 2021
Keywords
Multidisciplinary
National Category
Bioinformatics (Computational Biology) Medical Image Processing
Identifiers
urn:nbn:se:uu:diva-459847 (URN)10.1371/journal.pone.0258546 (DOI)000755689200030 ()34653209 (PubMedID)
Funder
Swedish Foundation for Strategic Research , BD150008Swedish Foundation for Strategic Research , ARC19-0016
Available from: 2021-11-29 Created: 2021-11-29 Last updated: 2024-01-15Bibliographically approved
4. Evaluating the utility of brightfield image data for mechanism of action prediction
Open this publication in new window or tab >>Evaluating the utility of brightfield image data for mechanism of action prediction
Show others...
2023 (English)In: PloS Computational Biology, ISSN 1553-734X, E-ISSN 1553-7358, Vol. 19, no 7, article id e1011323Article in journal (Refereed) Published
Abstract [en]

Fluorescence staining techniques, such as Cell Painting, together with fluorescence microscopy have proven invaluable for visualizing and quantifying the effects that drugs and other perturbations have on cultured cells. However, fluorescence microscopy is expensive, time-consuming, labor-intensive, and the stains applied can be cytotoxic, interfering with the activity under study. The simplest form of microscopy, brightfield microscopy, lacks these downsides, but the images produced have low contrast and the cellular compartments are difficult to discern. Nevertheless, by harnessing deep learning, these brightfield images may still be sufficient for various predictive purposes. In this study, we compared the predictive performance of models trained on fluorescence images to those trained on brightfield images for predicting the mechanism of action (MoA) of different drugs. We also extracted CellProfiler features from the fluorescence images and used them to benchmark the performance. Overall, we found comparable and largely correlated predictive performance for the two imaging modalities. This is promising for future studies of MoAs in time-lapse experiments for which using fluorescence images is problematic. Explorations based on explainable AI techniques also provided valuable insights regarding compounds that were better predicted by one modality over the other.

Place, publisher, year, edition, pages
Public Library of Science (PLoS), 2023
National Category
Pharmaceutical Sciences Medical Image Processing
Identifiers
urn:nbn:se:uu:diva-508612 (URN)10.1371/journal.pcbi.1011323 (DOI)001037077600001 ()37490493 (PubMedID)
Funder
Swedish Foundation for Strategic Research, BD15-0008SB16-0046Swedish Research Council, 2020-03731Swedish Research Council, 2020-01865Swedish Research Council Formas, 2022-00940Swedish Cancer Society, 22 2412 PjEU, Horizon Europe, 101057014 (PARC)EU, Horizon Europe, 101057442 (REMEDI4ALL)EU, European Research Council, ERC-2015-CoG 683810
Note

De två första författarna delar förstaförfattarskapet

De två sista författarna delar sistaförfattarskapet

Available from: 2023-08-04 Created: 2023-08-04 Last updated: 2023-10-13Bibliographically approved
5. Exploring the evolution of cellular morphological changes after drug administration based on brightfield image data
Open this publication in new window or tab >>Exploring the evolution of cellular morphological changes after drug administration based on brightfield image data
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Most image based studies of the morphological effects of compound treatments on cells, such as those for elucidating a compound's mechanism of action, use fixed-cell based approaches whereby the cells are fixated, stained, and imaged with fluorescence microscopy some time after compound administration. This snapshot data, however, cannot uncover any information on the temporal dynamics of the induced morphological changes. For instance regarding the rate at which these changes occur following compound perturbation. Live-cell compatible dyes can be used although are limited by technical difficulties, cytotoxicity and photobleaching. A simpler, cheaper and less harmful option is to use brightfield microscopy. Although brightfield images have less contrast than fluorescence images and cannot separate out the different cellular compartments, we here show that for compounds inducing morphological changes on cells, brightfield data, together with convolutional neural networks and feature projection techniques, can be used to extract such temporal information from time-lapse experiments.

National Category
Pharmaceutical Sciences
Identifiers
urn:nbn:se:uu:diva-508613 (URN)
Available from: 2023-08-04 Created: 2023-08-04 Last updated: 2023-08-14Bibliographically approved

Open Access in DiVA

UUThesis_Harrison,P-2023(1209 kB)201 downloads
File information
File name FULLTEXT01.pdfFile size 1209 kBChecksum SHA-512
eded59fa1c83d7ff7bf84ba32609cc11d89958d03a5c90c2aa421081f4dff155139553224637a2ec1b35fe2f64918671ef39c92ebe1073aa0ed24b2ac944f84f
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Harrison, Philip John
By organisation
Department of Pharmaceutical Biosciences
Medical and Health Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 204 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1841 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf