uu.seUppsala University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Super-resolution Reconstruction of Transmission Electron Microscopy Images using Deep Learning
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction.
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction.ORCID iD: 0000-0003-3557-4947
Vironova AB, Stockholm, Sweden.
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction.
2019 (English)In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), IEEE, 2019, p. 548-551Conference paper, Published paper (Refereed)
Abstract [en]

Deep learning techniques have shown promising outcomes in single image super-resolution (SR) reconstruction from noisy and blurry low resolution data. The SR reconstruction can cater the fundamental. limitations of transmission electron microscopy (TEM) imaging to potentially attain a balance among the trade-offs like imaging-speed, spatial/temporal resolution, and dose/exposure-time, which is often difficult to achieve simultaneously otherwise. In this work, we present a convolutional neural network (CNN) model, utilizing both local and global skip connections, aiming for 4 x SR reconstruction of TEM images. We used exact image pairs of a calibration grid to generate our training and independent testing datasets. The results are compared and discussed using models trained on synthetic (downsampled) and real data from the calibration grid. We also compare the variants of the proposed network with well-known classical interpolations techniques. Finally, we investigate the domain adaptation capacity of the CNN-based model by testing it on TEM images of a cilia sample, having different image characteristics as compared to the calibration-grid.

Place, publisher, year, edition, pages
IEEE, 2019. p. 548-551
Series
Biomedical Imaging, IEEE International Symposium on, E-ISSN 1945-7928
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:uu:diva-381797DOI: 10.1109/ISBI.2019.8759153ISI: 000485040000121ISBN: 978-1-5386-3641-1 (electronic)OAI: oai:DiVA.org:uu-381797DiVA, id: diva2:1304799
Conference
16th IEEE International Symposium on Biomedical Imaging (ISBI), APR 08-11, 2019, Venice, ITALY
Funder
Swedish Foundation for Strategic Research , SB16-0046Vinnova, 2016-02329Available from: 2019-04-13 Created: 2019-04-13 Last updated: 2019-10-23Bibliographically approved
In thesis
1. Methods for Processing and Analysis of Biomedical TEM Images
Open this publication in new window or tab >>Methods for Processing and Analysis of Biomedical TEM Images
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Transmission Electron Microscopy (TEM) has the high resolving capability and high clinical significance; however, the current manual diagnostic procedure using TEM is complicated and time-consuming, requiring rarely available expertise for analyzing TEM images of the biological specimen. This thesis addresses the bottlenecks of TEM-based analysis by proposing image analysis methods to automate and improve critical time-consuming steps of currently manual diagnostic procedures. The automation is demonstrated on the computer-assisted diagnosis of Primary Ciliary Dyskinesia (PCD), a genetic condition for which TEM analysis is considered the gold standard.

The methods proposed for the automated workflow mimic the manual procedure performed by the pathologists to detect objects of interest – diagnostically relevant cilia instances – followed by a computational step to combine information from multiple detected objects to enhance the important structural details. The workflow includes an approach for efficient search through a sample to identify objects and locate areas with a high density of objects of interest in low-resolution images, to perform high-resolution imaging of the identified areas. Subsequently, high-quality objects in high-resolution images are detected, processed, and the extracted information is combined to enhance structural details.

This thesis also addresses the challenges typical for TEM imaging, such as sample drift and deformation, or damage due to high electron dose for long exposure times. Two alternative paths are investigated: (i) different strategies combining short exposure imaging with suitable denoising techniques, including conventional approaches and a proposed deep learning based method, are explored; (ii) conventional interpolation approaches and a proposed deep learning based method are analyzed for super-resolution reconstruction using a single image. For both explored directions, in the best case scenario, the processing time is nearly 20 times faster as compared to the acquisition time for a single long exposure high illumination image. Moreover, the reconstruction approach (ii) requires nearly 16 times lesser data (storage space) and overcomes the need for high-resolution image acquisition.

Finally, the thesis addresses critical needs to enable objective and reliable evaluation of TEM image denoising approaches. A method for synthesizing realistic noise-free TEM reference images is proposed, and a denoising benchmark dataset is generated and made publicly available. The proposed dataset consists of noise-free references along with masks encompassing the critical diagnostic structures. This enables performance evaluation based on the capability of denoising methods to preserve structural details, instead of merely grading them based on the signal to noise ratio improvement and preservation of gross structures.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2019. p. 49
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, ISSN 1651-6214 ; 1807
Keywords
image analysis, image processing, deep learning, transmission electron microscopy, denoising, super-resolution reconstruction, registration, detection
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computerized Image Processing
Identifiers
urn:nbn:se:uu:diva-381800 (URN)978-91-513-0653-7 (ISBN)
Public defence
2019-06-05, Room 2446, ITC, Lägerhyddsvägen 2, Uppsala, 10:15 (English)
Opponent
Supervisors
Available from: 2019-05-15 Created: 2019-04-17 Last updated: 2019-06-18

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records BETA

Suveer, AmitGupta, AnindyaSintorn, Ida-Maria

Search in DiVA

By author/editor
Suveer, AmitGupta, AnindyaSintorn, Ida-Maria
By organisation
Division of Visual Information and InteractionComputerized Image Analysis and Human-Computer Interaction
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 52 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf