Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Fast computation of mutual information in the frequency domain with applications to global multimodal image alignment
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction.ORCID iD: 0000-0003-0253-9037
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction.ORCID iD: 0000-0001-7312-8222
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Visual Information and Interaction. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computerized Image Analysis and Human-Computer Interaction.ORCID iD: 0000-0002-6041-6310
2022 (English)In: Pattern Recognition Letters, ISSN 0167-8655, E-ISSN 1872-7344, Vol. 159, p. 196-203Article in journal (Refereed) Published
Abstract [en]

Multimodal image alignment is the process of finding spatial correspondences between images formed by different imaging techniques or under different conditions, to facilitate heterogeneous data fusion and correlative analysis. The information-theoretic concept of mutual information (MI) is widely used as a similarity measure to guide multimodal alignment processes, where most works have focused on local maximization of MI, which typically works well only for small displacements. This points to a need for global maximization of MI, which has previously been computationally infeasible due to the high run-time complexity of existing algorithms. We propose an efficient algorithm for computing MI for all discrete displacements (formalized as the cross-mutual information function (CMIF)), which is based on cross-correlation computed in the frequency domain. We show that the algorithm is equivalent to a direct method while superior in terms of run-time. Furthermore, we propose a method for multimodal image alignment for transformation models with few degrees of freedom (e.g., rigid) based on the proposed CMIF-algorithm. We evaluate the efficacy of the proposed method on three distinct benchmark datasets, containing remote sensing images, cytological images, and histological images, and we observe excellent success-rates (in recovering known rigid transformations), overall outperforming alternative methods, including local optimization of MI, as well as several recent deep learning-based approaches. We also evaluate the run-times of a GPU implementation of the proposed algorithm and observe speed-ups from 100 to more than 10,000 times for realistic image sizes compared to a GPU implementation of a direct method. Code is shared as open-source at github.com/MIDA-group/globalign.

Place, publisher, year, edition, pages
Elsevier BV Elsevier, 2022. Vol. 159, p. 196-203
Keywords [en]
Mutual information, Image alignment, Global optimization, Multimodal, Entropy
National Category
Medical Imaging Computer graphics and computer vision
Research subject
Computerized Image Processing
Identifiers
URN: urn:nbn:se:uu:diva-447807DOI: 10.1016/j.patrec.2022.05.022ISI: 000833390400013OAI: oai:DiVA.org:uu-447807DiVA, id: diva2:1575535
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Vinnova, 2017-02447Swedish Research Council, 2017-04385Available from: 2021-06-30 Created: 2021-06-30 Last updated: 2025-02-09Bibliographically approved
In thesis
1. Methods for Reliable Image Registration: Algorithms, Distance Measures, and Representations
Open this publication in new window or tab >>Methods for Reliable Image Registration: Algorithms, Distance Measures, and Representations
2022 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Much biomedical and medical research relies on the collection of ever-larger amounts of image data (both 2D images and 3D volumes, as well as time-series) and increasingly from multiple sources. Image registration, the process of finding correspondences between images based on the affinity of features of interest, is often required as a vital step towards the final analysis, which may consist of a comparison of images, measurement of movement, or fusion of complementary information. The contributions in this work are centered around reliable image registration methods for both 2D and 3D images with the aim of wide applicability: similarity and distance measures between images for image registration, algorithms for efficient computation of these, and other commonly used measures for both local and global optimization frameworks, and representations for multimodal image registration where the appearance and structures present in the images may vary dramatically.

The main contributions are: (i) distance measures for affine symmetric intensity image registration, combining intensity and spatial information based on the notion of alpha-cuts from fuzzy set theory; (ii) the extension of the affine registration method to more flexible deformable transformation models, leading to the framework Intensity and Spatial Information-Based Deformable Image Registration (INSPIRE); (iii) two efficient algorithms for computing the proposed distances and their spatial gradients and thereby enabling local gradient-based optimization; (iv) a contrastive representation learning method, Contrastive Multimodal Image Representation for Registration (CoMIR), utilizing deep learning techniques to obtain common representations that can be registered using methods designed for monomodal scenarios; (v) efficient algorithms for global optimization of mutual information and similarities of normalized gradient fields; (vi) a comparative study exploring the applicability of modern image-to-image translation methods to facilitate multimodal registration; (vii) the Stochastic Distance Transform, using the theory of discrete random sets to offer improved noise-insensitivity to distance computations; (viii) extensive evaluation of the proposed image registration methods on a number of different datasets mainly from (bio)medical imaging, where they exhibit excellent performance, and reliability, suggesting wide utility.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2022. p. 110
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, ISSN 1651-6214 ; 2107
Keywords
Image registration, alignment, local optimization, global optimization, mutual information, normalized gradient fields, representation learning
National Category
Computer graphics and computer vision Medical Imaging
Research subject
Computerized Image Processing
Identifiers
urn:nbn:se:uu:diva-463393 (URN)978-91-513-1382-5 (ISBN)
Public defence
2022-02-25, 101195, Ångström, Lägerhyddsvägen 1, Uppsala, 13:00 (English)
Opponent
Supervisors
Available from: 2022-02-03 Created: 2022-01-10 Last updated: 2025-02-09

Open Access in DiVA

fulltext(1988 kB)750 downloads
File information
File name FULLTEXT01.pdfFile size 1988 kBChecksum SHA-512
e9053ad7d9f978d1ba9659b8e07a2cf5fbce8eaa49bf0de9d330f3385b11510a768ed3ce854b6433fbaf3764fb168d239efe859a5c5f005323822a529bcbde2d
Type fulltextMimetype application/pdf

Other links

Publisher's full textPre-print in arXiv

Authority records

Öfverstedt, JohanLindblad, JoakimSladoje, Natasa

Search in DiVA

By author/editor
Öfverstedt, JohanLindblad, JoakimSladoje, Natasa
By organisation
Division of Visual Information and InteractionComputerized Image Analysis and Human-Computer Interaction
In the same journal
Pattern Recognition Letters
Medical ImagingComputer graphics and computer vision

Search outside of DiVA

GoogleGoogle Scholar
Total: 751 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 1241 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf