Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Robust Learning from Distributed and Heterogeneous Data
Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Scientific Computing. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Computational Science. Uppsala University, Science for Life Laboratory, SciLifeLab.ORCID iD: 0000-0001-9500-1791
2026 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Modern machine learning is increasingly expanding beyond centralized, mono-modal training toward systems that must also learn from data across distributed edge devices and heterogeneous data modalities. This transition breaks the foundational identical and independent distribution (i.i.d.) assumptions of traditional models, making robustness a first-class requirement for real-world applications. This thesis studies the mechanisms and methodologies necessary to achieve algorithmic robustness across three intersecting dimensions: distributed optimization, geometry-aware uncertainty quantification, and simulation-based inference.

The first dimension addresses statistical heterogeneity in Federated Learning (FL), a distributed training framework in which multiple participants collaboratively train a shared model without exchanging their local data. In FL, the non-i.i.d. nature of distributed data often induces performance degradation, convergence issues and fairness problems. Through an empirical study on drug discovery and the development of new algorithms, this work demonstrates that adaptive optimization and dynamic hyperparameter adjustment can mitigate training instabilities. These methods ensure equitable performance across diverse data silos, preventing the global model from favoring specific participants.

The second dimension explores the structural challenges of multi-modal language models, which map data of heterogeneous modalities onto complex, non-Euclidean manifolds. This research models aleatoric and epistemic uncertainty with directional distributions via parametric models and Riemannian Flow Matching. This geometry-aware approach allows models to respect the intrinsic geometric structure of the embedding space, providing a mathematically grounded framework for models to quantify their ignorance when confronted with ambiguous or out-of-distribution inputs.

The final dimension addresses the robustness of a unified framework which supports both forward and inverse processes for Bayesian inference. The proposed framework utilizes a unified Flow Matching model to learn the joint distribution of parameters and observations. By employing randomized masking, this architecture robustly handles partially observed or noisy data, integrating forward and inverse processes into a single cohesive neural network without the need for specialized retraining.  Collectively, this thesis contributes theoretical analyses, novel algorithms, and empirical validations that advance the robustness of machine learning across federated optimization, multi-modal uncertainty quantification, and simulation-based inference, bridging the gap between idealized training assumptions and the demands of real-world applications.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2026. , p. 62
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology, ISSN 1651-6214 ; 2665
Keywords [en]
Machine Learning, Distributed Optimization, Federated Learning, Probabilistic Modeling, Multi-modal Learning
National Category
Artificial Intelligence
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:uu:diva-583490ISBN: 978-91-513-2816-4 (print)OAI: oai:DiVA.org:uu-583490DiVA, id: diva2:2049839
Public defence
2026-06-04, 101195, Heinz-Otto Kreiss, Regementsvägen 10, Uppsala, 09:15 (English)
Opponent
Supervisors
Available from: 2026-05-07 Created: 2026-03-31 Last updated: 2026-05-13
List of papers
1. Federated learning for predicting compound mechanism of action based on image-data from cell painting
Open this publication in new window or tab >>Federated learning for predicting compound mechanism of action based on image-data from cell painting
2024 (English)In: Artificial Intelligence in the Life Sciences, E-ISSN 2667-3185, Vol. 5, article id 100098Article in journal (Refereed) Published
Abstract [en]

Having access to sufficient data is essential in order to train accurate machine learning models, but much data is not publicly available. In drug discovery this is particularly evident, as much data is withheld at pharmaceutical companies for various reasons. Federated Learning (FL) aims at training a joint model between multiple parties but without disclosing data between the parties. In this work, we leverage Federated Learning to predict compound Mechanism of Action (MoA) using fluorescence image data from cell painting. Our study evaluates the effectiveness and efficiency of FL, comparing to non-collaborative and data-sharing collaborative learning in diverse scenarios. Specifically, we investigate the impact of data heterogeneity across participants on MoA prediction, an essential concern in real-life applications of FL, and demonstrate the benefits for all involved parties. This work highlights the potential of federated learning in multi-institutional collaborative machine learning for drug discovery and assessment of chemicals, offering a promising avenue to overcome data-sharing constraints.

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Federated learning, Cell profiling, Cell painting, Artificial intelligence, Collaborative learning
National Category
Computer and Information Sciences
Research subject
Scientific Computing; Machine learning
Identifiers
urn:nbn:se:uu:diva-544328 (URN)10.1016/j.ailsci.2024.100098 (DOI)001333825000001 ()2-s2.0-85193044761 (Scopus ID)
Projects
eSSENCE - An eScience Collaboration
Funder
Uppsala UniversityeSSENCE - An eScience CollaborationSwedish Research Council, 2020-03731Swedish Research Council, 2020-01865Swedish Research Council Formas, 2022-00940Swedish Cancer Society, 22 2412EU, Horizon Europe, 101057014EU, Horizon Europe, 101057442Swedish Research Council, 2022-06725
Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2026-03-31Bibliographically approved
2. Accelerating Fair Federated Learning: Adaptive Federated Adam
Open this publication in new window or tab >>Accelerating Fair Federated Learning: Adaptive Federated Adam
2024 (English)In: IEEE Transactions on Machine Learning in Communications and Networking, E-ISSN 2831-316X, Vol. 2, p. 1017-1032Article in journal (Refereed) Published
Abstract [en]

Federated learning is a distributed and privacy-preserving approach to train a statistical model collaboratively from decentralized data held by different parties. However, when the datasets are not independent and identically distributed, models trained by naive federated algorithms may be biased towards certain participants, and model performance across participants is non-uniform. This is known as the fairness problem in federated learning. In this paper, we formulate fairness-controlled federated learning as a dynamical multi-objective optimization problem to ensure the fairness and convergence with theoretical guarantee. To solve the problem efficiently, we study the convergence and bias of Adam as the server optimizer in federated learning, and propose Adaptive Federated Adam (AdaFedAdam) to accelerate fair federated learning with alleviated bias. We validated the effectiveness, Pareto optimality and robustness of AdaFedAdam with numerical experiments and show that AdaFedAdam outperforms existing algorithms, providing better convergence and fairness properties of the federated scheme.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Distributed learning, fair machine learning, federated learning, federated optimization
National Category
Computer Sciences
Research subject
Machine learning
Identifiers
urn:nbn:se:uu:diva-544327 (URN)10.1109/tmlcn.2024.3423648 (DOI)001487803200002 ()
Projects
eSSENCE - An eScience Collaboration
Funder
National Academic Infrastructure for Supercomputing in Sweden (NAISS), 2023/22-1048eSSENCE - An eScience CollaborationUppsala University
Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2026-03-31Bibliographically approved
3. Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere
Open this publication in new window or tab >>Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere
Show others...
2025 (English)Conference paper, Published paper (Refereed)
National Category
Artificial Intelligence Computer graphics and computer vision Computational Mathematics
Research subject
Machine learning; Scientific Computing
Identifiers
urn:nbn:se:uu:diva-572611 (URN)
Conference
39th Conference on Neural Information Processing Systems (NeurIPS 2025)
Funder
Swedish Research Council, 2023-05167Swedish Research Council, 2023-05593Kjell and Marta Beijer FoundationKnut and Alice Wallenberg Foundation
Available from: 2025-12-07 Created: 2025-12-07 Last updated: 2026-03-31Bibliographically approved
4. Epistemic Uncertainty Quantification for Pre-trained VLMs via Riemannian Flow Matching
Open this publication in new window or tab >>Epistemic Uncertainty Quantification for Pre-trained VLMs via Riemannian Flow Matching
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Vision-Language Models (VLMs) are typically deterministic in nature and lack intrinsic mechanisms to quantify epistemic uncertainty, which reflects the model's lack of knowledge or ignorance of its own representations. We theoretically motivate negative log-density of an embedding as a proxy for the epistemic uncertainty, where low-density regions signify model ignorance. The proposed method REPVLM computes the probability density on the hyperspherical manifold of the VLM embeddings using Riemannian Flow Matching. We empirically demonstrate that REPVLM achieves near-perfect correlation between uncertainty and prediction error, significantly outperforming existing baselines. Beyond classification, we also demonstrate that the model also provides a scalable metric for out-of-distribution detection and automated data curation.

National Category
Computer Vision and Learning Systems
Identifiers
urn:nbn:se:uu:diva-582516 (URN)
Available from: 2026-03-17 Created: 2026-03-17 Last updated: 2026-03-31Bibliographically approved
5. OneFlowSBI: One Model, Many Queries for Simulation-Based Inference
Open this publication in new window or tab >>OneFlowSBI: One Model, Many Queries for Simulation-Based Inference
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

We introduce OneFlowSBI, a unified framework for simulation-based inference that learns a single flow-matching generative model over the joint distribution of parameters and observations. Leveraging a query-aware masking distribution during training, the same model supports multiple inference tasks, including posterior sampling, likelihood estimation, and arbitrary conditional distributions, without task-specific retraining. We evaluate OneFlowSBI on ten benchmark inference problems and two high-dimensional real-world inverse problems across multiple simulation budgets. OneFlowSBI is shown to deliver competitive performance against state-of-the-art generalized inference solvers and specialized posterior estimators, while enabling efficient sampling with few ODE integration steps and remaining robust under noisy and partially observed data.

National Category
Computer Sciences
Identifiers
urn:nbn:se:uu:diva-582537 (URN)
Available from: 2026-03-18 Created: 2026-03-18 Last updated: 2026-03-31Bibliographically approved

Open Access in DiVA

UUThesis_Ju,L-2026(1638 kB)92 downloads
File information
File name FULLTEXT01.pdfFile size 1638 kBChecksum SHA-512
58dec5cd558c53af6dcbb84ab7ec9b44258186ff66c62b7011b960d83ac1eeb1c3d9cee729072913ae0cf8854efe8f7b9580f5b3d393fb80a44a22dc00d67af5
Type fulltextMimetype application/pdf

Authority records

Ju, Li

Search in DiVA

By author/editor
Ju, Li
By organisation
Division of Scientific ComputingComputational ScienceScience for Life Laboratory, SciLifeLab
Artificial Intelligence

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 473 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf