Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
Refine search result
123 1 - 50 of 107
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1. Ancuti, Codruta O.
    et al.
    Luo, Ziwei
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Gustafsson, Fredrik K.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Zhao, Zheng
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Busch, Christoph
    NTIRE 2023 HR NonHomogeneous Dehazing Challenge Report2023In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancover: Institute of Electrical and Electronics Engineers (IEEE), 2023Conference paper (Refereed)
    Abstract [en]

    This study assesses the outcomes of the NTIRE 2023 Challenge on Non-Homogeneous Dehazing, wherein novel techniques were proposed and evaluated on new image dataset called HD-NH-HAZE. The HD-NH-HAZE dataset contains 50 high resolution pairs of real-life outdoor images featuring nonhomogeneous hazy images and corresponding haze-free images of the same scene. The nonhomogeneous haze was simulated using a professional setup that replicated real-world conditions of hazy scenarios. The competition had 246 participants and 17 teams that competed in the final testing phase, and the proposed solutions demonstrated the cutting-edge in image dehazing technology.

  • 2.
    Andersson, Carl R.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Wahlström, Niklas
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Learning deep autoregressive models for hierarchical data2021In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 529-534Conference paper (Refereed)
    Abstract [en]

    We propose a model for hierarchical structured data as an extension to the stochastic temporal convolutional network. The proposed model combines an autoregressive model with a hierarchical variational autoencoder and downsampling to achieve superior computational complexity. We evaluate the proposed model on two different types of sequential data: speech and handwritten text. The results are promising with the proposed model achieving state-of-the-art performance.

    Download full text (pdf)
    fulltext
  • 3.
    Baumann, Dominik
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Aalto Univ, Dept Elect Engn & Automat, Espoo, Finland..
    Kowalczyk, Krzysztof
    Wroc Iaw Univ Sci & Technol, Dept Control Syst & Mechatron, Wroc Iaw, Poland..
    Tiels, Koen
    Eindhoven Univ Technol, Dept Mech Engn, Eindhoven, Netherlands..
    Wachel, Pawe L.
    Wroc Iaw Univ Sci & Technol, Dept Control Syst & Mechatron, Wroc Iaw, Poland..
    A computationally lightweight safe learning algorithm2023In: 2023 62nd IEEE Conference on Decision and Control, (CDC), Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 1022-1027Conference paper (Refereed)
    Abstract [en]

    Safety is an essential asset when learning control policies for physical systems, as violating safety constraints during training can lead to expensive hardware damage. In response to this need, the field of safe learning has emerged with algorithms that can provide probabilistic safety guarantees without knowledge of the underlying system dynamics. Those algorithms often rely on Gaussian process inference. Unfortunately, Gaussian process inference scales cubically with the number of data points, limiting applicability to high-dimensional and embedded systems. In this paper, we propose a safe learning algorithm that provides probabilistic safety guarantees but leverages the Nadaraya-Watson estimator instead of Gaussian processes. For the Nadaraya-Watson estimator, we can reach logarithmic scaling with the number of data points. We provide theoretical guarantees for the estimates, embed them into a safe learning algorithm, and show numerical experiments on a simulated seven-degrees-of-freedom robot manipulator.

  • 4.
    Baumann, Dominik
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Solowjow, Friedrich
    Johansson, Karl Henrik
    Trimpe, Sebastian
    Identifying Causal Structure in Dynamical Systems2022In: Transactions on Machine Learning Research, E-ISSN 2835-8856Article in journal (Refereed)
  • 5. Bijl, Hildo
    et al.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    van Wingerden, Jan-Willem
    Verhaegen, Michel
    System identification through online sparse Gaussian process regression with input noise2017In: IFAC Journal of Systems and Control, ISSN 2468-6018, Vol. 2, p. 1-11Article in journal (Refereed)
  • 6. Birhane, Abeba
    et al.
    Sumpter, David J. T.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    The games we play: critical complexity improves machine learning2022In: HHAI2022: Augmenting Human Intellect / [ed] Stefan Schlobach; María Pérez-Ortiz; Myrthe Tielman, 2022, p. 3-16Conference paper (Refereed)
    Abstract [en]

    When mathematical modelling is applied to capture a complex system, multiple models are often created that characterise different aspects of that system. Often, a model at one level will produce a prediction which is contradictory at another level but both models are accepted because they are both useful. Rather than aiming to build a single unified model of a complex system, the modeller acknowledges the infinity of ways of capturing the system of interest, while offering their own specific insight. We refer to this pragmatic applied approach to complex systems — one which acknowledges that they are incompressible, dynamic, nonlinear, historical, contextual, and value-laden — as Open Machine Learning (Open ML). In this paper we define Open ML and contrast it with some of the grand narratives of ML of two forms: 1) Closed ML, ML which emphasizes learning with minimal human input (e.g. Google’s Alpha Zero) and 2) Partially Open ML, ML which is used to parameterize existing models. To achieve this, we use theories of critical complexity to both evaluate these grand narratives and contrast them with the Open ML approach. Specifically, we deconstruct grand ML ‘theories’ by identifying thirteen ‘games’ played in the ML community. These games lend false legitimacy to models, contribute to over-promise and hype about the capabilities of artificial intelligence, reduce wider participation in the subject, lead to models that exacerbate inequality and cause discrimination and ultimately stifle creativity in research. We argue that best practice in ML should be more consistent with critical complexity perspectives than with rationalist, grand narratives.

    Download full text (pdf)
    fulltext
  • 7.
    Biton, Shany
    et al.
    Faculty of Biomedical Engineering, Technion-IIT , Haifa, Israel.
    Gendelman, Sheina
    Faculty of Biomedical Engineering, Technion-IIT , Haifa, Israel.
    Horta Ribeiro, Antônio
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Miana, Gabriela
    Telehealth Center, Hospital das Clínicas , Belo Horizonte, Brazil;Department of Internal Medicine, Faculdade de Medicina, Universidade Federal de Minas Gerais , Belo Horizonte, Brazil.
    Moreira, Carla
    Telehealth Center, Hospital das Clínicas , Belo Horizonte, Brazil.
    Ribeiro, Antonio Luiz P
    Telehealth Center, Hospital das Clínicas , Belo Horizonte, Brazil;Department of Internal Medicine, Faculdade de Medicina, Universidade Federal de Minas Gerais , Belo Horizonte, Brazil.
    Behar, Joachim A
    Faculty of Biomedical Engineering, Technion-IIT , Haifa, Israel.
    Atrial fibrillation risk prediction from the 12-lead electrocardiogram using digital biomarkers and deep representation learning2021In: The European Heart Journal - Digital Health, E-ISSN 2634-3916, Vol. 2, no 4, p. 576-585Article in journal (Refereed)
  • 8.
    Bånkestad, Maria
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. RISE Research Institutes of Sweden.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Taghia, Jalil
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Variational Elliptical Processes2023In: Transactions on Machine Learning Research, E-ISSN 2835-8856Article in journal (Refereed)
    Abstract [en]

    We present elliptical processes—a family of non-parametric probabilistic models that subsumes Gaussian processes and Student's t processes. This generalization includes a range of new heavy-tailed behaviors while retaining computational tractability. Elliptical processes are based on a representation of elliptical distributions as a continuous mixture of Gaussian distributions. We parameterize this mixture distribution as a spline normalizing flow, which we train using variational inference. The proposed form of the variational posterior enables a sparse variational elliptical process applicable to large-scale problems. We highlight advantages compared to Gaussian processes through regression and classification experiments. Elliptical processes can supersede Gaussian processes in several settings, including cases where the likelihood is non-Gaussian or when accurate tail modeling is essential.

    Download full text (pdf)
    Fulltext
  • 9.
    Carlsson, Håkan
    et al.
    KTH Royal Inst Technol, Dept Intelligent Syst, S-11428 Stockholm, Sweden..
    Skog, Isaac
    Linköping Univ LIU, Dept Elect Engn, S-58183 Linköping, Sweden..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Jaldén, Joakim
    KTH Royal Inst Technol, Dept Intelligent Syst, S-11428 Stockholm, Sweden..
    Quantifying the Uncertainty of the Relative Geometry in Inertial Sensors Arrays2021In: IEEE Sensors Journal, ISSN 1530-437X, E-ISSN 1558-1748, Vol. 21, no 17, p. 19362-19373Article in journal (Refereed)
    Abstract [en]

    We present an algorithm to estimate and quantify the uncertainty of the accelerometers' relative geometry in an inertial sensor array. We formulate the calibration problem as a Bayesian estimation problem and propose an algorithm that samples the accelerometer positions' posterior distribution using Markov chain Monte Carlo. By identifying linear substructures of the measurement model, the unknown linear motion parameters are analytically marginalized, and the remaining non-linear motion parameters are numerically marginalized. The numerical marginalization occurs in a low dimensional space where the gyroscopes give information about the motion. This combination of information from gyroscopes and analytical marginalization allows the user to make no assumptions of the motion before the calibration. It thus enables the user to estimate the accelerometer positions' relative geometry by simply exposing the array to arbitrary twisting motion. We show that the calibration algorithm gives good results on both simulated and experimental data, despite sampling a high dimensional space.

    Download full text (pdf)
    fulltext
  • 10. Cheng, Liang
    et al.
    Frankemölle, Jonas
    Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Arts, Department of ALM.
    Axelsson, Adam
    Vats, Ekta
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Uncovering the Handwritten Text in the Margins: End-to-end Handwritten Text Detection and Recognition2024Conference paper (Refereed)
  • 11. Conde, Marcus V.
    et al.
    Luo, Ziwei
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Gustafsson, Fredrik K.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Zhao, Zheng
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Niu, JiXiang
    Lens-to-Lens Bokeh Effect Transformation: NTIRE 2023 Challenge Report2023In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Vancover: Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 1643-1659Conference paper (Other academic)
    Abstract [en]

    We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties. The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups. To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.

  • 12.
    Corenflos, Adrien
    et al.
    Aalto Univ, Dept Elect Engn & Automat, Espoo, Finland..
    Zhao, Zheng
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Särkkä, Simo
    Aalto Univ, Dept Elect Engn & Automat, Espoo, Finland..
    Temporal Gaussian Process Regression in Logarithmic Time2022In: 2022 25th International Conference on Information Fusion (FUSION), IEEE, 2022Conference paper (Refereed)
    Abstract [en]

    The aim of this article is to present a novel parallelization method for temporal Gaussian process (GP) regression problems. The method allows for solving GP regression problems in logarithmic O (log N) time, where N stands for the number of observations and test points. Our approach uses the state-space representation of GPs which, in its original form, allows for linear O(N) time GP regression by leveraging Kalman filtering and smoothing methods. By using a recently proposed parallelization method for Bayesian filters and smoothers, we are able to reduce the linear computational complexity of the temporal GP regression problems into logarithmic span complexity. This ensures logarithmic time complexity when parallel hardware such as a graphics processing unit (GPU) are employed. We experimentally show the computational benefits of our approach on simulated and real datasets via our open-source implementation leveraging the GPflow framework.

  • 13.
    Courts, Jarrad
    et al.
    Univ Newcastle, Fac Engn & Built Environm, Sch Engn, Callaghan, NSW 2308, Australia..
    Hendriks, Johannes
    Univ Newcastle, Fac Engn & Built Environm, Sch Engn, Callaghan, NSW 2308, Australia..
    Wills, Adrian
    Univ Newcastle, Fac Engn & Built Environm, Sch Engn, Callaghan, NSW 2308, Australia..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Ninness, Brett
    Univ Newcastle, Fac Engn & Built Environm, Sch Engn, Callaghan, NSW 2308, Australia..
    Variational State and Parameter Estimation2021In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 732-737Conference paper (Refereed)
    Abstract [en]

    This paper considers the problem of computing Bayesian estimates of both states and model parameters for nonlinear state-space models. Generally, this problem does not have a tractable solution and approximations must be utilised. In this work, a variational approach is used to provide an assumed density which approximates the desired, intractable, distribution. The approach is deterministic and results in an optimisation problem of a standard form. Due to the parametrisation of the assumed density selected first- and second-order derivatives are readily available which allows for efficient solutions. The proposed method is compared against state-of-the-art Hamiltonian Monte Carlo in two numerical examples.

    Download full text (pdf)
    fulltext
  • 14.
    Courts, Jarrad
    et al.
    Univ Newcastle, Fac Engn & Built Environm, Callaghan, NSW 2308, Australia..
    Wills, Adrian
    Univ Newcastle, Fac Engn & Built Environm, Callaghan, NSW 2308, Australia..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Gaussian Variational State Estimation for Nonlinear State-Space Models2021In: IEEE Transactions on Signal Processing, ISSN 1053-587X, E-ISSN 1941-0476, Vol. 69, p. 5979-5993Article in journal (Refereed)
    Abstract [en]

    In this paper, the problem of state estimation, in the context of both filtering and smoothing, for nonlinear state-space models is considered. Due to the nonlinear nature of the models, the state estimation problem is generally intractable as it involves integrals of general nonlinear functions and the filtered and smoothed state distributions lack closed-form solutions. As such, it is common to approximate the state estimation problem. In this paper, we develop an assumed Gaussian solution based on variational inference, which offers the key advantage of a flexible, but principled, mechanism for approximating the required distributions. Our main contribution lies in a new formulation of the state estimation problem as an optimisation problem, which can then be solved using standard optimisation routines that employ exact first- and second-order derivatives. The resulting state estimation approach involves a minimal number of assumptions and applies directly to nonlinear systems with both Gaussian and non-Gaussian probabilistic models. The performance of our approach is demonstrated on several examples; a challenging scalar system, a model of a simple robotic system, and a target tracking problem using a von Mises-Fisher distribution and outperforms alternative assumed Gaussian approaches to state estimation.

  • 15.
    Ek, Sofia
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Zachariah, Dave
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Johansson, Fredrik D.
    Stoica, Peter
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Off-Policy Evaluation with Out-of-Sample Guarantees2023In: Transactions on Machine Learning Research, E-ISSN 2835-8856Article in journal (Refereed)
  • 16.
    Ek, Sofia
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Zachariah, Dave
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Stoica, Peter
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control.
    Learning Pareto-Efficient Decisions with Confidence2022In: International Conference on Artificial Intelligence and Statistics / [ed] Camps-Valls, G Ruiz, FJR Valera, I, JMLR-JOURNAL MACHINE LEARNING RESEARCH , 2022, Vol. 151, p. 9969-9981Conference paper (Refereed)
    Abstract [en]

    The paper considers the problem of multi-objective decision support when outcomes are uncertain. We extend the concept of Pareto-efficient decisions to take into account the uncertainty of decision outcomes across varying contexts. This enables quantifying trade-offs between decisions in terms of tail outcomes that are relevant in safety-critical applications. We propose a method for learning efficient decisions with statistical confidence, building on results from the conformal prediction literature. The method adapts to weak or nonexistent context covariate overlap and its statistical guarantees are evaluated using both synthetic and real data.

  • 17. Emzir, Muhammad Fuady
    et al.
    Zhao, Zheng
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Särkkä, Simo
    Multidimensional projection filters via automatic differentiation and sparse-grid integration2023In: Signal Processing, ISSN 0165-1684, E-ISSN 1872-7557, Vol. 204, article id 108832Article in journal (Refereed)
    Abstract [en]

    The projection filter is a technique for approximating the solutions of optimal filtering problems. In projection filters, the Kushner–Stratonovich stochastic partial differential equation that governs the propagation of the optimal filtering density is projected to a manifold of parametric densities, resulting in a finite-dimensional stochastic differential equation. Despite the fact that projection filters are capable of representing complicated probability densities, their current implementations are limited to Gaussian family or unidimensional filtering applications. This work considers a combination of numerical integration and automatic differentiation to construct projection filter algorithms for more generic problems. Specifically, we provide a detailed exposition of this combination for the manifold of the exponential family, and show how to apply the projection filter to multidimensional cases. We demonstrate numerically that based on comparison to a finite-difference solution to the Kushner–Stratonovich equation and a bootstrap particle filter with systematic resampling, the proposed algorithm retains an accurate approximation of the filtering density while requiring a comparatively low number of quadrature points. Due to the sparse-grid integration and automatic differentiation used to calculate the expected values of the natural statistics and the Fisher metric, the proposed filtering algorithms are highly scalable. They therefore are suitable to many applications in which the number of dimensions exceeds the practical limit of particle filters, but where the Gaussian-approximations are deemed unsatisfactory.

  • 18. Fay, Dominik
    et al.
    Magnússon, Sindri
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Johansson, Mikael
    Adaptive Hyperparameter Selection for Differentially Private Gradient Descent2023In: Transactions on Machine Learning Research, E-ISSN 2835-8856Article in journal (Refereed)
    Abstract [en]

    We present an adaptive mechanism for hyperparameter selection in differentially private optimization that addresses the inherent trade-off between utility and privacy. The mechanism eliminates the often unstructured and time-consuming manual effort of selecting hyperparameters and avoids the additional privacy costs that hyperparameter selection otherwise incurs on top of that of the actual algorithm.

    We instantiate our mechanism for noisy gradient descent on non-convex, convex and strongly convex loss functions, respectively, to derive schedules for the noise variance and step size. These schedules account for the properties of the loss function and adapt to convergence metrics such as the gradient norm. When using these schedules, we show that noisy gradient descent converges at essentially the same rate as its noise-free counterpart. Numerical experiments show that the schedules consistently perform well across a range of datasets without manual tuning.

    Download full text (pdf)
    fulltext
  • 19. Fay, Dominik
    et al.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Oechtering, Tobias J.
    Private Learning Via Knowledge Transfer with High-Dimensional Targets2022In: ICASSP 2022: 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 3873-3877Conference paper (Refereed)
    Abstract [en]

    Preventing unintentional leakage of information about the training set has high relevance for many machine learning tasks, such as medical image segmentation. While differential privacy (DP) offers mathematically rigorous protection, the high output dimensionality of segmentation tasks prevents the direct application of state-of-the-art algorithms such as Private Aggregation of Teacher Ensembles (PATE). In order to alleviate this problem, we propose to learn dimensionality-reducing transformations to map the prediction target into a bounded lower-dimensional space to reduce the required noise level during the aggregation stage. To this end, we assess the suitability of principal component analysis (PCA) and autoencoders. We conclude that autoencoders are an effective means to reduce the noise in the target variables.

  • 20.
    Ferizbegovic, Mina
    et al.
    KTH Royal Inst Technol, Sch Elect Engn & Comp Sci, Div Decis & Control Syst, SE-10044 Stockholm, Sweden..
    Hjalmarsson, Hakan
    KTH Royal Inst Technol, Sch Elect Engn & Comp Sci, Div Decis & Control Syst, SE-10044 Stockholm, Sweden..
    Mattsson, Per
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Willems' fundamental lemma based on second-order moments2021In: 2021 60th IEEE Conference On Decision And Control (CDC), Institute of Electrical and Electronics Engineers (IEEE) Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 396-401Conference paper (Refereed)
    Abstract [en]

    In this paper, we propose variations of Willems' fundamental lemma that utilize second-order moments such as correlation functions in the time domain and power spectra in the frequency domain. We believe that using a formulation with estimated correlation coefficients is suitable for data compression, and possibly can reduce noise. Also, the formulations in the frequency domain can enable modeling of a system in a frequency region of interest.

  • 21.
    Ferizbegovic, Mina
    et al.
    KTH Royal Inst Technol, Div Decis & Control Syst, Stockholm, Sweden..
    Mattsson, Per
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Hjalmarsson, Håkan
    KTH Royal Inst Technol, Div Decis & Control Syst, Stockholm, Sweden..
    Bayes Control of Hammerstein Systems2021In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 755-760Conference paper (Refereed)
    Abstract [en]

    In this paper, we consider data driven control of Hammerstein systems. For such systems a common control structure is a transfer function followed by a static output nonlinearity that tries to cancel the input nonlinearity of the system, which is modeled as a polynomial or piece-wise linear function. The linear part of the controller is used to achieve desired disturbance rejection and tracking properties. To design a linear part of the controller, we propose a weighted average risk criterion with the risk being the average of the squared L2 tracking error. Here the average is with respect to the observations used in the controller and the weighting is with respect to how important it is to have good control for different impulse responses. This criterion corresponds to the average risk criterion leading to the Bayes estimator and we therefore call this approach Bayes control. By parametrizing the weighting function and estimating the corresponding hyperparameters we tune the weighting function to the information regarding the true impulse response contained in the data set available to the user for the control design. The numerical results show that the proposed methods result in stable controllers with performance comparable to the optimal controller, designed using the true input nonlinearity and true plant.

    Download full text (pdf)
    fulltext
  • 22.
    Ferizbegovic, Mina
    et al.
    KTH Royal Inst Technol, Sch Elect Engn & Comp Sci, Div Decis & Control Syst, S-10044 Stockholm, Sweden..
    Umenberger, Jack
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Hjalmarsson, Hakan
    KTH Royal Inst Technol, Sch Elect Engn & Comp Sci, Div Decis & Control Syst, S-10044 Stockholm, Sweden..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Learning Robust LQ-Controllers Using Application Oriented Exploration2020In: IEEE Control Systems Letters, E-ISSN 2475-1456, Vol. 4, no 1, p. 19-24Article in journal (Refereed)
    Abstract [en]

    This letter concerns the problem of learning robust LQ-controllers, when the dynamics of the linear system are unknown. First, we propose a robust control synthesis method to minimize the worst-case LQ cost, with probability 1 - delta, given empirical observations of the system. Next, we propose an approximate dual controller that simultaneously regulates the system and reduces model uncertainty. The objective of the dual controller is to minimize the worst-case cost attained by a new robust controller, synthesized with the reduced model uncertainty. The dual controller is subject to an exploration budget in the sense that it has constraints on its worst-case cost with respect to the current model uncertainty. In our numerical experiments, we observe better performance of the proposed robust LQ regulator over the existing methods. Moreover, the dual control strategy gives promising results in comparison with the common greedy random exploration strategies.

  • 23.
    Gedon, Daniel
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    On Deep Learning for Low-Dimensional Representations2024Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    In science and engineering, we are often concerned with creating mathematical models from data. These models are abstractions of observed real-world processes where the goal is often to understand these processes or to use the models to predict future instances of the observed process. Natural processes often exhibit low-dimensional structures which we can embed into the model. In mechanistic models, we directly include this structure into the model through mathematical equations often inspired by physical constraints. In contrast, within machine learning and particularly in deep learning we often deal with high-dimensional data such as images and learn a model without imposing a low-dimensional structure. Instead, we learn some kind of representations that are useful for the task at hand. While representation learning arguably enables the power of deep neural networks, it is less clear how to understand real-world processes from these models or whether we can benefit from including a low-dimensional structure in the model.

    Learning from data with intrinsic low-dimensional structure and how to replicate this structure in machine learning models is studied within this dissertation. While we put specific emphasis on deep neural networks, we also consider kernel machines in the context of Gaussian processes, as well as linear models, for example by studying the generalisation of models with an explicit low-dimensional structure. First, we argue that many real-world observations have an intrinsic low-dimensional structure. We can find evidence of this structure for example through low-rank approximations of many real-world data sets. Then, we face two open-ended research questions. First, we study the behaviour of machine learning models when they are trained on data with low-dimensional structures. Here we investigate fundamental aspects of learning low-dimensional representations and how well models with explicit low-dimensional structures perform. Second, we focus on applications in the modelling of dynamical systems and the medical domain. We investigate how we can benefit from low-dimensional representations for these applications and explore the potential of low-dimensional model structures for predictive tasks. Finally, we give a brief outlook on how we go beyond learning low-dimensional structures and identify the underlying mechanisms that generate the data to better model and understand these processes.

    This dissertation provides an overview of learning low-dimensional structures in machine learning models. It covers a wide range of topics from representation learning over the study of generalisation in overparameterized models to applications with time series and medical applications. However, each contribution opens up a range of questions to study in the future. Therefore this dissertation serves as a starting point to further explore learning of low-dimensional structure and representations.

    List of papers
    1. Uncertainty Estimation with Recursive Feature Machines
    Open this publication in new window or tab >>Uncertainty Estimation with Recursive Feature Machines
    (English)Manuscript (preprint) (Other academic)
    Abstract [en]

    In conventional regression analysis, predictions are typically represented as point estimates derived from covariates. The Gaussian Process (GP) offer a kernel-based framework that predicts and additionally quantifies associated uncertainties. However, kernel-based methods often underperform ensemble-based decision tree approaches in regression tasks involving tabular and categorical data. Recently, Recursive Feature Machines (RFMs) were proposed as a novel feature-learning kernel which strengthens the capabilities of kernel machines. In this study, we harness the power RFMs in a probabilistic GP-based approach to enhance uncertainty estimation through feature extraction within kernel methods. We employ this learned kernel for in-depth uncertainty analysis. On tabular datasets, our RFM-based method surpasses other leading uncertainty estimation techniques, including NGBoost and CatBoost-ensemble. Additionally, when assessing out-of-distribution performance, we found that boosting-based methods are surpassed by our RFM-based approach.

    National Category
    Other Electrical Engineering, Electronic Engineering, Information Engineering
    Identifiers
    urn:nbn:se:uu:diva-526129 (URN)
    Available from: 2024-04-04 Created: 2024-04-04 Last updated: 2024-04-07
    2. Invertible Kernel PCA With Random Fourier Features
    Open this publication in new window or tab >>Invertible Kernel PCA With Random Fourier Features
    2023 (English)In: IEEE Signal Processing Letters, ISSN 1070-9908, E-ISSN 1558-2361, Vol. 30, p. 563-567Article in journal (Refereed) Published
    Abstract [en]

    Kernel principal component analysis (kPCA) is a widely studied method to construct a low-dimensional data representation after a nonlinear transformation. The prevailing method to reconstruct the original input signal from kPCA-an important task for denoising-requires us to solve a supervised learning problem. In this paper, we present an alternative method where the reconstruction follows naturally from the compression step. We first approximate the kernel with random Fourier features. Then, we exploit the fact that the nonlinear transformation is invertible in a certain subdomain. Hence, the name invertible kernel PCA (ikPCA). We experiment with different data modalities and show that ikPCA performs similarly to kPCA with supervised reconstruction on denoising tasks, making it a strong alternative.

    Place, publisher, year, edition, pages
    Institute of Electrical and Electronics Engineers (IEEE), 2023
    Keywords
    Principal component analysis, Kernel, Image reconstruction, Dimensionality reduction, Noise reduction, Electrocardiography, Toy manufacturing industry, Denoising, ECG, Index Terms, kernel PCA, pre-image, random Fourier features, reconstruction
    National Category
    Signal Processing
    Identifiers
    urn:nbn:se:uu:diva-507434 (URN)10.1109/LSP.2023.3275499 (DOI)001010346600002 ()
    Funder
    Knut and Alice Wallenberg FoundationSwedish Research Council, 202104321
    Available from: 2023-07-11 Created: 2023-07-11 Last updated: 2024-04-07Bibliographically approved
    3. No Double Descent in Principal Component Regression: A High-Dimensional Analysis
    Open this publication in new window or tab >>No Double Descent in Principal Component Regression: A High-Dimensional Analysis
    (English)Manuscript (preprint) (Other academic)
    Abstract [en]

    Understanding the generalization properties of large-scale models necessitates incorporating realistic data assumptions into the analysis. Therefore, we consider Principal Component Regression (PCR)---combining principal component analysis and linear regression---on data from a low-dimensional manifold. We present an analysis of PCR when the data is sampled from a spiked covariance model, obtaining fundamental asymptotic guarantees for the generalization risk of this model. Our analysis is based on random matrix theory and allows us to provide guarantees for high-dimensional data. We additionally present an analysis of the distribution shift between training and test data. The results allow us to disentangle the effects of (1) the number of parameters, (2) the data-generating model and, (3) model misspecification on the generalization risk. The use of PCR effectively regularizes the model and prevents the interpolation peak of the double descent. Our theoretical findings are empirically validated in simulation, demonstrating their practical relevance.

    National Category
    Probability Theory and Statistics
    Identifiers
    urn:nbn:se:uu:diva-526128 (URN)
    Available from: 2024-04-04 Created: 2024-04-04 Last updated: 2024-04-15Bibliographically approved
    4. Deep State Space Models for Nonlinear System Identification
    Open this publication in new window or tab >>Deep State Space Models for Nonlinear System Identification
    2021 (English)In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 481-486Conference paper, Published paper (Refereed)
    Abstract [en]

    Deep state space models (SSMs) are an actively researched model class for temporal models developed in the deep learning community which have a close connection to classic SSMs. The use of deep SSMs as a black-box identification model can describe a wide range of dynamics due to the flexibility of deep neural networks. Additionally, the probabilistic nature of the model class allows the uncertainty of the system to be modelled. In this work a deep SSM class and its parameter learning algorithm are explained in an effort to extend the toolbox of nonlinear identification methods with a deep learning based method. Six recent deep SSMs are evaluated in a first unified implementation on nonlinear system identification benchmarks.

    Place, publisher, year, edition, pages
    ElsevierElsevier BV, 2021
    Keywords
    Nonlinear system identification, black box modeling, deep learning
    National Category
    Control Engineering
    Identifiers
    urn:nbn:se:uu:diva-457741 (URN)10.1016/j.ifacol.2021.08.406 (DOI)000696396200083 ()
    Conference
    19th IFAC Symposium on System Identification (SYSID), JUL 13-16, 2021, Padova, ITALY
    Funder
    Knut and Alice Wallenberg FoundationSwedish Research Council, 2016-06079Swedish Research Council, 2019-04956Wallenberg AI, Autonomous Systems and Software Program (WASP)
    Available from: 2021-11-12 Created: 2021-11-12 Last updated: 2024-04-07Bibliographically approved
    5. First Steps Towards Self-Supervised Pretraining of the 12-Lead ECG
    Open this publication in new window or tab >>First Steps Towards Self-Supervised Pretraining of the 12-Lead ECG
    2021 (English)In: 2021 Computing In Cardiology (CINC), Institute of Electrical and Electronics Engineers (IEEE), 2021Conference paper, Published paper (Refereed)
    Abstract [en]

    Self-supervised learning is a paradigm that extracts general features which describe the input space by artificially generating labels from the input without the need for explicit annotations. The learned features can then be used by transfer learning to boost the performance on a downstream task. Such methods have recently produced state of the art results in natural language processing and computer vision. Here, we propose a self-supervised learning method for 12-lead electrocardiograms (ECGs). For pretraining the model we design a task to mask out subsegements of all channels of the input signals and try to predict the actual values. As the model architecture, we use a U-ResNet containing an encoder-decoder structure. We test our method by self-supervised pretraining on the CODE dataset and then transfer the learnt features by finetuning on the PTBXL and CPSC benchmarks to evaluate the effect of our method in the classification of 12-leads ECGs. The method does provide modest improvements in performance when compared to not using pretraining. In future work we will make use of these ideas in smaller dataset, where we believe it can lead to larger performance gains.

    Place, publisher, year, edition, pages
    Institute of Electrical and Electronics Engineers (IEEE), 2021
    Series
    Computing in Cardiology Conference, ISSN 2325-8861, E-ISSN 2325-887X
    National Category
    Computer Sciences
    Identifiers
    urn:nbn:se:uu:diva-481355 (URN)10.23919/CinC53138.2021.9662748 (DOI)000821955000067 ()978-1-6654-7916-5 (ISBN)
    Conference
    Conference on Computing in Cardiology (CinC), SEP 12-15, 2021, Brno, CZECH REPUBLIC
    Funder
    Knut and Alice Wallenberg Foundation
    Available from: 2022-08-09 Created: 2022-08-09 Last updated: 2024-04-07Bibliographically approved
    6. Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients
    Open this publication in new window or tab >>Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients
    Show others...
    2022 (English)In: Scientific Reports, E-ISSN 2045-2322, Vol. 12, article id 19615Article in journal (Refereed) Published
    Abstract [en]

    Myocardial infarction diagnosis is a common challenge in the emergency department. In managed settings, deep learning-based models and especially convolutional deep models have shown promise in electrocardiogram (ECG) classification, but there is a lack of high-performing models for the diagnosis of myocardial infarction in real-world scenarios. We aimed to train and validate a deep learning model using ECGs to predict myocardial infarction in real-world emergency department patients. We studied emergency department patients in the Stockholm region between 2007 and 2016 that had an ECG obtained because of their presenting complaint. We developed a deep neural network based on convolutional layers similar to a residual network. Inputs to the model were ECG tracing, age, and sex; and outputs were the probabilities of three mutually exclusive classes: non-ST-elevation myocardial infarction (NSTEMI), ST-elevation myocardial infarction (STEMI), and control status, as registered in the SWEDEHEART and other registries. We used an ensemble of five models. Among 492,226 ECGs in 214,250 patients, 5,416 were recorded with an NSTEMI, 1,818 a STEMI, and 485,207 without a myocardial infarction. In a random test set, our model could discriminate STEMIs/NSTEMIs from controls with a C-statistic of 0.991/0.832 and had a Brier score of 0.001/0.008. The model obtained a similar performance in a temporally separated test set of the study sample, and achieved a C-statistic of 0.985 and a Brier score of 0.002 in discriminating STEMIs from controls in an external test set. We developed and validated a deep learning model with excellent performance in discriminating between control, STEMI, and NSTEMI on the presenting ECG of a real-world sample of the important population of all-comers to the emergency department. Hence, deep learning models for ECG decision support could be valuable in the emergency department.

    Place, publisher, year, edition, pages
    Springer Nature, 2022
    National Category
    Cardiac and Cardiovascular Systems
    Identifiers
    urn:nbn:se:uu:diva-489599 (URN)10.1038/s41598-022-24254-x (DOI)000885139000003 ()36380048 (PubMedID)
    Funder
    Knut and Alice Wallenberg FoundationEU, Horizon 2020, 101054643Swedish Research Council, sens2020005Swedish Research Council, sens2020598Swedish Research Council, 2018-05973Uppsala UniversityKjell and Marta Beijer Foundation
    Available from: 2022-12-02 Created: 2022-12-02 Last updated: 2024-04-07Bibliographically approved
    Download full text (pdf)
    UUThesis_D-Gedon-2024
    Download (jpg)
    preview image
  • 24.
    Gedon, Daniel
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Abedsoltan, Amirhesam
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Belkin, Mikhail
    Uncertainty Estimation with Recursive Feature MachinesManuscript (preprint) (Other academic)
    Abstract [en]

    In conventional regression analysis, predictions are typically represented as point estimates derived from covariates. The Gaussian Process (GP) offer a kernel-based framework that predicts and additionally quantifies associated uncertainties. However, kernel-based methods often underperform ensemble-based decision tree approaches in regression tasks involving tabular and categorical data. Recently, Recursive Feature Machines (RFMs) were proposed as a novel feature-learning kernel which strengthens the capabilities of kernel machines. In this study, we harness the power RFMs in a probabilistic GP-based approach to enhance uncertainty estimation through feature extraction within kernel methods. We employ this learned kernel for in-depth uncertainty analysis. On tabular datasets, our RFM-based method surpasses other leading uncertainty estimation techniques, including NGBoost and CatBoost-ensemble. Additionally, when assessing out-of-distribution performance, we found that boosting-based methods are surpassed by our RFM-based approach.

  • 25.
    Gedon, Daniel
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Horta Ribeiro, Antônio
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    No Double Descent in Principal Component Regression: A High-Dimensional AnalysisManuscript (preprint) (Other academic)
    Abstract [en]

    Understanding the generalization properties of large-scale models necessitates incorporating realistic data assumptions into the analysis. Therefore, we consider Principal Component Regression (PCR)---combining principal component analysis and linear regression---on data from a low-dimensional manifold. We present an analysis of PCR when the data is sampled from a spiked covariance model, obtaining fundamental asymptotic guarantees for the generalization risk of this model. Our analysis is based on random matrix theory and allows us to provide guarantees for high-dimensional data. We additionally present an analysis of the distribution shift between training and test data. The results allow us to disentangle the effects of (1) the number of parameters, (2) the data-generating model and, (3) model misspecification on the generalization risk. The use of PCR effectively regularizes the model and prevents the interpolation peak of the double descent. Our theoretical findings are empirically validated in simulation, demonstrating their practical relevance.

  • 26.
    Gedon, Daniel
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Horta Ribeiro, Antônio
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Wahlström, Niklas
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    First Steps Towards Self-Supervised Pretraining of the 12-Lead ECG2021In: 2021 Computing In Cardiology (CINC), Institute of Electrical and Electronics Engineers (IEEE), 2021Conference paper (Refereed)
    Abstract [en]

    Self-supervised learning is a paradigm that extracts general features which describe the input space by artificially generating labels from the input without the need for explicit annotations. The learned features can then be used by transfer learning to boost the performance on a downstream task. Such methods have recently produced state of the art results in natural language processing and computer vision. Here, we propose a self-supervised learning method for 12-lead electrocardiograms (ECGs). For pretraining the model we design a task to mask out subsegements of all channels of the input signals and try to predict the actual values. As the model architecture, we use a U-ResNet containing an encoder-decoder structure. We test our method by self-supervised pretraining on the CODE dataset and then transfer the learnt features by finetuning on the PTBXL and CPSC benchmarks to evaluate the effect of our method in the classification of 12-leads ECGs. The method does provide modest improvements in performance when compared to not using pretraining. In future work we will make use of these ideas in smaller dataset, where we believe it can lead to larger performance gains.

  • 27.
    Gedon, Daniel
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Ribeiro, Antônio H.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Wahlström, Niklas
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Invertible Kernel PCA With Random Fourier Features2023In: IEEE Signal Processing Letters, ISSN 1070-9908, E-ISSN 1558-2361, Vol. 30, p. 563-567Article in journal (Refereed)
    Abstract [en]

    Kernel principal component analysis (kPCA) is a widely studied method to construct a low-dimensional data representation after a nonlinear transformation. The prevailing method to reconstruct the original input signal from kPCA-an important task for denoising-requires us to solve a supervised learning problem. In this paper, we present an alternative method where the reconstruction follows naturally from the compression step. We first approximate the kernel with random Fourier features. Then, we exploit the fact that the nonlinear transformation is invertible in a certain subdomain. Hence, the name invertible kernel PCA (ikPCA). We experiment with different data modalities and show that ikPCA performs similarly to kPCA with supervised reconstruction on denoising tasks, making it a strong alternative.

  • 28.
    Gedon, Daniel
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Wahlström, Niklas
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Ljung, Lennart
    Linköping Univ, Div Automat Control, Linköping, Sweden.
    Deep State Space Models for Nonlinear System Identification2021In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 481-486Conference paper (Refereed)
    Abstract [en]

    Deep state space models (SSMs) are an actively researched model class for temporal models developed in the deep learning community which have a close connection to classic SSMs. The use of deep SSMs as a black-box identification model can describe a wide range of dynamics due to the flexibility of deep neural networks. Additionally, the probabilistic nature of the model class allows the uncertainty of the system to be modelled. In this work a deep SSM class and its parameter learning algorithm are explained in an effort to extend the toolbox of nonlinear identification methods with a deep learning based method. Six recent deep SSMs are evaluated in a first unified implementation on nonlinear system identification benchmarks.

    Download full text (pdf)
    fulltext
  • 29.
    Geng, Li-Hui
    et al.
    Tianjin Key Laboratory of Information Sensing and Intelligent Control, School of Automation and Electrical Engineering, Tianjin University of Technology and Education, Tianjin, China.
    Wills, Adrian George
    School of Engineering, The University of Newcastle, Callaghan, NSW, Australia.
    Ninness, Brett
    Faculty of Engineering and Built Environment, The University of Newcastle, Callaghan, NSW, Australia.
    Schön, Thomas Bo
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Smoothed State Estimation via Efficient Solution of Linear Equations2023In: IEEE Transactions on Automatic Control, ISSN 0018-9286, E-ISSN 1558-2523, Vol. 68, no 10, p. 5877-5889Article in journal (Refereed)
    Abstract [en]

    This article addresses the problem of computing fixed-interval smoothed state estimates of a linear time-varying Gaussian stochastic system. There already exist many algorithms that perform this computation, but all of them impose certain restrictions on system matrices in order for them to be applicable, and the restrictions vary considerably between the various existing algorithms. This article establishes a new sufficient condition for the fixed-interval smoothing density to exist in a Gaussian form that can be completely characterized by associated means and covariances. It then develops an algorithm to compute these means and covariances with no further assumptions required. This results in an algorithm more generally applicable than any one of the multitude of existing algorithms available to date.

  • 30.
    Glaser, Pierre
    et al.
    University College London, Gatsby Computational Neuroscience Unit, London, UK.
    Widmann, David
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Lindsten, Fredrik
    Linköping Univ, Div Stat & Machine Learning, Linköping, Sweden.
    Gretton, Arthur
    University College London, Gatsby Computational Neuroscience Unit, London, UK.
    Fast and Scalable Score-Based Kernel Calibration TestsManuscript (preprint) (Other academic)
    Abstract [en]

    We introduce the Kernel Calibration Conditional Stein Discrepancy test (KCCSD test), a non-parametric, kernel-based test for assessing the calibration of probabilistic models with well-defined scores. In contrast to previous methods, our test avoids the need for possibly expensive expectation approximations while providing control over its type-I error. We achieve these improvements by using a new family of kernels for score-based probabilities that can be estimated without probability density samples, and by using a conditional goodness-of-fit criterion for the KCCSD test’s U-statistic. The tractability of the KCCSD test widens the surface area of calibration measures to new promising use-cases, such as regularization during model training. We demonstrate the properties of our test on various synthetic settings.

  • 31. Gräfe, Alexander
    et al.
    Baumann, Dominik
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Trimpe, Sebastian
    Towards remote fault detection by analyzing communication priorities2022In: 2022 IEEE 61st Conference on Decision and Control (CDC), Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 1758-1763Conference paper (Refereed)
    Abstract [en]

    The ability to detect faults is an important safety feature for event-based multi-agent systems. In most existing algorithms, each agent tries to detect faults by checking its own behavior. But what if one agent becomes unable to recognize misbehavior, for example due to failure in its onboard fault detection? To improve resilience and avoid propagation of individual errors to the multi-agent system, agents should check each other remotely for malfunction or misbehavior. In this paper, we build upon a recently proposed predictive triggering architecture that involves communication priorities shared throughout the network to manage limited bandwidth. We propose a fault detection method that uses these priorities to detect errors in other agents. The resulting algorithms is not only able to detect faults, but can also run on a low-power microcontroller in real-time, as we demonstrate in hardware experiments.

  • 32.
    Gunnarsson, Niklas
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Kimstrand, Peter
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Latent linear dynamics in spatiotemporal medical dataManuscript (preprint) (Other academic)
  • 33.
    Gunnarsson, Niklas
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Kimstrand, Peter
    Elekta Instrument AB, Stockholm, Sweden..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Unsupervised dynamic modeling of medical image transformations2022In: 2022 25th International Conference on Information Fusion (FUSION 2022), Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 1-7Conference paper (Refereed)
    Abstract [en]

    Spatiotemporal imaging has applications in e.g. cardiac diagnostics, surgical guidance, and radiotherapy monitoring, In this paper, we explain the temporal motion by identifying the underlying dynamics, only based on the sequential images. Our dynamical model maps the inputs of observed high-dimensional sequential images to a low-dimensional latent space wherein a linear relationship between a hidden state process and the lower-dimensional representation of the inputs holds. For this, we use a conditional variational auto-encoder (CVAE) to nonlinearly map the higher dimensional image to a lower-dimensional space, wherein we model the dynamics with a linear Gaussian state-space model (LG-SSM). The model, a modified version of the Kalman variational auto-encoder, is end-to-end trainable, and the weights, both in the CVAE and LG-SSM, are simultaneously updated by maximizing the evidence lower bound of the marginal likelihood. In contrast to the original model, we explain the motion with a spatial transformation from one image to another. This results in sharper reconstructions and the possibility of transferring auxiliary information, such as segmentation, through the image sequence. Our experiments, on cardiac ultrasound time series, show that the dynamic model outperforms traditional image registration in execution time, to a similar performance. Further, our model offers the possibility to impute and extrapolate for missing samples.

  • 34.
    Gunnarsson, Niklas
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Elekta Instrument AB.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Elekta Instrument AB.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Learning a Deformable Registration Pyramid2021In: Segmentation, Classification, and Registration of Multi-modality Medical Imaging Data / [ed] Nadya Shusharina, Mattias P. Heinrich, Ruobing Huang, Springer Nature Springer Nature, 2021, Vol. 12587, p. 80-86Conference paper (Refereed)
    Abstract [en]

    We introduce an end-to-end unsupervised (or weakly supervised) image registration method that blends conventional medical image registration with contemporary deep learning techniques from computer vision. Our method downsamples both the fixed and the moving images into multiple feature map levels where a displacement field is estimated at each level and then further refined throughout the network. We train and test our model on three different datasets. In comparison with the initial registrations we find an improved performance using our model, yet we expect it would improve further if the model was fine-tuned for each task. The implementation is publicly available (https://github.com/ngunnar/learning-a-deformable-registration-pyramid).

    Download full text (pdf)
    fulltext
  • 35.
    Gunnarsson, Niklas
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Registration by tracking for sequential 2D MRIManuscript (preprint) (Other academic)
  • 36.
    Gustafsson, Fredrik K.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Towards Accurate and Reliable Deep Regression Models2023Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Regression is a fundamental machine learning task with many important applications within computer vision and other domains. In general, it entails predicting continuous targets from given inputs. Deep learning has become the dominant paradigm within machine learning in recent years, and a wide variety of different techniques have been employed to solve regression problems using deep models. There is however no broad consensus on how deep regression models should be constructed for best possible accuracy, or how the uncertainty in their predictions should be represented and estimated. 

    These open questions are studied in this thesis, aiming to help take steps towards an ultimate goal of developing deep regression models which are both accurate and reliable enough for real-world deployment within medical applications and other safety-critical domains.

    The first main contribution of the thesis is the formulation and development of energy-based probabilistic regression. This is a general and conceptually simple regression framework with a clear probabilistic interpretation, using energy-based models to represent the true conditional target distribution. The framework is applied to a number of regression problems and demonstrates particularly strong performance for 2D bounding box regression, improving the state-of-the-art when applied to the task of visual tracking.

    The second main contribution is a critical evaluation of various uncertainty estimation methods. A general introduction to the problem of estimating the predictive uncertainty of deep models is first provided, together with an extensive comparison of the two popular methods ensembling and MC-dropout. A number of regression uncertainty estimation methods are then further evaluated, specifically examining their reliability under real-world distribution shifts. This evaluation uncovers important limitations of current methods and serves as a challenge to the research community. It demonstrates that more work is required in order to develop truly reliable uncertainty estimation methods for regression.

    List of papers
    1. Energy-Based Models for Deep Probabilistic Regression
    Open this publication in new window or tab >>Energy-Based Models for Deep Probabilistic Regression
    2020 (English)Conference paper, Published paper (Refereed)
    National Category
    Computer Vision and Robotics (Autonomous Systems)
    Identifiers
    urn:nbn:se:uu:diva-492645 (URN)
    Conference
    European Conference on Computer Vision (ECCV)
    Available from: 2023-01-09 Created: 2023-01-09 Last updated: 2023-10-10Bibliographically approved
    2. How to Train Your Energy-Based Model for Regression
    Open this publication in new window or tab >>How to Train Your Energy-Based Model for Regression
    2020 (English)Conference paper, Published paper (Refereed)
    National Category
    Computer Vision and Robotics (Autonomous Systems)
    Identifiers
    urn:nbn:se:uu:diva-492644 (URN)
    Conference
    British Machine Vision Conference (BMVC)
    Available from: 2023-01-09 Created: 2023-01-09 Last updated: 2023-10-10Bibliographically approved
    3. Learning Proposals for Practical Energy-Based Regression
    Open this publication in new window or tab >>Learning Proposals for Practical Energy-Based Regression
    2022 (English)In: International conference on artificial intelligence and statistics, vol 151 / [ed] Camps-Valls, G Ruiz, FJR Valera, I, JMLR-JOURNAL MACHINE LEARNING RESEARCH , 2022, Vol. 151, p. 4685-4704Conference paper, Published paper (Refereed)
    Abstract [en]

    Energy-based models (EBMs) have experienced a resurgence within machine learning in recent years, including as a promising alternative for probabilistic regression. However, energy-based regression requires a proposal distribution to be manually designed for training, and an initial estimate has to be provided at test-time. We address both of these issues by introducing a conceptually simple method to automatically learn an effective proposal distribution, which is parameterized by a separate network head. To this end, we derive a surprising result, leading to a unified training objective that jointly minimizes the KL divergence from the proposal to the EBM, and the negative log-likelihood of the EBM. At test-time, we can then employ importance sampling with the trained proposal to efficiently evaluate the learned EBM and produce standalone predictions. Furthermore, we utilize our derived training objective to learn mixture density networks (MDNs) with a jointly trained energy-based teacher, consistently outperforming conventional MDN training on four real-world regression tasks within computer vision. Code is available at https://github.com/fregu856/ebms_proposals.

    Place, publisher, year, edition, pages
    JMLR-JOURNAL MACHINE LEARNING RESEARCH, 2022
    Series
    Proceedings of Machine Learning Research, ISSN 2640-3498
    National Category
    Business Administration
    Identifiers
    urn:nbn:se:uu:diva-482673 (URN)000828072704035 ()
    Conference
    International Conference on Artificial Intelligence and Statistics, MAR 28-30, 2022, ELECTR NETWORK
    Funder
    Swedish Foundation for Strategic Research, RIT15-0012Swedish Research Council, 621-2016-06079Kjell and Marta Beijer Foundation
    Available from: 2022-09-15 Created: 2022-09-15 Last updated: 2023-10-10Bibliographically approved
    4. Accurate 3D Object Detection using Energy-Based Models
    Open this publication in new window or tab >>Accurate 3D Object Detection using Energy-Based Models
    2021 (English)In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recogition Workshops (CVPRW 2021), Institute of Electrical and Electronics Engineers (IEEE) Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 2849-2858Conference paper, Published paper (Refereed)
    Abstract [en]

    Accurate 3D object detection (3DOD) is crucial for safe navigation of complex environments by autonomous robots. Regressing accurate 3D bounding boxes in cluttered environments based on sparse LiDAR data is however a highly challenging problem. We address this task by exploring recent advances in conditional energy-based models (EBMs) for probabilistic regression. While methods employing EBMs for regression have demonstrated impressive performance on 2D object detection in images, these techniques are not directly applicable to 3D bounding boxes. In this work, we therefore design a differentiable pooling operator for 3D bounding boxes, serving as the core module of our EBM network. We further integrate this general approach into the state-of-the-art 3D object detector SA-SSD. On the KITTI dataset, our proposed approach consistently outperforms the SA-SSD baseline across all 3DOD metrics, demonstrating the potential of EBM-based regression for highly accurate 3DOD. Code is available at https://github.com/fregu856/ebms_3dod.

    Place, publisher, year, edition, pages
    Institute of Electrical and Electronics Engineers (IEEE)Institute of Electrical and Electronics Engineers (IEEE), 2021
    Series
    IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, ISSN 2160-7508
    National Category
    Computer Vision and Robotics (Autonomous Systems)
    Identifiers
    urn:nbn:se:uu:diva-459209 (URN)10.1109/CVPRW53098.2021.00320 (DOI)000705890202105 ()978-1-6654-4899-4 (ISBN)
    Conference
    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), JUN 19-25, 2021, ELECTR NETWORK
    Funder
    Swedish Foundation for Strategic ResearchKnut and Alice Wallenberg FoundationKjell and Marta Beijer Foundation
    Available from: 2021-11-29 Created: 2021-11-29 Last updated: 2024-01-15Bibliographically approved
    5. Deep Energy-Based NARX Models
    Open this publication in new window or tab >>Deep Energy-Based NARX Models
    Show others...
    2021 (English)In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 505-510Conference paper, Published paper (Refereed)
    Abstract [en]

    This paper is directed towards the problem of learning nonlinear ARX models based on observed input output data. In particular, our interest is in learning a conditional distribution of the current output based on a finite window of past inputs and outputs. To achieve this, we consider the use of so-called energy-based models, which have been developed in allied fields for learning unknown distributions based on data. This energy-based model relies on a general function to describe the distribution, and here we consider a deep neural network for this purpose. The primary benefit of this approach is that it is capable of learning both simple and highly complex noise models, which we demonstrate on simulated and experimental data.

    Place, publisher, year, edition, pages
    ElsevierElsevier BV, 2021
    Keywords
    System Identification, Energy-Based Models, Deep Neural Networks
    National Category
    Computer Sciences
    Identifiers
    urn:nbn:se:uu:diva-457748 (URN)10.1016/j.ifacol.2021.08.410 (DOI)000696396200087 ()
    Conference
    19th IFAC Symposium on System Identification (SYSID), JUL 13-16, 2021, Padova, ITALY
    Funder
    Swedish Research CouncilKjell and Marta Beijer Foundation
    Available from: 2021-11-09 Created: 2021-11-09 Last updated: 2024-01-15Bibliographically approved
    6. Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
    Open this publication in new window or tab >>Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
    2020 (English)In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2020), IEEE Computer Society, 2020, p. 1289-1298Conference paper, Published paper (Refereed)
    Abstract [en]

    While deep neural networks have become the go-to approach in computer vision, the vast majority of these models fail to properly capture the uncertainty inherent in their predictions. Estimating this predictive uncertainty can be crucial, for example in automotive applications. In Bayesian deep learning, predictive uncertainty is commonly decomposed into the distinct types of aleatoric and epistemic uncertainty. The former can be estimated by letting a neural network output the parameters of a certain probability distribution. Epistemic uncertainty estimation is a more challenging problem, and while different scalable methods recently have emerged, no extensive comparison has been performed in a real-world setting. We therefore accept this task and propose a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning. Our proposed framework is specifically designed to test the robustness required in real-world computer vision applications. We also apply this framework to provide the first properly extensive and conclusive comparison of the two current state-of-the-art scalable methods: ensembling and MC-dropout. Our comparison demonstrates that ensembling consistently provides more reliable and practically useful uncertainty estimates. Code is available at https://github.com/fregu856/evaluating_bdl.

    Place, publisher, year, edition, pages
    IEEE Computer Society, 2020
    Series
    IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, ISSN 2160-7508, E-ISSN 2160-7516
    National Category
    Computer Vision and Robotics (Autonomous Systems)
    Identifiers
    urn:nbn:se:uu:diva-478393 (URN)10.1109/CVPRW50498.2020.00167 (DOI)000788279001046 ()978-1-7281-9360-1 (ISBN)
    Conference
    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), JUN 14-19, 2020, ELECTR NETWORK
    Funder
    Swedish Foundation for Strategic ResearchSwedish Research Council
    Available from: 2022-08-04 Created: 2022-08-04 Last updated: 2023-10-10Bibliographically approved
    7. How Reliable is Your Regression Model’s Uncertainty Under Real-World Distribution Shifts?
    Open this publication in new window or tab >>How Reliable is Your Regression Model’s Uncertainty Under Real-World Distribution Shifts?
    2023 (English)In: Transactions on Machine Learning Research, E-ISSN 2835-8856Article in journal (Refereed) Published
    National Category
    Computer Vision and Robotics (Autonomous Systems)
    Identifiers
    urn:nbn:se:uu:diva-513724 (URN)
    Available from: 2023-10-10 Created: 2023-10-10 Last updated: 2024-01-08Bibliographically approved
    8. ECG-Based Electrolyte Prediction: Evaluating Regression and Probabilistic Methods
    Open this publication in new window or tab >>ECG-Based Electrolyte Prediction: Evaluating Regression and Probabilistic Methods
    Show others...
    (English)Manuscript (preprint) (Other academic)
    National Category
    Signal Processing
    Identifiers
    urn:nbn:se:uu:diva-513725 (URN)
    Available from: 2023-10-10 Created: 2023-10-10 Last updated: 2023-12-24Bibliographically approved
    Download full text (pdf)
    UUThesis_Gustafsson,F-2023
    Download (jpg)
    presentationsbild
  • 37.
    Gustafsson, Fredrik K.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Danelljan, Martin
    Bhat, Goutam
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Energy-Based Models for Deep Probabilistic Regression2020Conference paper (Refereed)
  • 38.
    Gustafsson, Fredrik K.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Danelljan, Martin
    Swiss Fed Inst Technol, Comp Vis Lab, Zurich, Switzerland..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Accurate 3D Object Detection using Energy-Based Models2021In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recogition Workshops (CVPRW 2021), Institute of Electrical and Electronics Engineers (IEEE) Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 2849-2858Conference paper (Refereed)
    Abstract [en]

    Accurate 3D object detection (3DOD) is crucial for safe navigation of complex environments by autonomous robots. Regressing accurate 3D bounding boxes in cluttered environments based on sparse LiDAR data is however a highly challenging problem. We address this task by exploring recent advances in conditional energy-based models (EBMs) for probabilistic regression. While methods employing EBMs for regression have demonstrated impressive performance on 2D object detection in images, these techniques are not directly applicable to 3D bounding boxes. In this work, we therefore design a differentiable pooling operator for 3D bounding boxes, serving as the core module of our EBM network. We further integrate this general approach into the state-of-the-art 3D object detector SA-SSD. On the KITTI dataset, our proposed approach consistently outperforms the SA-SSD baseline across all 3DOD metrics, demonstrating the potential of EBM-based regression for highly accurate 3DOD. Code is available at https://github.com/fregu856/ebms_3dod.

  • 39.
    Gustafsson, Fredrik K.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Danelljan, Martin
    Swiss Fed Inst Technol, Comp Vis Lab, Zurich, Switzerland..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision2020In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2020), IEEE Computer Society, 2020, p. 1289-1298Conference paper (Refereed)
    Abstract [en]

    While deep neural networks have become the go-to approach in computer vision, the vast majority of these models fail to properly capture the uncertainty inherent in their predictions. Estimating this predictive uncertainty can be crucial, for example in automotive applications. In Bayesian deep learning, predictive uncertainty is commonly decomposed into the distinct types of aleatoric and epistemic uncertainty. The former can be estimated by letting a neural network output the parameters of a certain probability distribution. Epistemic uncertainty estimation is a more challenging problem, and while different scalable methods recently have emerged, no extensive comparison has been performed in a real-world setting. We therefore accept this task and propose a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning. Our proposed framework is specifically designed to test the robustness required in real-world computer vision applications. We also apply this framework to provide the first properly extensive and conclusive comparison of the two current state-of-the-art scalable methods: ensembling and MC-dropout. Our comparison demonstrates that ensembling consistently provides more reliable and practically useful uncertainty estimates. Code is available at https://github.com/fregu856/evaluating_bdl.

  • 40.
    Gustafsson, Fredrik K.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Danelljan, Martin
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    How Reliable is Your Regression Model’s Uncertainty Under Real-World Distribution Shifts?2023In: Transactions on Machine Learning Research, E-ISSN 2835-8856Article in journal (Refereed)
  • 41.
    Gustafsson, Fredrik K.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Danelljan, Martin
    Swiss Fed Inst Technol, Comp Vis Lab, Zurich, Switzerland..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Learning Proposals for Practical Energy-Based Regression2022In: International conference on artificial intelligence and statistics, vol 151 / [ed] Camps-Valls, G Ruiz, FJR Valera, I, JMLR-JOURNAL MACHINE LEARNING RESEARCH , 2022, Vol. 151, p. 4685-4704Conference paper (Refereed)
    Abstract [en]

    Energy-based models (EBMs) have experienced a resurgence within machine learning in recent years, including as a promising alternative for probabilistic regression. However, energy-based regression requires a proposal distribution to be manually designed for training, and an initial estimate has to be provided at test-time. We address both of these issues by introducing a conceptually simple method to automatically learn an effective proposal distribution, which is parameterized by a separate network head. To this end, we derive a surprising result, leading to a unified training objective that jointly minimizes the KL divergence from the proposal to the EBM, and the negative log-likelihood of the EBM. At test-time, we can then employ importance sampling with the trained proposal to efficiently evaluate the learned EBM and produce standalone predictions. Furthermore, we utilize our derived training objective to learn mixture density networks (MDNs) with a jointly trained energy-based teacher, consistently outperforming conventional MDN training on four real-world regression tasks within computer vision. Code is available at https://github.com/fregu856/ebms_proposals.

  • 42.
    Gustafsson, Fredrik K.
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Danelljan, Martin
    Timofte, Radu
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    How to Train Your Energy-Based Model for Regression2020Conference paper (Refereed)
  • 43.
    Gustafsson, Stefan
    et al.
    Uppsala University, Disciplinary Domain of Medicine and Pharmacy, Faculty of Medicine, Department of Medical Sciences, Clinical Epidemiology. Sence Res AB, Uppsala, Sweden.
    Gedon, Daniel
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Lampa, Erik
    Uppsala University, Disciplinary Domain of Medicine and Pharmacy, Faculty of Medicine, Department of Medical Sciences, Clinical Epidemiology.
    Horta Ribeiro, Antônio
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Holzmann, Martin J.
    Karolinska Univ Hosp, Funct Emergency Med, Stockholm, Sweden.;Karolinska Inst, Dept Med, Stockholm, Sweden.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Sundström, Johan
    Uppsala University, Disciplinary Domain of Medicine and Pharmacy, Faculty of Medicine, Department of Medical Sciences, Clinical Epidemiology. Univ New South Wales, George Inst Global Hlth, Sydney, NSW, Australia.
    Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients2022In: Scientific Reports, E-ISSN 2045-2322, Vol. 12, article id 19615Article in journal (Refereed)
    Abstract [en]

    Myocardial infarction diagnosis is a common challenge in the emergency department. In managed settings, deep learning-based models and especially convolutional deep models have shown promise in electrocardiogram (ECG) classification, but there is a lack of high-performing models for the diagnosis of myocardial infarction in real-world scenarios. We aimed to train and validate a deep learning model using ECGs to predict myocardial infarction in real-world emergency department patients. We studied emergency department patients in the Stockholm region between 2007 and 2016 that had an ECG obtained because of their presenting complaint. We developed a deep neural network based on convolutional layers similar to a residual network. Inputs to the model were ECG tracing, age, and sex; and outputs were the probabilities of three mutually exclusive classes: non-ST-elevation myocardial infarction (NSTEMI), ST-elevation myocardial infarction (STEMI), and control status, as registered in the SWEDEHEART and other registries. We used an ensemble of five models. Among 492,226 ECGs in 214,250 patients, 5,416 were recorded with an NSTEMI, 1,818 a STEMI, and 485,207 without a myocardial infarction. In a random test set, our model could discriminate STEMIs/NSTEMIs from controls with a C-statistic of 0.991/0.832 and had a Brier score of 0.001/0.008. The model obtained a similar performance in a temporally separated test set of the study sample, and achieved a C-statistic of 0.985 and a Brier score of 0.002 in discriminating STEMIs from controls in an external test set. We developed and validated a deep learning model with excellent performance in discriminating between control, STEMI, and NSTEMI on the presenting ECG of a real-world sample of the important population of all-comers to the emergency department. Hence, deep learning models for ECG decision support could be valuable in the emergency department.

    Download full text (pdf)
    FULLTEXT01
  • 44.
    Gyllingberg, Linnéa
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Mathematics.
    Tian, Yu
    Sumpter, David J. T.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Mathematics. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    A minimal model of cognition based on oscillatory and reinforcement processesManuscript (preprint) (Other academic)
    Abstract [en]

    Building mathematical models of brains is difficult because of the sheer complexity of the problem. One potential approach is to start by identifying models of basal cognition, which give an abstract representation of a range organisms without central nervous systems, including fungi, slime moulds and bacteria. We propose one such model, demonstrating how a combination of oscillatory and current-based reinforcement processes can be used to couple resources in an efficient manner. We first show that our model connects resources in an efficient manner when the environment is constant. We then show that in an oscillatory environment our model builds efficient solutions, provided the environmental oscillations are sufficiently out of phase. We show that amplitude differences can promote efficient solutions and that the system is robust to frequency differences. We identify connections between our model and basal cognition in biological systems and slime moulds, in particular, showing how oscillatory and problem-solving properties of these systems are captured by our model.

  • 45.
    Hendriks, Johannes N.
    et al.
    Univ Newcastle, Sch Engn, Callaghan, NSW, Australia..
    Gustafsson, Fredrik K.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Ribeiro, Antonio H.
    Univ Fed Minas Gerais, Dept Comp Sci, BR-31270901 Belo Horizonte, MG, Brazil..
    Wills, Adrian G.
    Univ Newcastle, Sch Engn, Callaghan, NSW, Australia..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Deep Energy-Based NARX Models2021In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 505-510Conference paper (Refereed)
    Abstract [en]

    This paper is directed towards the problem of learning nonlinear ARX models based on observed input output data. In particular, our interest is in learning a conditional distribution of the current output based on a finite window of past inputs and outputs. To achieve this, we consider the use of so-called energy-based models, which have been developed in allied fields for learning unknown distributions based on data. This energy-based model relies on a general function to describe the distribution, and here we consider a deep neural network for this purpose. The primary benefit of this approach is that it is capable of learning both simple and highly complex noise models, which we demonstrate on simulated and experimental data.

    Download full text (pdf)
    fulltext
  • 46.
    Hendriks, Johannes N.
    et al.
    Univ Newcastle, Sch Engn, Callaghan, NSW, Australia..
    Holdsworth, James R. Z.
    Univ Newcastle, Sch Engn, Callaghan, NSW, Australia..
    Wills, Adrian G.
    Univ Newcastle, Sch Engn, Callaghan, NSW, Australia..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Ninness, Brett
    Data to Controller for Nonlinear Systems: An Approximate Solution2022In: IEEE Control Systems Letters, E-ISSN 2475-1456, Vol. 6, p. 1196-1201Article in journal (Refereed)
    Abstract [en]

    This letter considers the problem of determining an optimal control action based on observed data. We formulate the problem assuming that the system can be modeled by a nonlinear state-space model, but where the model parameters, state and future disturbances are not known and are treated as random variables. Central to our formulation is that the joint distribution of these unknown objects is conditioned on the observed data. Crucially, as new measurements become available, this joint distribution continues to evolve so that control decisions are made accounting for uncertainty as evidenced in the data. The resulting problem is intractable which we obviate by providing approximations that result in finite dimensional deterministic optimization problems. The proposed approach is demonstrated in simulation on a nonlinear system.

  • 47.
    Hering, Alessa
    et al.
    Dept. of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, GA, NL.
    Hansen, Lasse
    Institute of Medical Informatics, Universität zu Lübeck, Lübeck, DE, Germany.
    Mok, Tony C. W.
    Dept. of Computer Science and Engineering, The Hong Kong University of Science and Technology, HK, China.
    Chung, Albert C. S.
    Dept. of Computer Science and Engineering, The Hong Kong University of Science and Technology, HK, China.
    Siebert, Hanna
    Institute of Medical Informatics, Universität zu Lübeck, Lübeck, DE, Germany.
    Hager, Stephanie
    Fraunhofer MEVIS, Institute for Digital Medicine, Lübeck, DE, Germany.
    Lange, Annkristin
    Fraunhofer MEVIS, Institute for Digital Medicine, Lübeck, DE, Germany.
    Kuckertz, Sven
    Fraunhofer MEVIS, Institute for Digital Medicine, Lübeck, DE, Germany.
    Heldmann, Stefan
    Fraunhofer MEVIS, Institute for Digital Medicine, Lübeck, DE, Germany.
    Shao, Wei
    Dept. of Radiology, Stanford University, Stanford, US.
    Vesal, Sulaiman
    Dept. of Urology, Stanford University, Stanford, US.
    Rusu, Mirabela
    Dept. of Radiology, Stanford University, Stanford, US.
    Sonn, Geoffrey
    Dept. of Urology, Stanford University, Stanford, US.
    Estienne, Theo
    Mathématiques et Informatique pour la Complexité et les Systèmes, Inria Saclay, Université Paris-Saclay, CentraleSupélec, Gif-sur-Yvette, FR.
    Vakalopoulou, Maria
    Mathématiques et Informatique pour la Complexité et les Systèmes, Inria Saclay, Université Paris-Saclay, CentraleSupélec, Gif-sur-Yvette, FR.
    Han, Luyi
    Dept. of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, NL.
    Huang, Yunzhi
    School of Automation, Nanjing University of Information Science and Technology, Nanjing, CN.
    Yap, Pew-Thian
    Dept. of Radiology and Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, US.
    Brudfors, Mikael
    King’s College, London, UK.
    Balbastre, Yael
    Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, US.
    Joutard, Samuel
    King’s College, London, UK.
    Modat, Marc
    King’s College, London, UK.
    Lifshitz, Gal
    Tel Aviv University, IL, USA.
    Raviv, Dan
    Tel Aviv University, IL, USA.
    Lv, Jinxin
    Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, CN.
    Li, Qiang
    Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, CN.
    Jaouen, Vincent
    UMR 1101 LaTIM, IMT Atlantique, Inserm, Brest, FR.
    Visvikis, Dimitris
    UMR 1101 LaTIM, IMT Atlantique, Inserm, Brest, FR.
    Fourcade, Constance
    Ecole Centrale de Nantes, LS2N, UMR CNRS, Nantes, FR.
    Rubeaux, Mathieu
    Keosys Medical Imaging, Saint Herblain, FR.
    Pan, Wentao
    Shenzhen International Graduate School, Tsinghua University, CN.
    Xu, Zhe
    Dept. of Biomedical Engineering, The Chinese University of Hong Kong, HK, China.
    Jian, Bailiang
    Chair for Computer Aided Medical Procedures and Augmented Reality, TUM, Garching, DE, Germany.
    De Benetti, Francesca
    Chair for Computer Aided Medical Procedures and Augmented Reality, TUM, Garching, DE, Germany.
    Wodzinski, Marek
    Dept. of Measurement and Electronics, AGH University of Science and Technology, Krakow, PL.
    Gunnarsson, Niklas
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control.
    Sjölund, Jens
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Grzech, Daniel
    Dept. of Computing, Imperial College London, UK.
    Qiu, Huaqi
    Dept. of Computing, Imperial College London, UK.
    Li, Zeju
    Dept. of Computing, Imperial College London, UK.
    Thorley, Alexander
    University of Birmingham, UK.
    Duan, Jinming
    University of Birmingham, UK.
    Grossbrohmer, Christoph
    Institute of Medical Informatics, Universität zu Lübeck, Lübeck, DE, Germany.
    Hoopes, Andrew
    Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, US.
    Reinertsen, Ingerid
    Dept. Health Research, SINTEF Digital, Trondheim, NO.
    Xiao, Yiming
    Western University, London, CA.
    Landman, Bennett
    Dept. of Electrical and Computer Engineering, Vanderbilt University, Nashville, US.
    Huo, Yuankai
    Dept. of Electrical and Computer Engineering, Vanderbilt University, Nashville, US.
    Murphy, Keelin
    Dept. of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, NL.
    Lessmann, Nikolas
    Dept. of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, NL.
    Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning2023In: IEEE Transactions on Medical Imaging, ISSN 0278-0062, E-ISSN 1558-254X, Vol. 42, no 3, p. 697-712Article in journal (Refereed)
    Abstract [en]

    Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https:// learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

    Download full text (pdf)
    fulltext
  • 48.
    Horta Ribeiro, Antônio
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Univ Fed Minas Gerais, Dept Comp Sci, Belo Horizonte, MG, Brazil..
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    How convolutional neural networks deal with aliasing2021In: 2021 IEEE International Conference On Acoustics, Speech And Signal Processing (ICASSP 2021), Institute of Electrical and Electronics Engineers (IEEE) Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 2755-2759Conference paper (Refereed)
    Abstract [en]

    The convolutional neural network (CNN) remains an essential tool in solving computer vision problems. Standard convolutional architectures consist of stacked layers of operations that progressively downscale the image. Aliasing is a well-known side-effect of downsampling that may take place: it causes high-frequency components of the original signal to become indistinguishable from its low-frequency components. While downsampling takes place in the max-pooling layers or in the strided-convolutions in these models, there is no explicit mechanism that prevents aliasing from taking place in these layers. Due to the impressive performance of these models, it is natural to suspect that they, somehow, implicitly deal with this distortion. The question we aim to answer in this paper is simply: "how and to what extent do CNNs counteract aliasing?" We explore the question by means of two examples: In the first, we assess the CNNs capability of distinguishing oscillations at the input, showing that the redundancies in the intermediate channels play an important role in succeeding at the task; In the second, we show that an image classifier CNN while, in principle, capable of implementing anti-aliasing filters, does not prevent aliasing from taking place in the intermediate layers.

  • 49.
    Horta Ribeiro, Antônio
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence. Univ Fed Minas Gerais, Grad Program Elect Engn, Belo Horizonte, MG, Brazil.
    Tiels, Koen
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Eindhoven Univ Technol, Dept Mech Engn, Eindhoven, Netherlands.
    Umenberger, Jack
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Schön, Thomas B.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Aguirre, Luis A.
    Univ Fed Minas Gerais, Dept Elect Engn, Belo Horizonte, MG, Brazil.
    On the smoothness of nonlinear system identification2020In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 121, article id 109158Article in journal (Refereed)
    Abstract [en]

    We shed new light on the smoothness of optimization problems arising in prediction error parameter estimation of linear and nonlinear systems. We show that for regions of the parameter space where the model is not contractive, the Lipschitz constant and β-smoothness of the objective function might blow up exponentially with the simulation length, making it hard to numerically find minima within those regions or, even, to escape from them. In addition to providing theoretical understanding of this problem, this paper also proposes the use of multiple shooting as a viable solution. The proposed method minimizes the error between a prediction model and the observed values. Rather than running the prediction model over the entire dataset, multiple shooting splits the data into smaller subsets and runs the prediction model over each subset, making the simulation length a design parameter and making it possible to solve problems that would be infeasible using a standard approach. The equivalence to the original problem is obtained by including constraints in the optimization. The new method is illustrated by estimating the parameters of nonlinear systems with chaotic or unstable behavior, as well as neural networks. We also present a comparative analysis of the proposed method with multi-step-ahead prediction error minimization.

  • 50.
    Hult, Ludvig
    et al.
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Zachariah, Dave
    Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Division of Systems and Control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Automatic control. Uppsala University, Disciplinary Domain of Science and Technology, Mathematics and Computer Science, Department of Information Technology, Artificial Intelligence.
    Inference of Causal Effects when Control Variables are Unknown2020In: Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, PMLR 161:1300-1309, 2021., 2020Conference paper (Refereed)
123 1 - 50 of 107
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf