Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
Link to record
Permanent link

Direct link
Publications (10 of 23) Show all publications
Zhang, R., Mattsson, P. & Wigren, T. (2023). Aiding reinforcement learning for set point control. In: IFAC (Ed.), Preprints IFAC World Congress: . Paper presented at 22nd IFAC World Congress (pp. 2748-2754).
Open this publication in new window or tab >>Aiding reinforcement learning for set point control
2023 (English)In: Preprints IFAC World Congress / [ed] IFAC, 2023, p. 2748-2754Conference paper, Published paper (Refereed)
Abstract [en]

While reinforcement learning has made great improvements, state-of-the-art  algorithms can till struggle with seemingly simple set-point feedback control problems. One reason for this is that the learned controller may not be able to excite the system dynamics well enough initially, and therefore it can take a long time to get data that is informative enough to learn for good control. The paper contributes by augmentation of reinforcement learning with a simple guiding feedback controller, for example, a proportional controller. The key advantage in set-point control is a much improved excitation that improves the convergence properties of the reinforcement learning controller significantly. This can be very important in real-world control where quick and accurate convergence is needed. The proposed method is evaluated with simulation and on a real-world double tank process with promising results.

National Category
Control Engineering
Research subject
Automatic Control
Identifiers
urn:nbn:se:uu:diva-516009 (URN)
Conference
22nd IFAC World Congress
Funder
Swedish Research Council, 621-2016-06079
Available from: 2023-11-16 Created: 2023-11-16 Last updated: 2023-12-24Bibliographically approved
Mattsson, P., Zachariah, D. & Stoica, P. (2023). Analysis of the Minimum-Norm Least-Squares Estimator and Its Double-Descent Behavior [Lecture Notes]. IEEE signal processing magazine (Print), 40(3), 39-75
Open this publication in new window or tab >>Analysis of the Minimum-Norm Least-Squares Estimator and Its Double-Descent Behavior [Lecture Notes]
2023 (English)In: IEEE signal processing magazine (Print), ISSN 1053-5888, E-ISSN 1558-0792, Vol. 40, no 3, p. 39-75Article in journal (Refereed) Published
Abstract [en]

Linear regression models have a wide range of applications in statistics, signal processing, and machine learning. In this Lecture Notes column we will examine the performance of the least-squares (LS) estimator with a focus on the case when there are more parameters than training samples, which is often overlooked in textbooks on estimation.

Place, publisher, year, edition, pages
IEEE, 2023
Keywords
Least squares methods, Linear regression, Estimation, Machine learning, Signal processing, Behavioral sciences
National Category
Probability Theory and Statistics
Identifiers
urn:nbn:se:uu:diva-504033 (URN)10.1109/MSP.2023.3242083 (DOI)000981974000005 ()
Available from: 2023-06-28 Created: 2023-06-28 Last updated: 2023-06-28Bibliographically approved
Wullt, B., Mattsson, P., Schön, T. B. & Norrlöf, M. (2023). Neural motion planning in dynamic environments. In: IFAC-PapersOnLine: . Paper presented at IFAC World Congress (pp. 10126-10131). Elsevier
Open this publication in new window or tab >>Neural motion planning in dynamic environments
2023 (English)In: IFAC-PapersOnLine, Elsevier, 2023, p. 10126-10131Conference paper, Published paper (Refereed)
Abstract [en]

Motion planning is a mature field within robotics with many successful solutions. Despite this, current state-of-the-art planners are still computationally heavy. To address this, recent work have employed ideas from machine learning, which have drastically reduced the computational cost once a planner has been trained. It is mainly static environments that have been studied in this way. We continue along the same research direction but expand the problem to include dynamic environments, hence increasing the difficulty of the problem. Analogously to previous work, we use imitation learning, where a planning policy is learnt from an expert planner in a supervised manner. Our main contribution is a planner mimicking an expert that considers the future movement of all the obstacles in the environment, which is key in order to learn a successful policy in dynamic environments. We illustrate this by evaluating our approach in a dynamic environment and by comparing our planner with a conventional planner that re-plans at every iteration, which is a common approach in dynamic motion planning. We observe that our approach yields a higher success rate, while also taking less time and accumulating less distance to reach the goal.

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Data-driven control, Learning for control, Robots manipulators, Motion planning, Imitation learning
National Category
Control Engineering
Identifiers
urn:nbn:se:uu:diva-518375 (URN)10.1016/j.ifacol.2023.10.885 (DOI)
Conference
IFAC World Congress
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Available from: 2023-12-18 Created: 2023-12-18 Last updated: 2023-12-24Bibliographically approved
Zhang, R., Mattsson, P. & Wigren, T. (2023). Observer-Feedback-Feedforward Controller Structures in Reinforcement Learning. In: IFAC (Ed.), Prep. IFAC World Congress: . Paper presented at 22nd IFAC World Congress (pp. 6807-6912). IFAC Papers Online
Open this publication in new window or tab >>Observer-Feedback-Feedforward Controller Structures in Reinforcement Learning
2023 (English)In: Prep. IFAC World Congress / [ed] IFAC, IFAC Papers Online, 2023, p. 6807-6912Conference paper, Published paper (Refereed)
Abstract [en]

The paper proposes the use of structured neural networks for reinforcement learning based nonlinear adaptive control. The focus is on partially observable systems, with separate neural networks for the state and feedforward observer and the state feedback and feedforward controller. The observer dynamics are modelled by recurrent neural networks while a standard network is used for the controller. As discussed in the paper, this leads to a separation of the observer dynamics to the recurrent neural network part, and the state feedback to the feedback and feedforward network. The structured approach reduces the computational complexity and gives the reinforcement learning based controller an understandable structure as compared to when one single neural network is used. As shown by simulation the proposed structure has the additional and main advantage that the training becomes significantly faster. Two ways to include feedforward structure are presented, one related to state feedback control and one related to classical feedforward control. The latter method introduces further structure with a separate recurrent neural network that processes only the measured disturbance. When evaluated with simulation on a cascaded double tank process, the method with most structure performs the best, with excellent feedforward disturbance rejection gains.

Place, publisher, year, edition, pages
IFAC Papers Online, 2023
National Category
Control Engineering
Research subject
Automatic Control
Identifiers
urn:nbn:se:uu:diva-516011 (URN)
Conference
22nd IFAC World Congress
Funder
Swedish Research Council, 621-2016-06079
Available from: 2023-11-16 Created: 2023-11-16 Last updated: 2023-12-24Bibliographically approved
Mattsson, P. & Schön, T. B. (2023). On the regularization in DeePC. In: IFAC-PapersOnLine: . Paper presented at IFAC World Congress (pp. 625-631). Elsevier, 56
Open this publication in new window or tab >>On the regularization in DeePC
2023 (English)In: IFAC-PapersOnLine, Elsevier, 2023, Vol. 56, p. 625-631Conference paper, Published paper (Refereed)
Abstract [en]

Data-enabled predictive control (DeePC) is an approach to data-driven direct control that has gained considerable interest recently. In this work we show that the main equations of DeePC can be derived from a linear regression model based on the input/output equations for a linear system that can be estimated using linear least squares. Our main contribution is an analysis showing that DeePC—using different types of regularization—gives predictions that equals those of an estimated model if the regularization weight is large enough. The numerical example indicate that the optimal weights are sufficiently large for all considered types of regularization. This suggests that the use of an indirect method based on the linear regression model implicitly estimated by DeePC can be beneficial, since this avoids tuning of weights, does not distort the control criterion and makes the online optimization considerably faster. Furthermore, a slightly modified estimate is considered, that reduces the number of unknown parameters in the linear regression model by taking causality into account.Keywords: Data-driven control; Predictive control; Regularization; Finite-horizon control

Place, publisher, year, edition, pages
Elsevier, 2023
National Category
Control Engineering
Identifiers
urn:nbn:se:uu:diva-518369 (URN)10.1016/j.ifacol.2023.10.1637 (DOI)
Conference
IFAC World Congress
Funder
Swedish Research Council, 2021-04301Swedish Research Council, 621-2016-06079
Available from: 2023-12-18 Created: 2023-12-18 Last updated: 2023-12-24Bibliographically approved
Mattsson, P., Zachariah, D. & Stoica, P. (2023). Regularized Linear Regression via Covariance Fitting. IEEE Transactions on Signal Processing, 71, 1175-1183
Open this publication in new window or tab >>Regularized Linear Regression via Covariance Fitting
2023 (English)In: IEEE Transactions on Signal Processing, ISSN 1053-587X, E-ISSN 1941-0476, Vol. 71, p. 1175-1183Article in journal (Refereed) Published
Abstract [en]

The linear minimum mean-square error estimator (LMMSE) can be viewed as a solution to a certain regularized least-squares problem formulated using model covariance matrices. However, the appropriate parameters of the model covariance matrices are unknown in many applications. This raises the question: how should we choose them using only the data? Using data-adaptive matrices obtained via the covariance fitting SPICE-methodology, we show that the empirical LMMSE is equivalent to tuned versions of various known regularized estimators - such as ridge regression, LASSO, and regularized least absolute deviation - depending on the chosen covariance structures. These theoretical results unify several important estimators under a common umbrella. Furthermore, through a number of numerical examples we show that the regularization parameters obtained via covariance fitting are close to optimal for a range of different signal conditions.

Place, publisher, year, edition, pages
IEEE, 2023
Keywords
Estimation theory, parameter estimation
National Category
Probability Theory and Statistics Signal Processing
Identifiers
urn:nbn:se:uu:diva-502506 (URN)10.1109/TSP.2023.3263363 (DOI)000976047000002 ()
Funder
Swedish Research Council, 621-2016-06079Swedish Research Council, 2018-05040Swedish Research Council, 2021-05022
Available from: 2023-05-29 Created: 2023-05-29 Last updated: 2023-05-29Bibliographically approved
Zhang, R., Mattsson, P. & Wigren, T. (2023). Robust nonlinear set-point control with reinforcement learning. In: 2023 American Control Conference (ACC): . Paper presented at American Control Conference (ACC), MAY 31-JUN 2, 2023, San Diego, CA, USA (pp. 84-91). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Robust nonlinear set-point control with reinforcement learning
2023 (English)In: 2023 American Control Conference (ACC), Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 84-91Conference paper, Published paper (Refereed)
Abstract [en]

There has recently been an increased interest in reinforcement learning for nonlinear control problems. However standard reinforcement learning algorithms can often struggle even on seemingly simple set-point control problems. This paper argues that three ideas can improve reinforcement learning methods even for highly nonlinear set-point control problems: 1) Make use of a prior feedback controller to aid amplitude exploration. 2) Use integrated errors. 3) Train on model ensembles. Together these ideas lead to more efficient training, and a trained set-point controller that is more robust to modelling errors and thus can be directly deployed to real-world nonlinear systems. The claim is supported by experiments with a real-world nonlinear cascaded tank process and a simulated strongly nonlinear pH-control system.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Series
Proceedings of the American Control Conference, ISSN 0743-1619, E-ISSN 2378-5861
National Category
Control Engineering
Identifiers
urn:nbn:se:uu:diva-512438 (URN)10.23919/ACC55779.2023.10156038 (DOI)001027160300012 ()979-8-3503-2806-6 (ISBN)979-8-3503-2807-3 (ISBN)978-1-6654-6952-4 (ISBN)
Conference
American Control Conference (ACC), MAY 31-JUN 2, 2023, San Diego, CA, USA
Funder
Swedish Research Council, 621-2016-06079
Available from: 2023-09-26 Created: 2023-09-26 Last updated: 2023-09-26Bibliographically approved
Ferizbegovic, M., Mattsson, P., Schön, T. B. & Hjalmarsson, H. (2021). Bayes Control of Hammerstein Systems. In: IFAC PapersOnLine: . Paper presented at 19th IFAC Symposium on System Identification (SYSID), JUL 13-16, 2021, Padova, ITALY (pp. 755-760). Elsevier BV Elsevier, 54(7)
Open this publication in new window or tab >>Bayes Control of Hammerstein Systems
2021 (English)In: IFAC PapersOnLine, Elsevier BV Elsevier, 2021, Vol. 54, no 7, p. 755-760Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we consider data driven control of Hammerstein systems. For such systems a common control structure is a transfer function followed by a static output nonlinearity that tries to cancel the input nonlinearity of the system, which is modeled as a polynomial or piece-wise linear function. The linear part of the controller is used to achieve desired disturbance rejection and tracking properties. To design a linear part of the controller, we propose a weighted average risk criterion with the risk being the average of the squared L2 tracking error. Here the average is with respect to the observations used in the controller and the weighting is with respect to how important it is to have good control for different impulse responses. This criterion corresponds to the average risk criterion leading to the Bayes estimator and we therefore call this approach Bayes control. By parametrizing the weighting function and estimating the corresponding hyperparameters we tune the weighting function to the information regarding the true impulse response contained in the data set available to the user for the control design. The numerical results show that the proposed methods result in stable controllers with performance comparable to the optimal controller, designed using the true input nonlinearity and true plant.

Place, publisher, year, edition, pages
ElsevierElsevier BV, 2021
Keywords
Bayesian methods, Model reference control, Hammerstein system
National Category
Control Engineering
Identifiers
urn:nbn:se:uu:diva-457747 (URN)10.1016/j.ifacol.2021.08.452 (DOI)000696396200125 ()
Conference
19th IFAC Symposium on System Identification (SYSID), JUL 13-16, 2021, Padova, ITALY
Funder
Swedish Research Council, 2016-06079Knut and Alice Wallenberg FoundationWallenberg AI, Autonomous Systems and Software Program (WASP)
Available from: 2021-11-03 Created: 2021-11-03 Last updated: 2024-01-15Bibliographically approved
Ferizbegovic, M., Hjalmarsson, H., Mattsson, P. & Schön, T. B. (2021). Willems' fundamental lemma based on second-order moments. In: 2021 60th IEEE Conference On Decision And Control (CDC): . Paper presented at 60th IEEE Conference on Decision and Control (CDC), DEC 13-17, 2021, ELECTR NETWORK (pp. 396-401). Institute of Electrical and Electronics Engineers (IEEE) Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Willems' fundamental lemma based on second-order moments
2021 (English)In: 2021 60th IEEE Conference On Decision And Control (CDC), Institute of Electrical and Electronics Engineers (IEEE) Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 396-401Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we propose variations of Willems' fundamental lemma that utilize second-order moments such as correlation functions in the time domain and power spectra in the frequency domain. We believe that using a formulation with estimated correlation coefficients is suitable for data compression, and possibly can reduce noise. Also, the formulations in the frequency domain can enable modeling of a system in a frequency region of interest.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE)Institute of Electrical and Electronics Engineers (IEEE), 2021
Series
IEEE Conference on Decision and Control, ISSN 0743-1546
National Category
Control Engineering
Identifiers
urn:nbn:se:uu:diva-475326 (URN)10.1109/CDC45484.2021.9683632 (DOI)000781990300058 ()978-1-6654-3659-5 (ISBN)
Conference
60th IEEE Conference on Decision and Control (CDC), DEC 13-17, 2021, ELECTR NETWORK
Funder
Knut and Alice Wallenberg Foundation, 2016-06079
Available from: 2022-06-01 Created: 2022-06-01 Last updated: 2024-01-15Bibliographically approved
Masud, N., Mattsson, P., Smith, C. & Isaksson, M. (2020). On stability and performance of disturbance observer-based-dynamic load torque compensator for assistive exoskeleton: A hybrid approach. Mechatronics (Oxford), 69, Article ID 102373.
Open this publication in new window or tab >>On stability and performance of disturbance observer-based-dynamic load torque compensator for assistive exoskeleton: A hybrid approach
2020 (English)In: Mechatronics (Oxford), ISSN 0957-4158, E-ISSN 1873-4006, Vol. 69, article id 102373Article in journal (Refereed) Published
Abstract [en]

A disturbance observer-based-dynamic load-torque compensator for current-controlled DC-drives, as joint actuator of assistive exoskeletons, has been recently proposed. It has been shown that this compensator can effectively linearize and decouple the coupled nonlinear dynamics of the human-exoskeleton system, by more effectively compensating the associated nonlinear load-torques of the exoskeleton at the joint level. In this paper, a detailed analysis of the current controlled DC drive-servo system using the said compensator, with respect to performance and stability is presented, highlighting the key factors and considerations affecting both the stability and performance of the compensated servo system. It is shown both theoretically and through simulation results that the stability of the compensated servo system is compromised as performance is increased and vice-versa. Based on the saturation state of the servo system, a new hybrid switching control strategy is then proposed to select stability or performance-based compensator and controller optimally. The strategy is then experimentally verified both at the joint and task space level by using the developed four active-degree of freedom exoskeleton test rig.

Keywords
Assistive-exoskeleton, Performance, Stability, Disturbance observer, Load-torque
National Category
Control Engineering
Identifiers
urn:nbn:se:uu:diva-427469 (URN)10.1016/j.mechatronics.2020.102373 (DOI)000571817900010 ()
Available from: 2020-12-09 Created: 2020-12-09 Last updated: 2020-12-09Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-2678-1330

Search in DiVA

Show all publications