Logo: to the web site of Uppsala University

Robotic pick and place solutions in workspaces shared with humans provide a challenge in obtaining high throughput while at the same time guaranteeing safety. A common simplification in motion planning is to decouple the spatial and temporal dimension, which works well in static settings. However, in dynamic environments this is no longer true since these two dimensions now exhibit strong dependencies. Here, we develop a model predictive control (MPC) solution capable of performing the required joint spatial and temporal planning. We use a recent construction, a configuration space corridor capable of representing a collision-free region in the configuration space. We show how the predicted future movement of the obstacles can be incorporated in the planning, by rapidly reshaping this corridor along the prediction horizon. We also propose a novel corridor planner that guides our MPC to the goal in a safe and efficient manner. From the numerical experiments, we observe higher success rates compared to decoupled motion planning, while also being faster and safer. Finally, we present statistics of our computation time, demonstrating real-time capability at a control rate of 20 Hz.

The paper derives analytical expressions for the asymptotic average updating direction of the adaptive moment generation (ADAM) algorithm when applied to recursive identification of nonlinear systems. It is proved that the standard hyper-parameter setting results in the same asymptotic average updating direction as a diagonally power normalized stochastic gradient algorithm. With the internal filtering turned off, the asymptotic average updating direction is instead equivalent to that of a sign-sign stochastic gradient algorithm. Global convergence to an invariant set follows, where a subset of parameters contain those that give a correct input-output description of the system. The paper also exploits a nonlinear dynamic model to embed structure in recurrent neural networks. A Monte-Carlo simulation study validates the results.

Diffusion models have shown impressive performance in capturing complex and multi-modal action distributions for game agents, but their slow inference speed prevents practical deployment in real-time game environments. While consistency models offer a promising approach for one-step generation, they often suffer from training instability and performance degradation when applied to policy learning. In this paper, we present CPQE (Consistency Policy with Q-Ensembles), which combines consistency models with Q-ensembles to address these challenges.CPQE leverages uncertainty estimation through Q-ensembles to provide more reliable value function approximations, resulting in better training stability and improved performance compared to classic double Q-network methods. Our extensive experiments across multiple game scenarios demonstrate that CPQE achieves inference speeds of up to 60 Hz -- a significant improvement over state-of-the-art diffusion policies that operate at only 20 Hz -- while maintaining comparable performance to multi-step diffusion approaches.CPQE consistently outperforms state-of-the-art consistency model approaches, showing both higher rewards and enhanced training stability throughout the learning process. These results indicate that CPQE offers a practical solution for deploying diffusion-based policies in games and other real-time applications where both multi-modal behavior modeling and rapid inference are critical requirements.

The current state-of-the art motion planners for mobile robots typically do not consider the future movement of moving obstacles. Instead they work by rapid replanning, which makes them reactively adapt to any changes in the environment. This can result in a sub-optimal behavior, which we address in this work by proposing a predictive motion planner that integrates motion predictions into all planning steps. We demonstrate the validity of our approach by evaluating our proposed planner in a dynamic environment where the robot moves slower than the moving obstacles. We benchmark our predictive planner with a reactive planning approach and observe better performance, both in avoiding collisions and maintaining the robots position in the goal region.

Digital twins have been gaining an immense interest in various fields over the last decade. Bringing conventional process simulation models into (near) real time are thought to provide valuable insights for operators, decision makers, and stakeholders in many industries. The objective of this paper is to describe two methods for implementing digital twins at water resource recovery facilities and highlight and discuss their differences and preferable use situations, with focus on the automated data transfer from the real process. Case 1 uses a tailor-made infrastructure for automated data transfer between the facility and the digital twin. Case 2 uses edge computing for rapid automated data transfer. The data transfer lag from process to digital twin is low compared to the simulation frequency in both systems. The presented digital twin objectives can be achieved using either of the presented methods. The method of Case 1 is better suited for automatic recalibration of model parameters, although workarounds exist for the method in Case 2. The method of Case 2 is well suited for objectives such as soft sensors due to its integration with the SCADA system and low latency. The objective of the digital twin, and the required latency of the system, should guide the choice of method.Practitioner Points Various methods can be used for automated data transfer between the physical system and a digital twin. Delays in the data transfer differ depending on implementation method. The digital twin objective determines the required simulation frequency. Implementation method should be chosen based on the required simulation frequency. This paper showcases two case studies for automated data transfer in digital twins. One uses a tailor-made infrastructure for data transfer between the facility and digital twin, the other uses edge computing. The digital twin objective should determine the required simulation frequency and, thus, sets boundaries on suitable implementation methods. image

Diffusion policy has shown a strong ability to express complex action distributions in offline reinforcement learning (RL). However, it suffers from overestimating Q-value functions on out-of-distribution (OOD) data points due to the offline dataset limitation. To address it, this paper proposes a novel entropy-regularized diffusion policy and takes into account the confidence of the Q-value prediction with Q-ensembles. At the core of our diffusion policy is a mean-reverting stochastic differential equation (SDE) that transfers the action distribution into a standard Gaussian form and then samples actions conditioned on the environment state with a corresponding reverse-time process. We show that the entropy of such a policy is tractable and that can be used to increase the exploration of OOD samples in offline RL training. Moreover, we propose using the lower confidence bound of Q-ensembles for pessimistic Q-value function estimation. The proposed approach demonstrates state-of-the-art performance across a range of tasks in the D4RL benchmarks, significantly improving upon existing diffusion-based policies. The code is available at https://github.com/ruoqizzz/entropy-offlineRL

In this paper, we discuss the problem of learning state observers for Recurrent Neural Network (RNN) black-box models of dynamical systems. State observers are indeed key to designing state-feedback control laws, such as nonlinear Model Predictive Control, with satisfactory closed-loop performance. Besides, they can also improve the training procedure of RNN models themselves. Then, we summarize recent developments aimed at jointly learning RNN models and neural networkbased state observers, and we propose a new structure based on the recent S5 architecture. We finally test various observer structures on a pH neutralization process benchmark system, showing the advantages and shortcomings of each architecture.

Recently, several direct Data-Driven Predictive Control (DDPC) methods have been proposed, advocating the possibility of designing predictive controllers from historical input-output trajectories without the need to identify a model. In this letter, we show their equivalence to a (relaxed) indirect approach, allowing us to reformulate direct methods in terms of estimated parameters and covariance matrices. This allows us to provide further insights into how these direct predictive control methods work, showing that, for unconstrained problems, the direct methods are equivalent to subspace predictive control with a reduced weight on the tracking cost, and analyzing the impact of the data length on tuning strategies. Via a numerical experiment, we also illustrate why the performance of direct DDPC methods with fixed regularization tends to degrade as the number of training samples increases.

In data-driven control design, an important prob-lem is to deal with uncertainty due to limited and noisydata. One way to do this is to use a min-max approach,which aims to minimize some design criteria for the worst-case scenario. However, a strategy based on this approachcan lead to overly conservative controllers. To overcome thisissue, we apply the idea of baseline regret, and it is seen thatminimizing the baseline regret under model uncertainty canguarantee safe controller improvement with less conservatismand variance in the resulting controllers. To exemplify theuse of baseline controllers, we focus on the output feedbacksetting and propose a two-step control design method; first,an uncertainty set is constructed by a data-driven systemidentification approach based on finite impulse response models;then a control design criterion based on model reference controlis used. To solve the baseline regret optimization problemefficiently, we use a convex approximation of the criterion andapply the scenario approach in optimization. The numericalexamples show that the inclusion of baseline regret indeedimproves the performance and reduces the variance of theresulting controller.