We examine the paradigm shift from that of the transformation model to that of stochastic profit models. Some of the anomalies undermining the transformation model are given graphically. In the last two sections we present volume II of Capital as an alternative starting point for thinking about the relation between value and price.
In this paper, we develop a new technique for estimating fine clock errors and range between two nodes simultaneously by two-way time-of-arrival measurements using impulse-radio ultrawideband signals. Estimators for clock parameters and the range are proposed, which are robust with respect to outliers. They are analyzed numerically and by means of experimental measurement campaigns. The technique and derived estimators achieve accuracies below 1 Hz for frequency estimation, below 1 ns for phase estimation, and 20 cm for range estimation, at a 4-m distance using 100-MHz clocks at both nodes. Therefore, we show that the proposed joint approach is practical and can simultaneously provide clock synchronization and positioning in an experimental system.
The paper considers the problem of multi-objective decision support when outcomes are uncertain. We extend the concept of Pareto-efficient decisions to take into account the uncertainty of decision outcomes across varying contexts. This enables quantifying trade-offs between decisions in terms of tail outcomes that are relevant in safety-critical applications. We propose a method for learning efficient decisions with statistical confidence, building on results from the conformal prediction literature. The method adapts to weak or nonexistent context covariate overlap and its statistical guarantees are evaluated using both synthetic and real data.
Air pollution is one of the major concerns in global urbanization. Data science can help to understand the dynamics of air pollution and build reliable statistical models to forecast air pollution levels. To achieve these goals, one needs to learn the statistical models which can capture the dynamics from the historical data and predict air pollution in the future. Furthermore, the large size and heterogeneity of today’s big urban data pose significant challenges on the scalability and flexibility of the statistical models. In this work, we present a scalable belief updating framework that is able to produce reliable predictions, using over millions of historical hourly air pollutant and meteorology records. We also present a non-parametric approach to learn the statistical model which reveals interesting periodical dynamics and correlations of the dataset. Based on the scalable belief update framework and the non-parametric model learning approach, we propose an iterative update algorithm to accelerate Gaussian process, which is notorious for its prohibitive computation with large input data. Finally, we demonstrate how to integrate information from heterogeneous data by regarding the beliefs produced by other models as the informative prior. Numerical examples and experimental results are presented to validate the proposed method.
Nonparametric regression using Gaussian Process (GP) models is a powerful but computationally demanding method. While various approximation methods have been developed to mitigate its computation complexity, few works have addressed the quality of the resulting approximations of the target posterior. In this paper we start from a general belief updating framework that can generate various approximations. We show that applying using composite likelihoods yields computationally scalable approximations for both GP learning and prediction. We then analyze the quality of the approximation in terms of averaged prediction errors as well as Kullback-Leibler (KL) divergences.
Predictors are learned using past training data which may contain features that are unavailable at the time of prediction. We develop an approach that is robust against outlying missing features, based on the optimality properties of an oracle predictor which observes them. The robustness properties of the approach are demonstrated on both real and synthetic data.
Smart maintenance strategies are becoming increasingly important in the industry, and can contribute to environmentally and economically sustainable production. In this paper a recently developed latent variable framework for nonlinear-system identification is considered for use in smart maintenance. A model is first identified using data from a system operating under normal conditions. Then the identified model is used to detect when the system begins to deviate from normal behavior. Furthermore, for systems that operate on separate batches (units), we develop a new method that identifies individual models for each batch. This can be used both to detect anomalous batches and changes in the system behavior. Finally, the two methods are evaluated on two different industrial case studies. In the first, the purpose is to detect fouling in a heat exchanger. In the second, the goal is to detect when the tool in a wood moulder machine should be changed.
Linear regression models have a wide range of applications in statistics, signal processing, and machine learning. In this Lecture Notes column we will examine the performance of the least-squares (LS) estimator with a focus on the case when there are more parameters than training samples, which is often overlooked in textbooks on estimation.
The linear minimum mean-square error estimator (LMMSE) can be viewed as a solution to a certain regularized least-squares problem formulated using model covariance matrices. However, the appropriate parameters of the model covariance matrices are unknown in many applications. This raises the question: how should we choose them using only the data? Using data-adaptive matrices obtained via the covariance fitting SPICE-methodology, we show that the empirical LMMSE is equivalent to tuned versions of various known regularized estimators - such as ridge regression, LASSO, and regularized least absolute deviation - depending on the chosen covariance structures. These theoretical results unify several important estimators under a common umbrella. Furthermore, through a number of numerical examples we show that the regularization parameters obtained via covariance fitting are close to optimal for a range of different signal conditions.
In this paper we study the problem of identifying linear and nonlinear feedback mechanisms, or controllers, operating in closed loop. A recently developed identification method for nonlinear systems, the LAVA method, is used for this purpose. Identification data is obtained from inertial sensors, that provide information about the movement of the system, in the form of linear acceleration and angular velocity measurements. This information is different from the information that is available to the controller to be identified, which makes use of unknown internal sensors instead. We provide two examples, a simulated neuromuscular controller in standing human balance, and a lead-filter controlling a physical position servo using a DC motor. Both linear and nonlinear controllers are used in the examples. We show that the LAVA method is able to identify sparse, parsimonious models of the controllers.
We address the problem of timing-based localization in wireless networks, when an unknown fraction of data is corrupted by non-ideal propagation conditions. While timing-based techniques can enable accurate localization, they are sensitive to corrupted data. We develop a robust method that is applicable to a range of localization techniques, including time-of-arrival, time-difference-of-arrival and time-difference in schedule-based transmissions. The method is distribution-free, is computationally efficient and requires only an upper bound on the fraction of corrupted data, thus obviating distributional assumptions on the corrupting noise. The robustness of the method is demonstrated in numerical experiments.
We address the problem of predicting spatio-temporal processes with temporal patterns that vary across spatial regions, when data is obtained as a stream. That is, when the training dataset is augmented sequentially. Specifically, we develop a localized spatio-temporal covariance model of the process that can capture spatially varying temporal periodicities in the data. We then apply a covariance-fitting methodology to learn the model parameters which yields a predictor that can be updated sequentially with each new data point. The proposed method is evaluated using both synthetic and real climate data which demonstrate its ability to accurately predict data missing in spatial regions over time.
We address the problem of inferring the causal effect of an exposure on an outcome across space, using observational data. The data is possibly subject to unmeasured confounding variables which, in a standard approach, must be adjusted for by estimating a nuisance function. Here we develop a method that eliminates the nuisance function, while mitigating the resulting errors-in-variables. The result is a robust and accurate inference method for spatially varying heterogeneous causal effects. The properties of the method are demonstrated on synthetic as well as real data from Germany and the US.
A spatial point process can be characterized by an intensity function which predicts the number of events that occur across space. In this paper, we develop a method to infer predictive intensity intervals by learning a spatial model using a regularized criterion. We prove that the proposed method exhibits out-of-sample prediction performance guarantees which, unlike standard estimators, are valid even when the spatial model is misspecified. The method is demonstrated using synthetic as well as real spatial data.
We consider the online learning of linear smoother predictors based on a covariance model of the outcomes. To control its degrees of freedom in an appropriate manner, the covariance model parameters are often learned using cross-validation or maximum-likelihood techniques. However, neither technique is suitable when training data arrives in a streaming fashion. Here we consider a covariance-fitting method to learn the model parameters, initially used in spectral estimation. We show that this results in a computation efficient online learning method in which the resulting predictor can be updated sequentially. We prove that, with high probability, its out-of-sample error approaches the minimum achievable level at root-$n$ rate. Moreover, we show that the resulting predictor enjoys two different robustness properties. First, it minimizes the out-of-sample error with respect to the least favourable distribution within a given Wasserstein distance from the empirical distribution. Second, it is robust against errors in the covariate training data. We illustrate the performance of the proposed method in a numerical experiment.
We consider the problem of learning decision parameters from data obtained in different contexts. When future context information is inaccessible, we consider the resulting (i) worst-case and (ii) overall out-of-sample performance of the learned parameters. We propose a robust approach that trades off these two performance criteria based on the partial information obtained about the unknown context distribution. The proposed method overcomes the overly conservative nature of the minimax method, while robustifying the empirical risk minimization method in a statistically motivated manner. We illustrate the performance of the method in a classification task.
We consider a general statistical learning problem where an unknown fraction of the training data is corrupted. We develop a robust learning method that only requires specifying an upper bound on the corrupted data fraction. The method minimizes a risk function defined by a non-parametric distribution with unknown probability weights. We derive and analyse the optimal weights and show how they provide robustness against corrupted data. Furthermore, we give a computationally efficient coordinate descent algorithm to solve the risk minimization problem. We demonstrate the wide range applicability of the method, including regression, classification, unsupervised learning and classic parameter estimation, with state-of-the-art performance.
In this paper a new method for heat load prediction in district energy systems is proposed. The method uses a nominal model for the prediction of the outdoor temperature dependent space heating load, and a data driven latent variable model to predict the time dependent residual heat load. The residual heat load arises mainly from time dependent operation of space heating and ventilation, and domestic hot water production. The resulting model is recursively updated on the basis of a hyper-parameter free implementation that results in a parsimonious model allowing for high computational performance. The approach is applied to a single multi-dwelling building in Lulea, Sweden, predicting the heat load using a relatively small number of model parameters and easily obtained measurements. The results are compared with predictions using an artificial neural network, showing that the proposed method achieves better prediction accuracy for the validation case. Additionally, the proposed methods exhibits explainable behavior through the use of an interpretable physical model.
The choice of model class is fundamental in statistical learning and system identification, no matter whether the class is derived from physical principles or is a generic black-box. We develop a method to evaluate the specified model class by assessing its capability of reproducing data that is similar to the observed data record. This model check is based on the information-theoretic properties of models viewed as data generators and is applicable to e.g. sequential data and nonlinear dynamical models. The method can be understood as a specific two-sided posterior predictive test. We apply the information-theoretic model check to both synthetic and real data and compare it with a classical whiteness test.
Measuring blade displacement using blade tip timing (BTT) enables nonintrusive monitoring of rotating blades and their vibration frequencies. The average sampling frequency of BTT is the product of the number of measurement probes and rotational frequency, which is usually far less than the blade natural frequency due to the limited number of probes. The pattern of the aliasing that arises from under-sampling is rather complex under uneven probe placement. In this article, we consider a probe placement design that is based on minimizing the maximum sidelobe level of the spectral window to suppress the aliasing frequencies in the spectrum. Based on a signal model containing both asynchronous and synchronous sinusoids, we then develop an extended version of the RELAX method (ERELAX) to estimate their parameters simultaneously. Model order selection rules are also used to determine the number of asynchronous sinusoids. The frequency ambiguity that arises from periodic nonuniform sampling (PNS) is also discussed based on the convolution in the frequency domain. Numerical simulations and results of a curved-blade experiment show that the proposed method has a mean squared estimation error less than 25% of that of two state-of-the-art methods (Block-OMP and MUSIC), requires 40% of the data length needed by the latter methods to achieve the same estimation accuracy, and has the smallest standard deviation of the reconstruction errors. Simulation codes are available at https://github.com/superjdg/RELAX_BTT.
Most supervised machine learning tasks are subject to irreducible prediction errors. Probabilistic predictive models address this limitation by providing probability distributions that represent a belief over plausible targets, rather than point estimates. Such models can be a valuable tool in decision-making under uncertainty, provided that the model output is meaningful and interpretable. Calibrated models guarantee that the probabilistic predictions are neither over- nor under-confident. In the machine learning literature, different measures and statistical tests have been proposed and studied for evaluating the calibration of classification models. For regression problems, however, research has been focused on a weaker condition of calibration based on predicted quantiles for real-valued targets. In this paper, we propose the first framework that unifies calibration evaluation and tests for probabilistic predictive models. It applies to any such model, including classification and regression models of arbitrary dimension. Furthermore, the framework generalizes existing measures and provides a more intuitive reformulation of a recently proposed framework for calibration in multi-class classification.
In safety-critical applications a probabilistic model is usually required to be calibrated, i.e., to capture the uncertainty of its predictions accurately. In multi-class classification, calibration of the most confident predictions only is often not sufficient. We propose and study calibration measures for multi-class classification that generalize existing measures such as the expected calibration error, the maximum calibration error, and the maximum mean calibration error. We propose and evaluate empirically different consistent and unbiased estimators for a specific class of measures based on matrix-valued kernels. Importantly, these estimators can be interpreted as test statistics associated with well-defined bounds and approximations of the p-value under the null hypothesis that the model is calibrated, significantly improving the interpretability of calibration measures, which otherwise lack any meaningful unit or scale.
A probabilistic predictive model tries to capture the uncertainty in its predictions by returning probability distributions of predictions rather than mere point estimates. In safety-critical applications it is particularly important that the uncertainty predicted by the model corresponds to empirically observed uncertainties. Such models, whose predictions are consistent with empirical observations, are called calibrated or reliable. In this article, we present CalibrationAnalysis.jl, a Julia package that can be used to analyze if a probabilistic model is calibrated. Main features of CalibrationAnalysis.jl are the recently proposed kernel calibration error and a set of hypothesis tests of calibration.
Parametric prediction error methods constitute a classical approach to the identification of linear dynamic systems with excellent large-sample properties. A more recent regularized approach, inspired by machine learning and Bayesian methods, has also gained attention. Methods based on this approach estimate the system impulse response with excellent small-sample properties. In several applications, however, it is desirable to obtain a compact representation of the system in the form of a parametric model. By viewing the identification of such models as a decision, we develop a decision-theoretic formulation of the parametric system identification problem that bridges the gap between the classical and regularized approaches above. Using the output-error model class as an illustration, we show that this decision-theoretic approach leads to a regularized method that is robust to small sample-sizes as well as overparameterization.
This paper considers the quantification of the prediction performance in Gaussian process regression. The standard approach is to base the prediction error bars on the theoretical predictive variance, which is a lower bound on the mean square-error (MSE). This approach, however, does not take into account that the statistical model is learned from the data. We show that this omission leads to a systematic underestimation of the prediction errors. Starting from a generalization of the CramÃ©r-Rao bound, we derive a more accurate MSE bound which provides a measure of uncertainty for prediction of Gaussian processes. The improved bound is easily computed and we illustrate it using synthetic and real data examples.
Clock synchronization is ubiquitous in wireless systems for communication, sensing, and control. In this paper, we design a scalable system in which an indefinite number of passively receiving wireless units can synchronize to a single master clock at the level of discrete clock ticks. Accurate synchronization requires an estimate of the node positions to compensate the time-of-flight transmission delay in line-of-sight environments. If such information is available, the framework developed here takes position uncertainties into account. In the absence of such information, as in indoor scenarios, we propose an auxiliary localization mechanism. Furthermore, we derive the Cramer-Rao bounds for the system, which show that it enables synchronization accuracy at sub-nanosecond levels. Finally, we develop and evaluate an online estimation method, which is statistically efficient.
In this paper we predict spatial wireless channel characteristics using a stochastic model that takes into account both distance dependent pathloss and random spatial variation due to fading. This information is valuable for resource allocation, interference management, design in wireless communication systems. The spatial field model is trained using a convex covariance-based learning method which can be implemented online. The resulting joint learning and prediction method is suitable for large-scale or streaming data. The online method is first demonstrated on a synthetic dataset which models pathloss and medium-scale fading. We compare the method with a state-of-the-art scalable batch method. It is subsequently tested in a real dataset to capture small-scale variations.