Logo: to the web site of Uppsala University

Word embeddings have become an increasingly important class of models for analysis of textual data. We introduce probabilistic embeddings using Laplacian graph priors (PELP). The proposed model enables incorporating graph side-information into static word embeddings. We theoretically show that the model unifies several previously proposed embedding methods under one umbrella, such as graph-enhanced, group, dynamic, and cross-lingual static word embeddings.  Furthermore, we empirically show that our model matches or improves the performance of previous models that are special cases of the PELP model. PELP also enables any combination of these previous models in a  straightforward fashion for flexible modeling of textual data in a Bayesian fashion. We demonstrate the flexibility of using the graph-prior for embeddings by applying it to the comparison of political sociolects over time. Finally, we provide code as a TensorFlow implementation, enabling flexible and scalable estimation for settings with millions of documents.

We study Bayesian inference for probabilistic word embedding models, including skip-gram with negative sampling (SGNS) and continuous bag-of-words (CBoW). These models are widely used for inference, but their posteriors are non-identifiable under general linear transformations, which prevents meaningful Bayesian uncertainty quantification. We provide a simple and principled identifiability constraint that yields a well-defined posterior without altering the data model. Building on this, we develop scalable, fully Bayesian posterior sampling algorithms for SGNS and CBoW using Pólya–Gamma augmentation. Our block Gibbs samplers yield exact samples from the posterior and scale well to large corpora due to their straightforward parallelisation. We discuss the efficient implementation of the Gibbs sampler, and provide a fast TensorFlow reference implementation. In a combination of simulations and empirical experiments, we demonstrate that the sampler is both accurate and computationally efficient. It matches the accuracy of NUTS-HMC at a fraction of the computational cost and avoids the variance underestimation of mean-field variational inference. On large real datasets, we show that posterior means yield improved predictive performance, while posterior credible intervals provide meaningful uncertainty quantification for semantic similarity.

Data accuracy is crucial for reliable research, accurate decision-making, and high-performance machine learning.However, maintaining and improving data accuracy, especially at scale, is challenging and difficult to verify. By combining statistical quality control and branching process theory, we formalize an iterative data curation framework that scales well to large data sets. The framework is agnostic to the source and reliability of proposed data edits, allowing human annotators, automated rules, machine learning models, large language models, and external data sources to be combined within a common statistical process. We prove that under mild assumptions, the proposed iterative approach asymptotically eliminates all errors with probability one for both tabular and textual data.Additionally, we provide guarantees that data accuracy testing further accelerates error reduction. We corroborate these results through simulations on text and tabular data, as well as a real-world application, demonstrating the framework's effectiveness in improving the accuracy of data at scale.

The Swedish parliamentary records are an important source material for social science and humanities researchers. We introduce a new research corpus, the Swedish Parliament Corpus, which is larger and more developed than previously available research corpora for the Swedish parliament. The corpus contains annotated and structured parliamentary records over more than 150 years, through the bicameral parliament (1867–1970) and the unicameral parliament (1971–). In addition to the records, which contain all speeches in the parliament, we also provide a database of all members of parliament over the same period. Along with the corpus, we describe procedures to ensure data quality. The corpus facilitates detailed analysis of parliamentary speeches in several research fields.