This thesis aims to explore ways to efficiently annotate text segments in the records of the Swedish parliamentary proceedings, with the classification of titles and decisions. As the corpus consists of more than 11.5 million sequences with varying structure and quality, spanning a time period of more than a hundred years, the work of curating the corpus is challenging, and the cost of annotating the data is substantial. Therefore, active learning in conjunction with Bidirectional Encoder Representations from Transformers (BERT) models is used. The active learning strategy used is uncertainty sampling with prediction entropy as the uncertainty measure. On average, the active learning strategy outperforms a random sampling strategy, suggesting that there is some benefit to using deep pre-trained neural networks in conjunction with active learning methods in classifying titles and decisions in parliamentary records. However,it did not substantially improve the performance on the test set when compared to a baseline model trained on the initial training set already performing well with a micro-averaged F1 score of 0.9556 and subset accuracy of 0.9843. The resulting sets of selected unlabeled sequences with the highest entropy suggest that using an uncertainty sampling method based on prediction entropy in conjunction with BERT on records of parliamentary proceedings diminished the model uncertainty surrounding text sequences from underrepresented periods with lower data quality.