Logo: to the web site of Uppsala University

This work evaluates the performance of longitudinal item response (IR) theory models in shortened assessments using an existing model for part II and III of the MDS-UPDRS score. Based on the item information content, the assessment was reduced by removal of items in multiple increments and the models' ability to recover the item characteristics of the remaining items at each level was evaluated. This evaluation was done for both simulated and real data. The metric of comparison in both cases was the item information function. For real data, the impact of shortening on the estimated disease progression and drug effect was also studied. In the simulated data setting, the item characteristics did not differ between the full and the shortened assessments down to the lowest level of information remaining; indicating a considerable independence between items. In contrast when reducing the assessment in a real data setting, a substantial change in item information was observed for some of the items. Disease progression and drug effect estimates also decreased in the reduced assessments. These changes indicate a shift in the measured construct of the shortened assessment and warrant caution when comparing results from a partial assessment with results from the full assessment.

There are examples in the literature demonstrating different approaches to defining the item characteristic functions (ICF) and characterizing the latent variable time-course within a pharmacometrics item response theory (IRT) framework. One such method estimates both the ICF and latent variable time-course simultaneously, and another method establishes the ICF first then models the latent variable directly. To date, a direct comparison of the "simultaneous" and "sequential" methodologies described in this work has not yet been systematically investigated. Item parameters from a graded response IRT model developed from Parkinson's Progression Marker Initiative (PPMI) study data were used as simulation parameters. Each method was evaluated under the following conditions: (i) with and without drug effect and (ii) slow progression rate with smaller sample size and rapid progression rate with larger sample size. Overall, the methods performed similarly, with low bias and good precision for key parameters and hypothesis testing for drug effect. The ICF parameters were well determined when the model was correctly specified, with an increase in precision in the scenario with rapid progression. In terms of drug effect, both methods had large estimation bias for the slow progression rate; however, this bias can be considered small relative to overall progression rate. Both methods demonstrated type 1 error control and similar discrimination between model with and without drug effect. The simultaneous method was slightly more precise than the sequential method while the sequential method was more robust towards longitudinal model misspecification and offers practical advantages in model building.

Approximately 15% of patients suspected of having Parkinson's disease (PD) present dopamine active transporter (DaT) scans without evidence of dopaminergic deficits (SWEDD), most of which will never develop PD. Leveraging Movement Disorders Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) scores from the Parkinson's Progression Markers Initiative, three different models of varying complexity, (total score, item response theory (IRT) and artificial neural network (ANN)) were evaluated to determine their ability to differentiate between PD and SWEDDs. Each of the models provided as output a predicted probability of having PD (P_DeNoPD). Both the IRT and ANN methods performed well as classifiers; ROC AUC > 80%, sensitivity > 93%, and precision ~90% when assuming a probability cutoff of P_DeNoPD ≥ 50%. Specificity was 43% and 38% for IRT and ANN respectively. Matthews correlation coefficient (MCC) was also evaluated as a metric to address potential bias of majority positive class. At all cutoffs at or above 50%, the IRT and ANN model performed similarly and achieved a MCC of at least 0.3, indicating at least a moderate positive relationship for classifier performance. In contrast, the total score model was a poor classifier, for all metrics and cutoffs. Using item-level data the proposed methodologies differentiated PD patients from SWEDDs with a degree of sensitivity and specificity that may compete with clinical examination and could aid in selecting DaTscan candidates. The choice of cutoff criteria, quality metric, and classifier model are contingent upon specific clinical needs.

In Alzheimer’s disease (AD), ADAS-Cog is a composite assessment of cognitive function that is considered a gold standard for diagnosis and often serves as the endpoint in clinical trials seeking to demonstrate cognitive benefit. Symptomatic treatments are a much-needed option for quick improvement of AD symptoms to provide patients a better quality of life.  In development of symptomatic treatments, the ability to identify a meaningful treatment effect earlier and in trials of limited duration and size are critical. However, in the context of variability in ADAS-Cog, detection of a symptomatic treatment effect may be challenging.  Item response theory (IRT) is a statistical methodology for the analysis of composite scores which describes the relationship between a subject disease severity and the probability of a response at the item level and often with increased precision, to traditional approaches. This work explores the use of IRT to detect symptomatic treatment effects on ADAS-Cog. Based on clinical trial simulations, IRT-based analysis had higher power to detect a treatment effect associated with a 2-point change in ADAS-Cog11 compared to pairwise comparison and traditional longitudinal modeling of composite score, highlighting the potential to apply IRT to analysis of clinical trials of limited duration and size.