Modelling Partial Ranking Data

Ranking is one of the most used methods in not only in statistics but also in other field such as computer science and psychology. This method helps us determine order of objects in a group such as preference of animal species, and has very broad applications. However, when the number of objects to be ranked becomes larger, the uncertainty of the ranking typically increases since it is harder for the ranker to express their preference accurately. This leads to the idea of partial ranking which allows rankers to rank just a subset of objects in the group and then combine their results together to form the global ranking. This thesis focuses on this type of data. The main challenge is how to accurately analyze partially ranked data and decide the global ranking. There are several models that address this kind of problem such as the Bradley-Terry (BT) model and the Plackett-Luce (PL) model.

The BT model is for paired comparisons while the PL model is for any number of ranked objects. The PL model is slow to fit using existing R packages. We implement the algorithms in R and do empirical studies using simulated data. The results show that our algorithms perform faster than the existing packages in R. We also implement R code for computing the observed information matrix. Rank-breaking methods are also considered in order to be able to use the BT model with different weightings instead of using the PL model. We examine the performance of various weightings by experimental studies with the simulated data and with real-world data. Our BTw-Sqrt weighting performs best when the number of rankers is small.

In order to choose subsets of objects to be ranked, we consider three existing criteria which are D-optimality, E-optimality, and Wald and we propose three new methods. Experiments have been done using simulated data and the results compared with random selection. Our result shows that the existing criteria sometimes perform better than random selection. Our proposed methods usually ensure that the PL model can be fitted to data from fewer rankers than random selection.

We describe two extensions of the PL model, the Rank-Ordered Logit (ROL) model and the Benter model. The ROL model extends the PL model by allowing covariates to be incorporated and the Benter model allows preferences for higher-ranked items to be stronger than for lower-ranked items. Both extensions improve the fit of the PL model to an example dataset when using the Likelihood Ratio (LR) test to compare models. We combine these two extensions to give a model that incorporates covariates and allows for a dampening effect. The combined model further improves the fit to our example data when compared with the ROL model by using LR test. We implement R codes for analyzing and computing the observed information matrices of the ROL, Benter, and combined models.

We also explore another type of partial ranking data where individuals are allowed to mention any objects rather than being given a predefined list of objects to rank. We consider the idea of Participatory Risk Mapping (PRM) which provides severity and incidence scores. The severity and incidence scores can be modelled using the PL model and a new proposed model, respectively.