Probabilistic Clustering and Shape Modelling of White Matter Fibre Bundles using Regression Mixtures

We present a novel approach for probabilistic clustering of white matter fibre pathways using curve-based regression mixture modelling techniques in 3D curve space. The clustering algorithm is based on a principled method for probabilistic modelling of a set of fibre trajectories as individual sequences of points generated from a finite mixture model consisting of multivariate polynomial regression model components. Unsupervised learning is carried out using maximum likelihood principles. Specifically, conditional mixture is used together with expectation-maximisation (EM) algorithm to estimate cluster membership. The result of clustering is the probabilistic assignment of fibre trajectories to each cluster and an estimate of the cluster parameters. A statistical model is calculated for each clustered fibre bundles using fitted parameters of the probabilistic clustering. We illustrate the potential of our clustering approach on synthetic data and real data.