Inter-observer agreement of thoracolumbar fascia morphology: an exploratory analysis of ultrasound images

BACKGROUND: Ultrasound imaging (USI) has been shown to be a valid method to investigate the morphology of the thoracolumbar fascia (TLF) [1]. A USI-based study has demonstrated that the TLF of subjects with chronic lower back pain (LBP) is on average 25% thicker and more disorganised compared to a control group [1]. The aim of this study is to explore inter-observer agreement between a range of clinicians on (dis)organisation of TLF in ultrasound images. There are currently no validated methods for the evaluation of USI of TLF.

METHODS: Design: an exploratory analysis using a fully crossed design of inter-observer agreement. This study was approved by the University of Kent’s School of Sport and Exercise Sciences Research and Ethics Committee (Prop. 163 – 2013). Participants: Thirty observers consisting of 21 (70%) Medical Doctors, 7 (23%) physiotherapists and 2 (6%) radiologists, with a combined total average of 13 years of clinical experience (± SD 9.4). 57% had no experience in USI, 36% had experience ranging from monthly to daily evaluations of USI, no observers had experience in evaluating USI of TLF. Protocol: A sub-set of thirty ultrasound scans of TLF were randomly selected from a data set of 308 scans of subjects with and without LBP (from a larger study conducted by the first author). All scans were anonymised and displayed on a desktop computer, or projected on a screen. All observers viewed and rated each of the 30 scans independently on a Likert-type scale from 1(very disorganised) to 10 (very organised). Inter-observer agreement was assessed using a two-way mixed, consistency, average measures intra-class correlation (ICC), the Cronbach’s Alpha, to assess consistency among observers. The Krippendorff’s Alpha (Kalpha) [2] reliability estimate was used to assess agreement.

RESULTS: The resulting ICC was in the excellent range, ICC = 0.98, indicating that observers had a high degree of consistency, suggesting that (dis)organisation was rated similarly across observers. Observers without USI experience scored an ICC = 0.96, observers with USI experience scored an ICC = 0.95, again both in the excellent range. In this small cohort, experience in USI does not appear to impact on consistency. The Krippendorff’s ordinal alpha ? was .621, indicating a modest degree of agreement.

CONCLUSIONS: The high ICC and modest Kalpha suggest that a minimal amount of measurement error was introduced by the independent observers, and therefore statistical power for subsequent analyses is not substantially reduced. This will allow for further analysis of USI images of TLF in terms of morphology and classification. This could ultimately, lead to a meaningful evaluation of treatments of TLF.