Machine learning methods applied to audit of surgical outcomes after treatment for cancer of the head and neck

Most surgical specialties have attempted to address concerns about unfair comparison of outcomes by “risk-adjusting” data to benchmark specialty-specific outcomes that are indicative of the quality of care. We are building on previous work in head and neck surgery to address the current need for a robust validated means of risk adjustment. A dataset of care episodes, which were recorded as a clinical audit of complications after operations for squamous cell carcinoma (SCC) of the head and neck (n = 1254), was analysed with the Waikarto Environment for Knowledge Analysis (WEKA) machine learning tool. This produced 4 classification models that could predict complications using data on the preoperative demographics of the patients, operation, functional status, and tumour stage. Three of them performed acceptably: one that predicted “any complication” within 30 days (area under the receiver operating characteristic curve (AUROC) 0.72), one that predicted severe complications (Clavien-Dindo grade 3 or above) within 30 days (AUROC 0.70), and one that predicted a prolonged duration of hospital stay of more than 15 days, (AUROC 0.81). The final model, which was developed on a subgroup of patients who had free tissue transfer (n = 443), performed poorly (AUROC 0.59). Subspecialty groups within oral and maxillofacial surgery are seeking metrics that will allow a meaningful comparison of the quality of care delivered by surgical units in the UK. For these metrics to be effective they must show variation between units and be amendable to change by service personnel. Published baseline data must also be available. They should be modelled effectively so that meaningful comparison, which takes account of variations in the complexity of the patients’ needs or care, is possible.