Investigation into Machine Learning and Emotional and Engagement Tracking Tools to Support and Enable At-Home Immersive Virtual Therapies

In recent years, the strain on the health system, exacerbated by the pandemic, has impacted mental health services, leading to growing pressures, shortages in mental health staff, and a lengthy waiting list for therapy. As of April 2023, there are 21,754 patients awaiting therapy, with 31.9% waiting for more than 18 weeks. To address these challenges, virtual therapy emerges as a promising solution, capable of alleviating stress on mental health services by offering frequent therapy and continuous monitoring. Utilising virtual reality (VR) technology, this form of psychotherapy creates simulated environments for treating various psychological conditions such as anxiety disorders, phobias, PTSD, and depression. Virtual therapy stands out due to its safe and controlled environment, high customisation potential and enhanced patient engagement. Virtual therapy has shown to be effective in treating many mental health disorders; advancements in technology, research, and accessible hardware have the possibility to expand on this research and create more personalised and remote therapies. To realise this future, ongoing investigations into mechanisms and technology for monitoring patients' reactions, feelings, and progress throughout therapy are essential. These mechanisms become even more important when you consider self-guided or automated at-home therapy.

This thesis presents several outcomes. First, we successfully constructed machine learning (ML) models capable of monitoring the mental health levels of patients diagnosed with treatment- resistant depression undergoing therapy, utilising data collected from Fitbit devices. Additionally, our next study demonstrated the effectiveness of VR in eliciting emotions compared to non- immersive stimuli. We established baseline ML results using a newly verified and published dataset, then enhanced these results through the implementation of deep learning techniques. To achieve real-time detection, we employed small data chunks. Addressing the interpretability challenge inherent in deep learning models, we developed a post hoc XAI system, offering visual explanations for predictions both locally and globally. These explanations were compared with medical and ML literature to gain deeper insights into the model's decision-making process. Furthermore, we investigated ML systems for detecting engagement during virtual experiences using EEG data, providing insights into feature importance and electrode placement. Overall, our research has not only investigated and initiated the development but also verified the viability of multiple tools essential for enabling immersive virtual therapies. Finally, we have offered guidelines on the utilisation of various physiological signals, hardware, and their most effective applications for future endeavours in this field.