Skip to main content
Kent Academic Repository

Challenges Encountered and Lessons Learned When Using a Novel Anonymised Linked Dataset of Health and Social Care Records for Public Health Intelligence: The Sussex Integrated Dataset

Ford, Elizabeth, Tyler, Richard, Johnston, Natalie, Spencer-Hughes, Vicki, Evans, Graham, Elsom, Jon, Madzvamuse, Anotida, Clay, Jacqueline, Gilchrist, Kate, Rees-Roberts, Melanie and others. (2023) Challenges Encountered and Lessons Learned When Using a Novel Anonymised Linked Dataset of Health and Social Care Records for Public Health Intelligence: The Sussex Integrated Dataset. Information, 14 (2). Article Number 106. ISSN 2078-2489. (doi:10.3390/info14020106) (KAR id:100116)


Background: In the United Kingdom National Health Service (NHS), digital transformation programmes have resulted in the creation of pseudonymised linked datasets of patient-level medical records across all NHS and social care services. In the Southeast England counties of East and West Sussex, public health intelligence analysts based in local authorities (LAs) aimed to use the newly created “Sussex Integrated Dataset” (SID) for identifying cohorts of patients who are at risk of early onset multiple long-term conditions (MLTCs). Analysts from the LAs were among the first to have access to this new dataset. Methods: Data access was assured as the analysts were employed within joint data controller organisations and logged into the data via virtual machines following approval of a data access request. Analysts examined the demographics and medical history of patients against multiple external sources, identifying data quality issues and developing methods to establish true values for cases with multiple conflicting entries. Service use was plotted over timelines for individual patients. Results: Early evaluation of the data revealed multiple conflicting within-patient values for age, sex, ethnicity and date of death. This was partially resolved by creating a “demographic milestones” table, capturing demographic details for each patient for each year of the data available in the SID. Older data (≥5 y) was found to be sparse in events and diagnoses. Open-source code lists for defining long-term conditions were poor at identifying the expected number of patients, and bespoke code lists were developed by hand and validated against other sources of data. At the start, the age and sex distributions of patients submitted by GP practices were substantially different from those published by NHS Digital, and errors in data processing were identified and rectified. Conclusions: While new NHS linked datasets appear a promising resource for tracking multi-service use, MLTCs and health inequalities, substantial investment in data analysis and data architect time is necessary to ensure high enough quality data for meaningful analysis. Our team made conceptual progress in identifying the skills needed for programming analyses and understanding the types of questions which can be asked and answered reliably in these datasets.

Item Type: Article
DOI/Identification number: 10.3390/info14020106
Uncontrolled keywords: Health data; electronic health records; data linkage; data quality; public health
Subjects: H Social Sciences
Divisions: Divisions > Division for the Study of Law, Society and Social Justice > School of Social Policy, Sociology and Social Research > Centre for Health Services Studies
SWORD Depositor: JISC Publications Router
Depositing User: JISC Publications Router
Date Deposited: 17 Feb 2023 11:02 UTC
Last Modified: 20 Feb 2023 11:09 UTC
Resource URI: (The current URI for this page, for reference purposes)

University of Kent Author Information

Rees-Roberts, Melanie.

Creator's ORCID:
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.