Skip to main content
Kent Academic Repository

Recognizing study lineage is not enough: Over-crediting same-trial evidence in LLM-based medical verification

Krishna, Mukand and Harit, Anoushka and Xiao, Chenghao and Sun, Zhongtian (2026) Recognizing study lineage is not enough: Over-crediting same-trial evidence in LLM-based medical verification. [Preprint] (doi:10.48550/arXiv.2610.23721) (The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided) (KAR id:117532)

The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided.
Official URL:
https://doi.org/10.48550/arXiv.2610.23721

Abstract

Publication count is not study count. A model can identify shared trial lineage correctly in a direct task yet over-credit a second publication from the same trial when aggregating evidence. We evaluate this behavior in medical LLM verification using 14 source-audited clinical families and five model conditions. The families span provenance, clinical applicability, result direction, and composition; the provenance families provide the direct lineage tests. Recognition and aggregation are separate stateless calls, and repeated calls assess within-input stability. Across all three original provenance families, GPT-5.5 identifies the independent update in 10/10 recognition calls per family but over-credits the same-trial report in 5/5 separate aggregation calls per family. Selected Qwen3-4B and Ministral-3-8B conditions show the same cross-task contrast. A study-grouping instruction reduces GPT-5.5 over-credit from 4/5 to 0/5 in each later pure-provenance family. Other model conditions show mixed behavior, and rejected outputs limit several comparisons. The results identify a behavioral gap between elicited lineage recognition and evidence aggregation. The code and data artifacts are available at https://anonymous.4open.science/r/study-lineage-review-artifact.

Item Type: Preprint
DOI/Identification number: 10.48550/arXiv.2610.23721
Refereed: No
Other identifier: https://arxiv.org/abs/2610.23721
Name of pre-print platform: arXiv
Subjects: Q Science > QA Mathematics (inc Computing science)
Institutional Unit: Schools > School of Computing
Former Institutional Unit:
There are no former institutional units.
Funders: University of Kent (https://ror.org/00xkeyj56)
Depositing User: Zhongtian Sun
Date Deposited: 05 Nov 2026 15:41 UTC
Last Modified: 06 Nov 2026 10:31 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/117532 (The current URI for this page, for reference purposes)

University of Kent Author Information

Krishna, Mukand.

Creator's ORCID:
CReDIT Contributor Roles:

Sun, Zhongtian.

Creator's ORCID: https://orcid.org/0000-0003-0489-5203
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views of this page since July 2020. For more details click on the image.