Krishna, Mukand and Harit, Anoushka and Xiao, Chenghao and Sun, Zhongtian (2026) Recognizing study lineage is not enough: Over-crediting same-trial evidence in LLM-based medical verification. [Preprint] (doi:10.48550/arXiv.2610.23721) (The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided) (KAR id:117532)
| The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided. | |
| Official URL: https://doi.org/10.48550/arXiv.2610.23721 |
|
Abstract
Publication count is not study count. A model can identify shared trial lineage correctly in a direct task yet over-credit a second publication from the same trial when aggregating evidence. We evaluate this behavior in medical LLM verification using 14 source-audited clinical families and five model conditions. The families span provenance, clinical applicability, result direction, and composition; the provenance families provide the direct lineage tests. Recognition and aggregation are separate stateless calls, and repeated calls assess within-input stability. Across all three original provenance families, GPT-5.5 identifies the independent update in 10/10 recognition calls per family but over-credits the same-trial report in 5/5 separate aggregation calls per family. Selected Qwen3-4B and Ministral-3-8B conditions show the same cross-task contrast. A study-grouping instruction reduces GPT-5.5 over-credit from 4/5 to 0/5 in each later pure-provenance family. Other model conditions show mixed behavior, and rejected outputs limit several comparisons. The results identify a behavioral gap between elicited lineage recognition and evidence aggregation. The code and data artifacts are available at https://anonymous.4open.science/r/study-lineage-review-artifact.
| Item Type: | Preprint |
|---|---|
| DOI/Identification number: | 10.48550/arXiv.2610.23721 |
| Refereed: | No |
| Other identifier: | https://arxiv.org/abs/2610.23721 |
| Name of pre-print platform: | arXiv |
| Subjects: | Q Science > QA Mathematics (inc Computing science) |
| Institutional Unit: | Schools > School of Computing |
| Former Institutional Unit: |
There are no former institutional units.
|
| Funders: | University of Kent (https://ror.org/00xkeyj56) |
| Depositing User: | Zhongtian Sun |
| Date Deposited: | 05 Nov 2026 15:41 UTC |
| Last Modified: | 06 Nov 2026 10:31 UTC |
| Resource URI: | https://kar.kent.ac.uk/id/eprint/117532 (The current URI for this page, for reference purposes) |
- Export to:
- RefWorks
- EPrints3 XML
- BibTeX
- CSV
- Depositors only (login required):

https://orcid.org/0000-0003-0489-5203
Altmetric
Altmetric