Skip to main content

Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers

Burcham, Morgan, Al-Zyoud, Mahran, Carver, Jeffrey C., Alsaleh, Mohammed, Du, Hongying, Gilani, Fida, Jiang, Jun, Rahman, Akond, Kafalı, Özgur, Al-Shaer, Ehab, and others. (2017) Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers. In: Proceedings of the Hot Topics in Science of Security: Symposium and Bootcamp. . pp. 13-23. ACM ISBN 978-1-4503-5274-1. (doi:10.1145/3055305.3055307) (The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided)

The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided. (Contact us about this Publication)
Official URL
http://dx.doi.org/10.1145/3055305.3055307

Abstract

Scientific advancement is fueled by solid fundamental research, followed by replication, meta-analysis, and theory building. To support such advancement, researchers and government agencies have been working towards a "science of security". As in other sciences, security science requires high-quality fundamental research addressing important problems and reporting approaches that capture the information necessary for replication, meta-analysis, and theory building. The goal of this paper is to aid security researchers in establishing a baseline of the state of scientific reporting in security through an analysis of indicators of scientific research as reported in top security conferences, specifically the 2015 ACM CCS and 2016 IEEE S&P proceedings. To conduct this analysis, we employed a series of rubrics to analyze the completeness of information reported in papers relative to the type of evaluation used (e.g. empirical study, proof, discussion). Our findings indicated some important information is often missing from papers, including explicit documentation of research objectives and the threats to validity. Our findings show a relatively small number of replications reported in the literature. We hope that this initial analysis will serve as a baseline against which we can measure the advancement of the science of security.

Item Type: Conference or workshop item (Proceeding)
DOI/Identification number: 10.1145/3055305.3055307
Uncontrolled keywords: Science of Security, Literature Review
Subjects: Q Science > Q Science (General)
Divisions: Faculties > Sciences > School of Computing
Faculties > Sciences > School of Computing > Security Group
Depositing User: Ozgur Kafali
Date Deposited: 04 Feb 2018 12:31 UTC
Last Modified: 24 Jul 2019 10:26 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/65876 (The current URI for this page, for reference purposes)
  • Depositors only (login required):