Skip to main content

Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers

Burcham, Morgan and Al-Zyoud, Mahran and Carver, Jeffrey C. and Alsaleh, Mohammed and Du, Hongying and Gilani, Fida and Jiang, Jun and Rahman, Akond and Kafalı, Özgur and Al-Shaer, Ehab and Williams, Laurie (2017) Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers. In: Proceedings of the Hot Topics in Science of Security: Symposium and Bootcamp. ACM-ICPS International Conference Proceeding Series . ACM, New York, USA, pp. 13-23. ISBN 978-1-4503-5274-1. (doi:10.1145/3055305.3055307) (The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided)

The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided. (Contact us about this Publication)
Official URL
http://dx.doi.org/10.1145/3055305.3055307

Abstract

Scientific advancement is fueled by solid fundamental research, followed by replication, meta-analysis, and theory building. To support such advancement, researchers and government agencies have been working towards a "science of security". As in other sciences, security science requires high-quality fundamental research addressing important problems and reporting approaches that capture the information necessary for replication, meta-analysis, and theory building. The goal of this paper is to aid security researchers in establishing a baseline of the state of scientific reporting in security through an analysis of indicators of scientific research as reported in top security conferences, specifically the 2015 ACM CCS and 2016 IEEE S&P proceedings. To conduct this analysis, we employed a series of rubrics to analyze the completeness of information reported in papers relative to the type of evaluation used (e.g. empirical study, proof, discussion). Our findings indicated some important information is often missing from papers, including explicit documentation of research objectives and the threats to validity. Our findings show a relatively small number of replications reported in the literature. We hope that this initial analysis will serve as a baseline against which we can measure the advancement of the science of security.

Item Type: Book section
DOI/Identification number: 10.1145/3055305.3055307
Uncontrolled keywords: Science of Security, Literature Review
Subjects: Q Science > Q Science (General)
Divisions: Faculties > Sciences > School of Computing > Security Group
Depositing User: Ozgur Kafali
Date Deposited: 04 Feb 2018 12:31 UTC
Last Modified: 24 Sep 2019 09:09 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/65876 (The current URI for this page, for reference purposes)
  • Depositors only (login required):