Skip to main content

Tension in big data using machine learning: Analysis and applications

Wang, Huamao, Yao, Yumei, Salhi, Said (2020) Tension in big data using machine learning: Analysis and applications. Technological Forecasting and Social Change, 158 . Article Number 120175. ISSN 0040-1625. (doi:10.1016/j.techfore.2020.120175) (Access to this publication is currently restricted. You may be able to access a copy if URLs are provided) (KAR id:82537)

PDF Author's Accepted Manuscript
Language: English

Restricted to Repository staff only until 29 December 2021.

Contact us about this Publication
[thumbnail of Tension_Big_Data (OpenAccess).pdf]
Official URL


The access of machine learning techniques in popular programming languages and the exponentially expanding big data from social media, news, surveys, and markets provide exciting challenges and invaluable opportunities for organizations and individuals to explore implicit information for decision making. Nevertheless, the users of machine learning usually find that these sophisticated techniques could incur a high level of tensions caused by the selection of the appropriate size of the training data set among other factors. In this paper, we provide a systematic way of resolving such tensions by examining practical examples of predicting popularity and sentiment of posts on Twitter and Facebook, blogs on Mashable, news on Google and Yahoo, the US house survey, and Bitcoin prices. Interesting results show that for the case of big data, using around 20% of the full sample often leads to a better prediction accuracy than opting for the full sample. Our conclusion is found to be consistent across a series of experiments. The managerial implication is that using more is not necessarily the best and users need to be cautious about such an important sensitivity as the simplistic approach may easily lead to inferior solutions with potentially detrimental consequences.

Item Type: Article
DOI/Identification number: 10.1016/j.techfore.2020.120175
Uncontrolled keywords: Big data, Machine learning, Data size, Prediction accuracy, Social media
Subjects: H Social Sciences
Divisions: Divisions > Kent Business School - Division > Centre for Logistics and Heuristic Optimisation (do not use)
Depositing User: Said Salhi
Date Deposited: 21 Aug 2020 10:23 UTC
Last Modified: 16 Feb 2021 14:14 UTC
Resource URI: (The current URI for this page, for reference purposes)
Salhi, Said:
  • Depositors only (login required):


Downloads per month over past year