Indian Journal of Science and Technology

Abstract

Objectives:In transportation field, a huge amount of data collected by IoTsystems, remote sensing and other data collection tools brings new challenges, the size of this data becomes extremely big and more complex for traditional techniques of data mining. To deal with this challenge, Apache Spark stand as a powerful large scale distributed computing platform that can be used successfully for machine learning against very large databases. This work employed large-scale machine learning techniques especially Decision Tree with Apache Spark framework for big data analysis to build a model that can predict the factors lead to road accidents based on several input variables related to traffic accidents. Based on this, the predicting model first preprocesses the big accident data and analyze it to create data for a learning system. Empirical results show that the proposed model could provide new information that can assist the decision makers to analyze and improve road safety.

Keywords: Data mining, Big Data, Road accident, Decision Tree, Apache Spark, Mllib