Indian Journal of Science and Technology

Abstract

Objective: With the emergence of the notion of “Internet of Things (IoT)”, colossal amount of information is being generated through the sensors and other computing devices and chips. This paper is an attempt to provide a lucid comparison among three prominent technologies used for handling Big Data, viz. Hadoop Distributed File System, Cassandra file system and Quantcast file system. Apart for these three premier file systems, the paper also explores a newly proposed A train Distributed System for handling Big Data. Methods: An inner perspective of the above stated file systems in details considering various aspects for handling big data has been described. The paper also provides sagacity on the situations wherein these technologies are useful. Findings: Effective tackling of the five V’s (Variety, Volume, Velocity, Veracity and Value) of Big Data has become a challenging task for the researcher around the world. Hadoop is one such technology which is open source and is capable of handling big data in an effective manner. It breaks the big data into fixed sized chunks known as block and these blocks are saved at distinct locations in a distributed manner. The Cassandra file system is an alternative to Hadoop which eliminates the single point failure problem of Hadoop as it follows master-less peer to peer distributed ring architecture instead of client server architecture. The third technology is the quantcast file system which is written in C++ language. It also handles the big data in an effective and efficient manner. Moreover it claims to save upto fifty percent of the disk space by implementing erasure encoding. Application: The concerned organization to use any of these available frameworks for handling big data depending upon their nature of needs.

Keywords: A Train, ADS, Cassandra, Hadoop, HDFS, HD Insight, Quantcast