Indian Journal of Science and Technology

Abstract

Data in this era is generating at tremendous rate so now it is need of today to handle the data to gain useful insight, this data can be useful for researcher and accommodation to do analysis. As we know traditional system cannot handle more than terabytes of data since it affects performance and also storage is very costly. Big data is an innovative technique analyze, store, manage, distributes and capture datasets. Objective: To achieve compressed storage, we implement a parallel mining algorithm called as Implementation of Parallel Mining for Big data. Method: Hadoop is a platform which enables the distributing processing using map reduces programming. This help in getting result at very fast rate as result in less time help in competing for growth of business. Unstructured datasets is taken for analysis which is real time is taken and converted to structured format and process in map reduces. It is found in literature existing mining algorithm for the real time datasets which always lacks in fault tolerance, load balancing, data distribution and automatic parallelization. To overcome these disadvantages we implement map reduce for association analysis. Finding/Improvement: In IPB we improve performance in the computing node the load is distributed. In our proposed solution we use real-world celestial spectral data. The graphical representation of traditional system comparison with Hadoop is shown in this paper.

Keywords: Big Data, IPB (Implementation of Parallel Mining for Big Data), Map Reduce, Parallel Mining, Hadoop Association Analysis