Equi-Depth Histogram Construction Methodology for Big Data Tools

Home Page
About
Submit A Journal
Submit A Conference
Submit Paper/Book
- Submit a Preprint
- Submit a Book
Contact

Politeknik Dergisi
Volume:23 Issue:3
Equi-Depth Histogram Construction Methodology for Big Data Tools

Equi-Depth Histogram Construction Methodology for Big Data Tools

Authors : Tolga BÜYÜKTANIR, Ahmet Ercan TOPCU

Pages : 859-865

Doi:10.2339/politeknik.620198

View : 58 | Download : 9

Publication Date : 2020-09-01

Article Type : Research Paper

Abstract :In recent decades, countless data sources such as social media, machines, and networks are constantly pushing data into the digital world. The size of the data has been growing exponentially. To understand the statistical information of data query optimization, equi-depth histograms are essential. In this paper, we present approximate equi-depth histogram construction for big data using both Apache Pig Scripts and Java Web Interface interacting with Apache Hadoop. We use equi-depth histogram construction with quality guarantees for big data approaches and implement them with Apache Hadoop Map-Reduce and Apache Pig user-defined functions. We introduce a prototype implementation of the construction of the approximate equi-depth histogram from the Java Server Face page using Apache Hadoop jobs and the Hadoop Distributed Files System, and we evaluate these methods using the demonstration. We explain Apache Pig Scripts techniques to create equi-depth histograms using big data. The results indicate that our system provides the capability of writing multiple jobs using Apache Pig, and programmers can make use of the advantages of Apache Pig to create histograms and eliminate the complex implementation of Map-Reduce jobs.
Keywords : Approximate histogram, merging histograms, big data, log files, hadoop distributed file system

ORIGINAL ARTICLE URL

* There may have been changes in the journal, article,conference, book, preprint etc. informations. Therefore, it would be appropriate to follow the information on the official page of the source. The information here is shared for informational purposes. IAD is not responsible for incorrect or missing information.