Reducing the Network Cost and Handover the Corresponding Request using Bigdata Hadoop |
Author(s): |
| Sabiya Subramanian , Prince shri venkateswara padmavathy engineering college; Pavithra Sivakumar, Prince shri venkateswara padmavathy engineering college; Sakthi Madhusudanan, Prince shri venkateswara padmavathy engineering college; Gayathiri. E, Prince shri venkateswara padmavathy engineering college; Veeralakshmi Ponnuramu, Prince shri venkateswara padmavathy engineering college |
Keywords: |
| MapReduce, Data Partition, Data Aggregation, Data Locality, Key/Value Pair |
Abstract |
|
The MapReduce programming model deals with large scale data processing is fully based on shuffling and sorting the data by exploiting parallel map task and reduce tasks. For the performance enhancement, they ignore network traffic generated in the shuffle phase by the MapReduce Jobs. A hash function is used to partition the intermediate data among reduce tasks which is not traffic optimal due to network topology and data size connected with each key. In the proposed system, we study to reduce the network traffic with the help of data partitioning scheme, we consider to aggregate data with the same keys before sending them to remote reduce tasks. The proposed method named decomposition-based distributed algorithm is dealing with the large-scale optimization problem for big data application. In a dynamic manner, data partition and aggregation is done using online algorithm. |
Other Details |
|
Paper ID: IJSRDV5I10232 Published in: Volume : 5, Issue : 1 Publication Date: 01/04/2017 Page(s): 326-329 |
Article Preview |
|
|
|
|
