High Impact Factor : 4.396 icon | Submit Manuscript Online icon |

Map Reduce Optimizer with an Intermediary Cache Manager


Ms Vishakha Mehendale , SES?SFOE College of Engineering, Diksal, Raigad, Mumbai University, India.; Prof Mrs. Shital Dhamal, Lokmanya Tilak College of Engineering, Koparkhairne , Navi Mumbai, Mumbai University,India.


Big data, MapReduce, Incremental Processing, Cache


Big data is a term used to address the data set whose size is beyond the ability of traditional software technologies to capture, store, manage and process within a tolerable elapsed time. In the world of big data, we observe daily based data which is generated daily e.g. Google, Facebook, and Amazon etc. This large volume of data is un reliable to store, manage and analyze which runs on commodity hardware. As data is in large size it takes more time to execute. The MapReduce framework generates a large amount of intermediate data. These data thrown away after the tasks finish. MapReduce is unable to utilize these data. To improve the efficiency of MapReduce functionality by reducing repeated jobs in data nodes, we proposed an Intermediary cache management system inside the MapReduce framework. In which, tasks submit their intermediate results to the cache manager. Before executing the actual computing work, task queries the cache manager. In an Intermediary cache Management, cache request and cache reply mechanisms are designed. It detects the occurrence of repeated job in the incremental data process.

Other Details

Paper ID: IJSRDV5I120119
Published in: Volume : 5, Issue : 12
Publication Date: 01/03/2018
Page(s): 148-151

Article Preview

Download Article