Dynamic Data Partitioning & Virtual Chunking for Effective Data Retrival Using Big Data & Cloud

Today’s science is generating signi�cantly larger volume of data than before, data compression can potentially improve the application performance. In some high performance computing system data compression could ameloirate the I/O pressure. A new concept called as virtual chunking is introduced that aims to better employ the compression algorithms in parallel. ABE based keys are distributed among the corresponding admin so that access policy is applied. Data is splitted and stored using dynamic virtualization concept. A small number of references is appended to the end of file. The data’s are partitioned into three layers of connectivity. The details of the user will be stored in the database. Once the User login to an account, and request the Job from the Service Provider the results would be displayed based on the user’s query. Based on the request, the Service Provider will process the job and respond to them. Using map reduce technique the data’s are classified according to the query. Time frame is being generated at each level of connectivity. This process will reduce the time for the fetching any results from the table.