High Impact Factor : 4.396 icon | Submit Manuscript Online icon |

Frequent Closed Item-Sets for Association Rules based on Hadoop


Parkale Bhagyashri Ganpat , SVPMCOE Malegaon(bk.); Jagtap Aarati Shrirang, SVPMCOE Malegaon(bk.); Shedge Kajal Hanumant, SVPMCOE Malegaon(bk.); Tarade Pratidnya Rajendra, SVPMCOE Malegaon(bk.)


Association Rules Mining; Closed Itemsets; Map Reduce; Hadoop; Big Data


In this paper we introduce Traditional parallel algorithms for mining frequent itemsets aim to balance load by equally partitioning data among a group of computing nodes. Proposed system start this study by discovering a serious performance problem of the existing parallel Frequent Itemset Mining algorithms. Assign a large dataset, data partitioning strategies in the existing solutions suffer high communication and mining overhead induced by redundant transactions transmitted among computing nodes. This paper address problem by developing a data partitioning approach called FiDoop-DP using the MapReduce programming model. The Overall goal of FiDoop-DP is to boost the performance of parallel Frequent Itemset Mining on Hadoop clusters. At the heart of FiDoop-DP is the Voracity diagram-based data partitioning technique, which exploits correlations among transactions. Include the similarity metric and the Locality-Sensitive Hashing technique, FiDoop-DP places highly similar transactions into a data partition to improve locality without creating an excessive number of redundant transactions. This paper implement FiDoop-DP on a 24-node Hadoop cluster, driven by a wide range of datasets created by IBM Quest Market-Basket Synthetic Data Generator. Experimental results disclose that FiDoop-DP is conducive to decrease network and computing loads by the virtue of eliminating redundant transactions on Hadoop nodes. FiDoop-DP importantly improves the performance of the existing parallel frequent-pattern scheme by up to 31% with an average of 18%.

Other Details

Paper ID: IJSRDV6I90082
Published in: Volume : 6, Issue : 9
Publication Date: 01/12/2018
Page(s): 23-25

Article Preview

Download Article