摘要(英) |
The DGAGC algorithm, developed by National Central University, is a classification algorithm based on association-rule mining and searching. The DGAGC algorithm also specifies a distributed computing approach for model training, which is implemented on top of Hadoop MapReduce. In this study, we propose a new distributed computing approach for the DGAGC algorithm based on Apache Spark. With the support of in-memory computing by Spark, the new distributed DGAGC algorithm can achieve less average execution time for model training, given four different training data sets. In addition, we also propose a distributed version of the DGAGC for data classification. |
參考文獻 |
[1] PRIYANK PANDEY ,MANOJ KUMAR and PRAKHAR SRIVASTAVA,”Classification Techniques for Big Data:A Survey”, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), pp:3625-3629,2016.
[2] Min-Yi Tsai, Ping-Fang Chiang, Shao-Jui Chen, Wei-Jen Wang ,”A Decision Generation Algorithm Based on Granular Computing”, 2012 IEEE International Conference on Granular Computing, pp:475-480, 2012.
[3] AMDOUNI Hamida, GAMMOUDI Mohamed Mohsen,” Algorithms of Association Rules Extraction: State of the Art ”,2011 IEEE 3rd International Conference on Communication Software and Networks, pp:698-703, 2011.
[4] A. Bargiela and W. Pedrycz, ”The roots of Granular Computing,” Proceedings of IEEE Granular Computing Conference, pp.741, 2006.
[5]Y.Y. Yao, and J.T. Yao, ”Induction of Classification Rules by Granular Computing”, The Seventh International Conference on Rough Sets and Current Trends in Computing, pp:331-338,2002.
[6] B.Zang,and L.Zhang,”The Quotient Space Theory of Problem Solving”,Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing, lecture Notes in Computer Science, Vol. 2639/2003, pp:585,2003.
[7] Apache Software Foundation,http://Hadoop.apache.org/
[8] Apache Software Foundation,https://Spark.apache.org/
[9] W. Pedrycz, ”Granular Computing: an introduction,” IFSA World Congress and 20th NAFIPS International Conference, pp:1349-1354, 2001.
[10] OpenStack Foundation, https://www.openstack.org/
[11] UCI Machine Learning Repository,https://archive.ics.uci.edu/ml/datasets.html
[12] Lei Gu, Huan Li,“Memory or Time: Performance Evaluation for Iterative Operation on Hadoop and Spark”2013 IEEE International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing, pp:721-727,2013. |