基於分散式運算的知識圖譜建置系統

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：41

、訪客IP：18.117.101.7

姓名

李昱德(Yu-De Li) 查詢紙本館藏

畢業系所

資訊工程學系在職專班

論文名稱

基於分散式運算的知識圖譜建置系統
(Distributed Computing for Building Knowledge Graph System)

相關論文

★ 整合GRAFCET虛擬機器的智慧型控制器開發平台	★ 分散式工業電子看板網路系統設計與實作
★ 設計與實作一個基於雙攝影機視覺系統的雙點觸控螢幕	★ 智慧型機器人的嵌入式計算平台
★ 一個即時移動物偵測與追蹤的嵌入式系統	★ 一個固態硬碟的多處理器架構與分散式控制演算法
★ 基於立體視覺手勢辨識的人機互動系統	★ 整合仿生智慧行為控制的機器人系統晶片設計
★ 嵌入式無線影像感測網路的設計與實作	★ 以雙核心處理器為基礎之車牌辨識系統
★ 基於立體視覺的連續三維手勢辨識	★ 微型、超低功耗無線感測網路控制器設計與硬體實作
★ 串流影像之即時人臉偵測、追蹤與辨識─嵌入式系統設計	★ 一個快速立體視覺系統的嵌入式硬體設計
★ 即時連續影像接合系統設計與實作	★ 基於雙核心平台的嵌入式步態辨識系統

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2027-6-25以後開放)

摘要(中)

在知識圖譜的快速發展，如今相關應用越來越豐富，但伴隨著資料量日漸龐大，使圖譜建立上的複雜度也提高，如果建立圖譜的時間成本過高，則會影響服務的實時性，所以如何在大資料量的情境下，高效的建立知識圖譜是一個重要的議題。本論文以基於Hadoop + Spark分散式運算的架構來針對RDB-to-RDF的情境來建立知識圖譜建置系統，並搭建了實驗環境，與Antidot公司開源DB2Triples系統作為比較的對象。在實驗環境中，採用了圖書館借閱開放資料，一共約有近九千三百萬筆的資料，來模擬現實中大量資料的情境，比較時採以漸進式的方式，從小的資料量開始累進至大的資料量，來比較在不同的資料量時的圖譜建置效能，論文最後則依據實驗數據來做綜合性的評比。從實驗結果可以得知，在千筆資料以內時，DB2Triples擁有比較快速的圖譜建置時間，但約莫到兩千筆資料時，本論文所實現的分散式系統已實現反超，到達一萬筆時，已經快了約六倍，且隨著資料量的累進，差距則越來越明顯。

摘要(英)

With the rapid development of knowledge graphs, related applications are becoming more and more abundant, but with the increasing amount of data, the complexity of graph establishment has also increased. If the time cost of establishing graphs is too long, it will affect the real-time performance of services. Therefore, how to efficiently build a knowledge graph in the context of a large amount of data is an important issue. In this paper, according to the distributed computing architecture based on Hadoop + Spark, a knowledge graph construction system is established for the RDF-to-RDF situation, and an experimental environment is built, which is compared with Antidot′s open-source DB2Triples system. In the experiment, the library’s open-source materials were borrowed, with a total of nearly 93 million pieces of materials, to simulate the situation of a large number data source in reality. This experiment adopts a progressive method when comparing, in order to compare the building performance with different amounts of data. Starting from a small amount of data and increasing to a large amount of data. Finally, make a comprehensive evaluation based on the experimental data. From the experimental results, when there is less than 1000 data , DB2Triples has a faster building time. But when it reaches about 2,000 data, distributed computing has surpassed the former. When it reaches about 10,000 data, distributed computing is now 6 times faster. The gap increases as the amount of data increases

關鍵字(中)

★ 知識圖譜建置
★ 分散式運算
★ 知識圖譜
★ RDB-to-RDF

關鍵字(英)

★ knowledge graphs building
★ distributed computing
★ knowledge graphs
★ RDB-to-RDF
★ spark

論文目次

摘要 i
Abstract ii
目錄 iii
圖目錄 v
表目錄 vii
第一章、緒論 1
1.1 研究背景 1
1.2 研究目標 3
1.3 論文架構 4
第二章、技術回顧 4
2.1 知識圖譜技術回顧 4
2.2 知識儲存技術回顧 6
2.3 知識取出技術回顧 10
2.4 Hadoop分散式系統架構技術回顧 13
第三章、系統架構 18
3.1 MIAT系統設計方法論 18
3.1.1 IDF0階層模組化設計 19
3.1.2 GRAFCET離散事件建模 21
3.2 知識圖譜建置系統模組 23
3.3 知識圖譜建置系統GRAFCET 26
3.3.1 資源初始化系統GRAFCET 27
3.3.2 資料擷取系統GRAFCET 28
3.3.3 知識圖譜格式轉換系統GRAFCET 30
第四章、實驗 32
4.1 實驗硬體介紹 32
4.2 實驗軟體與框架介紹 33
4.2.1 實驗系統參數介紹 33
4.3 實驗數據介紹 34
4.4 實驗結果與比較 38
第五章、結論與未來展望 43
5.1 結論 43
5.2 未來展望 44
參考文獻 45

參考文獻

[1] S. Ji, S. Pan, E. Cambria, P. Marttinen, and S. Y. Philip, "A survey on knowledge graphs: Representation, acquisition, and applications," IEEE Transactions on Neural Networks and Learning Systems, 2021.
[2] Y. Ma, P. A. Crook, R. Sarikaya, and E. Fosler-Lussier, "Knowledge graph inference for spoken dialog systems," in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5346-5350.
[3] O. Deshpande, D. S. Lamba, M. Tourn, S. Das, S. Subramaniam, A. Rajaraman, V. Harinarayan, and A. Doan, "Building, maintaining, and using knowledge bases: a report from the trenches," in Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, 2013, pp. 1209-1220.
[4] Q. Guo, F. Zhuang, C. Qin, H. Zhu, X. Xie, H. Xiong, and Q. He, "A survey on knowledge graph-based recommender systems," IEEE Transactions on Knowledge and Data Engineering, 2020.
[5] X. Wang, X. He, Y. Cao, M. Liu, and T.-S. Chua, "Kgat: Knowledge graph attention network for recommendation," in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 950-958.
[6] C. Asamoah, L. Tao, K. Gai, and N. Jiang, "Powering filtration process of cyber security ecosystem using knowledge graph," in 2016 IEEE 3rd International Conference on Cyber Security and Cloud Computing (CSCloud), 2016, pp. 240-246.
[7] S. Decker, S. Melnik, F. Van Harmelen, D. Fensel, M. Klein, J. Broekstra, M. Erdmann, and I. Horrocks, "The semantic web: The roles of XML and RDF," IEEE Internet computing, vol. 4, no. 5, 2000, pp. 63-73.
[8] H. Arnaout and S. Elbassuoni, "Effective searching of RDF knowledge graphs," Journal of Web Semantics, vol. 48, 2018, pp. 66-84.
[9] B. He, M. Patel, Z. Zhang, and K. C.-C. Chang, "Accessing the deep web," Communications of the ACM, vol. 50, no. 5, 2007, pp. 94-101.
[10] P. Shvaiko and J. Euzenat, "Ontology matching: state of the art and future challenges," IEEE Transactions on knowledge and data engineering, vol. 25, no. 1, 2011, pp. 158-176.
[11] P. Vassiliadis, A. Simitsis, and S. Skiadopoulos, "Conceptual modeling for ETL processes," in Proceedings of the 5th ACM international workshop on Data Warehousing and OLAP, 2002, pp. 14-21.
[12] S. Jabbar, K. R. Malik, M. Ahmad, O. Aldabbas, M. Asif, S. Khalid, K. Han, and S. H. Ahmed, "A methodology of real-time data fusion for localized big data analytics," IEEE Access, vol. 6, 2018, pp. 24510-24520.
[13] F. Michel, J. Montagnat, and C. F. Zucker, "A survey of RDB to RDF translation approaches and tools," I3S, 2014.
[14] O. Erling, "Virtuoso, a Hybrid RDBMS/Graph Column Store," IEEE Data Eng. Bull., vol. 35, no. 1, 2012, pp. 3-8.
[15] H. Yu and D. Wang, "Research and implementation of massive health care data management and analysis based on hadoop," in 2012 Fourth International Conference on Computational and Information Sciences, 2012, pp. 514-517.
[16] M. Banane and A. Belangour, "A new system for massive RDF data management using Big Data query languages Pig, Hive, and Spark," International Journal of Computing and Digital Systems, vol. 9, no. 2, 2020, pp. 259-270.
[17] T. Berners-Lee, J. Hendler, and O. Lassila, "The semantic web," Scientific american, vol. 284, no. 5, 2001, pp. 34-43.
[18] C. Bizer, T. Heath, K. Idehen, and T. Berners-Lee, "Linked data on the web (LDOW2008)," in Proceedings of the 17th international conference on World Wide Web, 2008, pp. 1265-1266.
[19] T. Heath and C. Bizer, "Linked data: Evolving the web into a global data space," Synthesis lectures on the semantic web: theory and technology, vol. 1, no. 1, 2011, pp. 1-136.
[20] W. Z. Khan, E. Ahmed, S. Hakak, I. Yaqoob, and A. Ahmed, "Edge computing: A survey," Future Generation Computer Systems, vol. 97, 2019, pp. 219-235.
[21] M. Husain, J. McGlothlin, M. M. Masud, L. Khan, and B. M. Thuraisingham, "Heuristics-based query processing for large RDF graphs using cloud computing," IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 9, 2011, pp. 1312-1327.
[22] J. Sun and Q. Jin, "Scalable rdf store based on hbase and mapreduce," in 2010 3rd international conference on advanced computer theory and engineering (ICACTE), 2010, vol. 1, pp. V1-633-V1-636.
[23] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber, "Bigtable: A distributed storage system for structured data," ACM Transactions on Computer Systems (TOCS), vol. 26, no. 2, 2008, pp. 1-26.
[24] D. Hou, Z. Zhao, and S. Hu, "Multi-label learning with visual-semantic embedded knowledge graph for diagnosis of radiology imaging," IEEE Access, vol. 9, 2021, pp. 15720-15730.
[25] H. Yan, J. Yang, and J. Wan, "KnowIME: a system to construct a knowledge graph for intelligent manufacturing equipment," IEEE Access, vol. 8, 2020, pp. 41805-41813.
[26] A. Katal, M. Wazid, and R. H. Goudar, "Big data: issues, challenges, tools and good practices," in 2013 Sixth international conference on contemporary computing (IC3), 2013, pp. 404-409.

指導教授

陳慶瀚

審核日期

2022-7-4

推文