巨量物聯網資料之多重屬性索引架構

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：53

、訪客IP：18.117.106.23

姓名

張有睿(Yu-Jui Chang) 查詢紙本館藏

畢業系所

土木工程學系

論文名稱

巨量物聯網資料之多重屬性索引架構
(An adaptively multi-attribute index framework for big IoT data)

相關論文

★ 物聯網制動功能之互操作性解決方案	★ 地理網路爬蟲：具擴充及擴展性之地理網路資源爬行架構
★ TDR監測資訊平台之改善與感測器觀測服務之建立	★ 利用高解析衛星立體像對產製近岸水底地形
★ 整合oneM2M 及OGC SensorThings API 標準建立開放式物聯網架構	★ 高效率異質性時序資料表示法辨別系統
★ A TOA-reflectance-based Spatial-temporal Image Fusion Method for Aerosol Optical Depth Retrieval	★ An Automatic Embedded Device Registration Procedure for the OGC SensorThings API
★ 基於本體論與使用者興趣之個人化地理網路搜尋引擎	★ 利用本體論整合城市模型及物聯網開放式標準探討智慧城市之應用
★ 運用無人機及影像套合法進行混凝土橋梁裂縫檢測	★ GeoRank: A Geospatial Web Ranking Algorithm for a GeoWeb Search Engine
★ 應用高時空解析度遙測影像融合於海水覆蓋率之監測	★ LoRaWAN Positioning based on Time Difference of Arrival and Differential Correction
★ 類神經網路逆向工程理解遙測資訊：以Landsat 8植被分類為例	★ 基於語意網技術與WordNet促進地理網路資源之探索

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

近年來，由於物聯網具有良好的自動監控特性，使其逐漸被廣泛應用在各個領域，然而隨著越來越多的物聯網裝置，巨量物聯網資料面臨著如何提升擴充性的議題。為此許多研究提及key-value儲存模式是一種相較於傳統關聯式資料庫更適合管理巨量資料的選擇。然而，除此之外物聯網資料還具有多重屬性的特徵，例如: 時間、空間、主題等屬性，因此如何建構出一套有效率的多重屬性索引架構也成為一重要課題。在此研究中，我們採納了四種常見的屬性包括時間、空間、關鍵字、數值，由於每種屬性適合的索引方式不同，在整合其所有不同的索引架構時，我們發現其索引整合的順序會對查詢效能有顯著的影響。然而許多現存的研究僅採取一種固定的索引整合順序設計其結合式索引(combined index)。因此本研究提出一套適應性選擇最有效率結合式索引之方法，藉由估計個別的索引效能以及篩選率來達成。主要的概念是利用將篩選率高的索引設計在較優先的位置處理查詢，用以最少化查詢中的中間結果，進而提升查詢效能。本篇研究提出一套多重索引框架，考慮所有可能的結合式索引順序，並且根據不同的查詢適應性的選出其中最佳之結合式索引。根據結果，在一百萬筆資料量下，本研究提出的系統相較於使用單一順序的索引架構，有百分之99到百分之94的機會節省25到51倍的查詢時間，並且相較於傳統的關聯式資料庫PostGIS，反應時間也快出兩倍。

摘要(英)

In recent years, the concept of the Internet of Things (IoT) has been attracting attention from various fields as IoT devices can continuously monitor various environmental properties. While the number of IoT devices increases rapidly, managing large volume of IoT data faces a serious scalability issue. To address this issue, many studies have shown that the performance of key-value storages is better than traditional relational databases. However, IoT data have multi-dimensional attributes including spatial, temporal and thematic attributes. How to construct an efficient multi-attribute combined index is an important topic. In this research, we consider four main types of attributes and their corresponding queries, which are spatial, temporal, keyword, and value attributes. While each attribute has its own suitable index method, integrating the indexes into a combined index usually requires a certain sequence of indexes, which significantly decides the query performance. As many literatures directly present their designed combined index, this research proposes an adaptive method to decide the most efficient combined index by estimating the selectivity and query performance of individual query criterion. The main idea is that highly-selective queries should be performed first to reduce the number of intermediate results, which can improve the query performance of following queries. Hence, this research proposes an index framework considering every possible sequence and automatically identifying the most efficient combined index for each query. According to the result, the proposed system has 94-99% chance to save 25 to 51 times response time comparing to using a single combined index, and is twice faster than PostGIS on average when querying a one-million-record real-world dataset.

關鍵字(中)

★ 索引
★ 資料管理
★ 多重屬性
★ 適應性
★ 篩選率

關鍵字(英)

★ index
★ data management
★ multi-attribute
★ selectivity
★ adaptivity

論文目次

摘要 vi
Abstract vii
Table of Contents viii
List of Figures and Illustrations x
List of Tables x
1. Introduction 1
1.1. Internet of things (IoT) 1
1.1.2. Multiple attributes in IoT data 2
1.2. Problems and objectives 7
2. Literature review 10
3. Methodology 15
3.1. System architecture 15
3.2. Combined index 19
3.3 Index structure 19
3.3.1 Selector 20
3.3.2 Data filtering 24
3.4 Load balancing strategy 25
4. Evaluation 27
4.1 Data 27
4.3 The trade-off between accuracy and performance 32
4.4 Comparison with PostGIS 34
5. Conclusions and future work 37
Reference 38

參考文獻

[1] ITU (Intermational Telecommunication Union), “Overview of the Internet of things,” Recomm. ITU-T Y.2060, 2012.
[2] M.Marjani et al., “Big IoT Data Analytics: Architecture, Opportunities, and Open Research Challenges,” IEEE Access, vol. 5, pp. 5247–5261, 2017.
[3] Z.Khan, A.Anjum, andS. L.Kiani, “Cloud based big data analytics for smart future cities,” Proc. - 2013 IEEE/ACM 6th Int. Conf. Util. Cloud Comput. UCC 2013, pp. 381–386, 2013.
[4] L.Catarinucci et al., “An IoT-Aware Architecture for Smart Healthcare Systems,” IEEE Internet Things J., vol. 2, no. 6, pp. 515–526, 2015.
[5] C.-Y. H.Steve Liang, “Geospatial Cyberinfrastructure for Addressing the Big Data Challenges on the Worldwide Sensor Web Steve,” 2010.
[6] B. F.Cooper, N.Sample, andM.Shadmon, “A Fast index for semistructured data,” p. 890, 2003.
[7] T. K.Sellis, “Multiple-query optimization,” ACM Trans. Database Syst., vol. 13, no. 1, pp. 23–52, 1988.
[8] Z.Ding, J.Xu, andQ.Yang, “SeaCloudDM: A database cluster framework for managing and querying massive heterogeneous sensor sampling data,” J. Supercomput., vol. 66, no. 3, pp. 1260–1284, 2013.
[9] OGC, “Open Geospatial Consortium - Observations and Measurements - XML Implement - Version 2.0,” OGC® Implement., vol. OGC 10-025, 2011.
[10] A.Ahmad, A.Paul, M.Rathore, andH.Chang, “An Efficient Multidimensional Big Data Fusion Approach in Machine-to-Machine Communication,” ACM Trans. Embed. Comput. Syst., vol. 15, no. 2, pp. 1–25, 2016.
[11] J.Zhou, L.Hu, F.Wang, H.Lu, andK.Zhao, “An efficient multidimensional fusion algorithm for iot data based on partitioning,” Tsinghua Sci. Technol., vol. 18, no. 4, pp. 369–378, 2013.
[12] V. Y.Lum, “Multi-attribute retrieval with combined indexes,” Commun. ACM, vol. 13, no. 11, pp. 660–665, 1970.
[13] D.Lee andS. H. L.Liang, “Geopot: A cloud-based geolocation data service for mobile applications,” Int. J. Geogr. Inf. Sci., vol. 25, no. 8, pp. 1283–1301, 2011.
[14] Y.Ma et al., “An efficient index for massive IOT data in cloud environment,” 2012, p. 2129.
[15] B.Harangsri, J.Shepherd, andA.Ngu, “Selectivity Estimation fot Joins Using Systematic Sampling,” pp. 384–389, 1997.
[16] Y. S.Kang, I. H.Park, J.Rhee, andY. H.Lee, “MongoDB-Based Repository Design for IoT-Generated RFID/Sensor Big Data,” IEEE Sens. J., vol. 16, no. 2, pp. 485–497, 2016.
[17] T.Li, Y.Liu, Y.Tian, S.Shen, andW.Mao, “A storage solution for massive IoT data based on NoSQL,” Proc. - 2012 IEEE Int. Conf. Green Comput. Commun. GreenCom 2012, Conf. Internet Things, iThings 2012 Conf. Cyber, Phys. Soc. Comput. CPSCom 2012, pp. 50–57, 2012.
[18] D.Comer, “Ubiquitous B-Tree,” ACM Comput. Surv., vol. 11, no. 2, pp. 121–137, 1979.
[19] A.Guttman, “R-trees a dynamic index structure for spatial searching,” ACM SIGMOD Rec., vol. 14, no. 2, p. 47, 2005.

指導教授

黃智遠(Chih-Yuan Huang)

審核日期

2019-8-16

推文