巨量物聯網資料之多重屬性索引架構;An adaptively multi-attribute index framework for big IoT data

NCUIR > Engineering College > Graduate Institute of Civil Engineering > Electronic Thesis & Dissertation > Item 987654321/80154

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/80154

Title:	巨量物聯網資料之多重屬性索引架構;An adaptively multi-attribute index framework for big IoT data
Authors:	張有睿;Chang, Yu-Jui
Contributors:	土木工程學系
Keywords:	索引;資料管理;多重屬性;適應性;篩選率;index;data management;multi-attribute;selectivity;adaptivity
Date:	2019-08-16
Issue Date:	2019-09-03 12:14:55 (UTC+8)
Publisher:	國立中央大學
Abstract:	近年來，由於物聯網具有良好的自動監控特性，使其逐漸被廣泛應用在各個領域，然而隨著越來越多的物聯網裝置，巨量物聯網資料面臨著如何提升擴充性的議題。為此許多研究提及key-value儲存模式是一種相較於傳統關聯式資料庫更適合管理巨量資料的選擇。然而，除此之外物聯網資料還具有多重屬性的特徵，例如: 時間、空間、主題等屬性，因此如何建構出一套有效率的多重屬性索引架構也成為一重要課題。在此研究中，我們採納了四種常見的屬性包括時間、空間、關鍵字、數值，由於每種屬性適合的索引方式不同，在整合其所有不同的索引架構時，我們發現其索引整合的順序會對查詢效能有顯著的影響。然而許多現存的研究僅採取一種固定的索引整合順序設計其結合式索引(combined index)。因此本研究提出一套適應性選擇最有效率結合式索引之方法，藉由估計個別的索引效能以及篩選率來達成。主要的概念是利用將篩選率高的索引設計在較優先的位置處理查詢，用以最少化查詢中的中間結果，進而提升查詢效能。本篇研究提出一套多重索引框架，考慮所有可能的結合式索引順序，並且根據不同的查詢適應性的選出其中最佳之結合式索引。根據結果，在一百萬筆資料量下，本研究提出的系統相較於使用單一順序的索引架構，有百分之99到百分之94的機會節省25到51倍的查詢時間，並且相較於傳統的關聯式資料庫PostGIS，反應時間也快出兩倍。;In recent years, the concept of the Internet of Things (IoT) has been attracting attention from various fields as IoT devices can continuously monitor various environmental properties. While the number of IoT devices increases rapidly, managing large volume of IoT data faces a serious scalability issue. To address this issue, many studies have shown that the performance of key-value storages is better than traditional relational databases. However, IoT data have multi-dimensional attributes including spatial, temporal and thematic attributes. How to construct an efficient multi-attribute combined index is an important topic. In this research, we consider four main types of attributes and their corresponding queries, which are spatial, temporal, keyword, and value attributes. While each attribute has its own suitable index method, integrating the indexes into a combined index usually requires a certain sequence of indexes, which significantly decides the query performance. As many literatures directly present their designed combined index, this research proposes an adaptive method to decide the most efficient combined index by estimating the selectivity and query performance of individual query criterion. The main idea is that highly-selective queries should be performed first to reduce the number of intermediate results, which can improve the query performance of following queries. Hence, this research proposes an index framework considering every possible sequence and automatically identifying the most efficient combined index for each query. According to the result, the proposed system has 94-99% chance to save 25 to 51 times response time comparing to using a single combined index, and is twice faster than PostGIS on average when querying a one-million-record real-world dataset.
Appears in Collections:	[Graduate Institute of Civil Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	256	View/Open

社群 sharing

Loading...