基於強化學習之NB-IoT隨機存取與資源配置方法之研究

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：64

、訪客IP：18.222.166.162

姓名

游基正(Ji-Zheng You) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

基於強化學習之NB-IoT隨機存取與資源配置方法之研究
(Study of Reinforcement Learning for NB-IoT Random Access and Resource Allocation)

相關論文

★ 應用MSPP至DWDM都會光纖網路的設計	★ 光網路與WiMAX整合架構研究及其簡化雛型實驗
★ 以Linux系統為基礎之NAT效能優化研究及其實作	★ 光波長劃分多工網路之路徑保護機制研究
★ 標籤交換網路下具有服務品質路由安排之研究	★ 以訊務相關性為基礎的整合性服務可調整QoS排程器之研究
★ 以群體播送支援IPv6環境下移動式網路連結更新之研究	★ 無線區域網路資源動態分配之效能研究
★ 在微觀移動環境下有效資源保留之路徑管理研究	★ 無線網路交握程序之預先認證方法分析與比較
★ 無線區域網路虛擬允入控制之研究	★ IPv6環境下移動網路之連結更新程序及其效能之研究
★ 具有限數量波長轉換節點的分波多工網路之群播波長分配與容量計算研究	★ 階層化行動式IPv6移動錨點選擇機制研究
★ 具高能量移動節點之叢集式感測網路效能研究	★ 預先註冊之快速換手階層化行動式IPv6研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

為因應物聯網需求，第三代合作夥伴計畫(3rd Generation Partnership Project, 3GPP)於Release 13中以Long Term Evolution (LTE)為基礎提出了窄頻物聯網(Narrowband Internet of Thing, NB-IoT)技術，其修改並簡化了LTE規格使其更符合物聯網裝置需求並可與現行系統並存，而在Release 14中更開放非錨載波(non-anchor carrier)支援隨機存取(Random Access)功能，如此可舒緩原UE只能透過錨載波進行隨機存取而造成網路壅塞之問題。
而原先非錨載波上行用於Narrowband Physical Uplink Shared Channel (NPUSCH)資料傳輸，當調度其支援於Narrowband Physical Random Access Channel (NPRACH)隨機存取將壓縮可傳輸之資源量，而當隨機存取資源不足將造成網路壅塞使UE無法建立radio resource control (RRC)連線上傳資料，因此eNB端如何配置非錨載波支援隨機存取而不造成資源浪費，使資源有效利用為一討論議題。
本論文提出Prediction based Random Access Resource Allocation scheme (PRARA)透過強化學習預測所需開放之資源，並以碰撞子載波(subcarrier)個數預測及動態開放二次競爭之資源，在資源許可的情況下有效使用資源並提升隨機存取之效能。

摘要(英)

3rd Generation Partnership Project (3GPP) proposed NarrowBand Internet of Things (NB-IoT) based on Long Term Evolution (LTE) for IoT application in Release 13. It modifies and simplifies the LTE specification to let it be compatible with IoT devices and can coexist with existing LTE systems. In Release 14, the random access procedure can be supported in non-anchor carriers, which alleviate the problem that network congestion may occurs if UE can only random access via anchor carrier.
The non-anchor carriers in uplink are used for data transmission. However, if eNB schedules non-anchor carrier for Narrowband Physical Random Access Channel (NPRACH) then it also compresses the Narrowband Physical Uplink Shared Channel (NPUSCH) resource. When NPRACH resource is insufficient, which leads to network congestion, UEs might not be able to complete radio resource control (RRC) connection. So how to configure non-anchor carriers to support random access procedure without causing waste of resources is an importance issue.
In this thesis, we propose a Prediction based Random Access Resource Allocation scheme (PRARA), which firstly predicts the number of required resources based on reinforcement learning, and secondly, dynamically allocates the number of secondary contention resources according to the number of collided subcarriers. We aim to increase the performance and resource efficiency of random access in condition of limited resource.

關鍵字(中)

★ 窄頻物聯網
★ 非錨載波
★ 隨機存取
★ 強化學習

關鍵字(英)

★ NB-IoT
★ Non-Anchor Carrier
★ Random Access
★ Reinforcement Learning

論文目次

摘要 I
ABSTRACT II
致謝 III
目錄 IV
圖目錄 VI
表目錄 IX
1. 第一章緒論 1
1.1. 研究背景 1
1.2. 研究動機與目的 1
1.3. 章節概要 2
2. 第二章相關研究背景 3
2.1. NB-IoT基本介紹 3
2.1.1. 部署模式 3
2.1.2. 傳輸及訊框架構 4
2.1.3. 通道配置 4
2.1.4. 上行通道與資源 5
2.1.5. 下行通道與資源 6
2.1.6. 錨載波(anchor carrier)與非錨載波(non-anchor carrier) 8
2.1.7. 隨機存取程序(Random Access Procedure，RAP) 9
2.2. 機器學習基本介紹 11
2.2.1. Q-learning 12
2.2.2. Deep Q Network (DQN) 13
2.2.3. ε-greedy 14
2.2.4. Experience replay與fixed Q target 15
2.3. 相關文獻 15
2.3.1. 機器學習應用 15
2.3.2. NB-IoT及隨機存取相關議題 16
3. 第三章研究方法 22
3.1. 系統架構 22
3.2. 二次競爭資源 (Part B resource) 22
3.3. 系統流程 23
3.3.1. 系統參數 23
3.3.2. DQN架構流程 25
3.3.3. 模型訓練 27
3.3.4. eNB系統流程 29
3.3.5. UE端流程 30
4. 第四章模擬結果與討論 32
4.1. 模擬環境 32
4.2. Reward function 33
4.3. 模擬結果分析 39
4.3.1. 不同S權重之效能影響分析 40
4.3.2. 不同WB之效能影響分析 55
4.3.3. 動態關閉Part B資源之影響 62
4.3.4. 模擬現實情境下之效能分析 65
4.4. 模擬討論 68
5. 第五章結論 69
6. 參考文獻 72

參考文獻

[1] Y.-P. E. Wang, X. Lin, A. Adhikary, A. Grovlen, Y. Sui, Y. Blankenship, J. Bergman and H. S. Razaghi, "A Primer on 3GPP Narrowband Internet of Things," IEEE Communications Magazine, vol. 55, pp. 117-123, 2017.
[2] [Online]. Available: https://www.2cm.com.tw/2cm/zh-tw/magazine/-Technology/64BA704E800148C781D35F664663A5B4. [Accessed 23 5 2019].
[3] B. V. N. M. A. G. Rapeepat Ratasuk, "NB-IoT System for M2M Communication," in Wireless Communications and Networking Conference (WCNC), pp. 428–432.,Aug. 2016.
[4] L. Feltrin, G. Tsoukaneri, M. Condoluci, C. Buratti, T. Mahmoodi, M. Dohler and R. Verdone, "Narrowband IoT: A Survey on Downlink and Uplink Perspectives," IEEE Wireless Communications ( Volume: 26 , Issue: 1 ), vol. 26, pp. 78-86, February 2019.
[5] [Online]. Available: http://www.sharetechnote.com/html/Handbook_LTE_NB_FrameStructure_DL.html. [Accessed 31 5 2019].
[6] [Online]. Available: http://www.sharetechnote.com/html/Handbook_LTE_NB_MultiCarrier.html. [Accessed 25 5 2019].
[7] "Narrowband Internet of Things Whitepaper," [Online]. Available: https://cdn.rohde-schwarz.com/pws/dl_downloads/dl_application/application_notes/1ma266/1MA266_0e_NB_IoT.pdf. [Accessed 3 6 2019].
[8] X. Lin, A. Adhikary and Y.-P. E. Wang, "Random Access Preamble Design and Detection for 3GPP Narrowband IoT Systems," IEEE Wireless Communications Letters ( Volume: 5 , Issue: 6 ), vol. 5, pp. 640-643, 12 2016.
[9] D. Pandey and P. Pandey, "Approximate Q-Learning: An Introduction," in 2010 Second International Conference on Machine Learning and Computing, Bangalore, India, 2010.
[10] J. Li, Z. Zhao and R. Li, "Machine learning-based IDS for software-defined 5G network," IET Networks, vol. 7, no. 2, pp. 53-60, 1 3 2018.
[11] K. Lee and J. W. Jang, "An Efficient Contention Resolution Scheme for Massive IoT Devices in Random Access to LTE-A Networks," IEEE Access ( Volume: 6 ), pp. 67118 - 67130, 11 2018.
[12] C. Di, B. Zhang, Q. Liang, S. Li and Y. Guo, "Learning Automata based Access Class Barring Scheme for Massive Random Access in Machine-to-Machine Communications," IEEE Internet of Things Journal ( Early Access ), 2019.
[13] Z. Chen and D. B. Smith, "Heterogeneous Machine-Type Communications in Cellular Networks: Random Access Optimization by Deep Reinforcement Learning," in IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 2018.
[14] N. Jiang, Y. Deng, A. Nallanathan and J. A. Chambers, "Reinforcement Learning for Real-Time Optimization in NB-IoT Networks," IEEE Journal on Selected Areas in Communications ( Volume: 37 , Issue: 6 ), pp. 1424 - 1440, June 2019.
[15] Y. Zhao, K. Liu, H. Yan and L. Huang, "A classification back-off method for capacity optimization in NB-IOT random access," in 2017 11th IEEE International Conference on Anti-counterfeiting, Security, and Identification (ASID), Xiamen, China, 2017.
[16] R. Ratasuk, N. Mangalvedhe, Z. Xiong, M. Robert and D. Bhatoolaul, "Enhancements of narrowband IoT in 3GPP Rel-14 and Rel-15," in 2017 IEEE Conference on Standards for Communications and Networking (CSCN), Helsinki, Finland, 2017.
[17] 3GPP TR 21.914, "Release 14 Description ;Summary of Rel-14 Work Items," V14.0.0, May 2018.
[18] W. S. Jeon, S. B. Seo and D. G. Jeong, "Effective Frequency Hopping Pattern for ToA Estimation in NB-IoT Random Access," IEEE Transactions on Vehicular Technology ( Volume: 67 , Issue: 10 , Oct. 2018 ), pp. 10150 - 10154, 10 2018.
[19] G.-Y. Xue, "Study of Random Access Scheme and Its Resource Adaptive Allocation in NB-IoT Network," National Central University, thesis, 2018.
[20] 3GPP TR 37.868, "Study on RAN Improvements for Machine-type Communications".
[21] "Deep Deterministic Policy Gradient," [Online]. Available: https://spinningup.openai.com/en/latest/algorithms/ddpg.html#id1. [Accessed 19 7 2019].
[22] A. P. B. M. G. H. P. L. S. K. Volodymyr Mnih, "Asynchronous Methods for Deep Reinforcement Learning," 2016.

指導教授

陳彥文(Yen-Wen Chen)

審核日期

2019-7-26

推文