低軌道衛星網路環境下基於強化學習之 負載平衡路由方法

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：141

、訪客IP：18.220.188.4

姓名

蔡孟澤(Meng-Tse Tsai) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

低軌道衛星網路環境下基於強化學習之負載平衡路由方法
(Load-Balancing Routing Based on Reinforcement Learning in Low-Earth-Orbit Satellite Network)

相關論文

★ 非結構同儕網路上以特徵相似度為基準之搜尋方法	★ 以階層式叢集聲譽為基礎之行動同儕網路拓撲架構
★ 線上RSS新聞資料流中主題性事件監測機制之設計與實作	★ 耐延遲網路下具密度感知的路由方法
★ 整合P2P與UPnP內容分享服務之家用多媒體閘道器：設計與實作	★ 家庭網路下簡易無縫式串流影音播放服務之設計與實作
★ 耐延遲網路下訊息傳遞時間分析與高效能路由演算法設計	★ BitTorrent P2P 檔案系統下載端網路資源之可調式配置方法與效能實測
★ 耐延遲網路中利用訊息編碼重組條件之資料傳播機制	★ 耐延遲網路中基於人類移動模式之路由機制
★ 車載網路中以資料匯集技術改善傳輸效能之封包傳送機制	★ 適用於交叉路口環境之車輛叢集方法
★ 車載網路下結合路側單元輔助之訊息廣播機制	★ 耐延遲網路下以靜態中繼節點（暫存盒）最佳化訊息傳遞效能之研究
★ 耐延遲網路下以動態叢集感知建構之訊息傳遞機制	★ 跨裝置影音匯流平台之設計與實作

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

隨著5G技術的快速發展，網路流量不斷增加且對於低延遲的需求日益強烈。衛星網路因此成為下一代通訊網路的關鍵組成部分。然而，由於地球上陸地和海洋分佈的不均，導致全球人口分布存在差異。這使得在衛星網路中使用傳統的路由算法可能導致衛星節點在穿越人口密集區域時出現傳輸壅塞的情況。為了確保衛星網路的性能，並應對不斷增長的網路流量，本研究提出了一種基於強化學習的路由方法，旨在實現流量的有效分配。我們的方法採用集中式管理，融合衛星鏈路的動態資訊，包括剩餘頻寬、傳輸延遲、可用時間和接收節點的平均等待時間作為學習參數。我們採用雙重深度強化學習網路（DDQN）模型來最佳化路由決策。通過與其他負載平衡路由方法進行性能比較，我們在不同網路場景和傳輸需求下進行模擬，結果顯示我們提出的路由方法在較小傳輸延遲的情況下，在剩餘頻寬和封包遺失率等方面表現出色。

摘要(英)

With the rapid development of 5G technology, there is a continuous increase in network traffic and a growing demand for low latency. Consequently, satellite networks have become a crucial component of the next-generation communication infrastructure. However, due to the uneven distribution of land and oceans on Earth, there exist disparities in the global population distribution. This disparity can lead to transmission congestion for satellite nodes when passing through densely populated areas, if traditional routing algorithms are employed. In order to ensure the performance of satellite networks and address the ever-growing network traffic, this study introduces a routing approach based on reinforcement learning aimed at achieving effective traffic allocation. Our method involves centralized management and integrates dynamic information from satellite links, including remaining bandwidth, transmission delay, available time, and average waiting time of receiving nodes, as learning parameters. We utilize a Double Deep Q-Network (DDQN) model to optimize routing decisions. Through performance comparisons with other load-balancing routing methods, we conduct simulations under various network scenarios and transmission requirements. The results demonstrate that our proposed routing method exhibits superior performance in terms of reduced transmission delay, improved remaining bandwidth utilization, and reduced packet loss rate.

關鍵字(中)

★ 衛星網路
★ 路由算法
★ 負載平衡

關鍵字(英)

★ satellite network
★ routing algorithm
★ load balancing

論文目次

目錄
摘要 i
Abstract ii
圖目錄 iv
表目錄 v
第1章簡介 1
1.1 前言 1
1.2 研究動機 2
第2章背景與相關文獻探討 3
2.1衛星星座 3
2.1.1衛星軌道 3
2.1.2 LEO衛星星座結構 5
2.2 衛星路由 7
2.2.1 衛星間鏈路 7
2.2.2 衛星路由算法 8
第3章系統架構 9
3.1 網路架構 9
3.1.1衛星網路架構 9
3.1.2 衛星拓譜 11
3.1.3 傳輸模型 12
3.2 問題定義 13
3.3 強化學習 15
3.3.1 Double Deep Q-Learning Network 16
3.3.2 基於DDQN之路由算法 18
第4章實驗與結果分析 20
4.1實驗環境與設備 20
4.2 實驗設計 21
4.2.1衛星環境設計 21
4.2.2模型參數設計 22
4.3 實驗結果與分析 24
4.3.1 剩餘頻寬 24
4.3.2 封包遺失率 26
4.3.3平均延遲 28
第5章結論 30
參考文獻 31

參考文獻

[1] Junfeng Wang , Lei Li , Mingtian Zhou, “Topological dynamics characterization for LEO satellite networks”, ScienceDirect Computer Network, Pages 43-53 , 2007

[2] L. Zhang et al., "A Routing Algorithm Based on Link State Information for LEO Satellite Networks," 2020 IEEE Globecom Workshops (GC Wkshps, Taipei, Taiwan, 2020, pp. 1-6, doi: 10.1109/GCWkshps50303.2020.9367496.

[3] H. Liming, K. Shaoli, S. Shaohui, M. Deshan, H. Bo and Z. Meiting, "A load balancing routing method based on real time traffic in LEO satellite constellation space networks," 2022 IEEE 95th Vehicular Technology Conference: (VTC2022-Spring), Helsinki, Finland, 2022, pp. 1-5, doi: 10.1109/VTC2022-Spring54318.2022.9860540.

[4] C. Dong, X. Xu, A. Liu and X. Liang, "Load balancing routing algorithm based on extended link states in LEO constellation network," in China Communications, vol. 19, no. 2, pp. 247-260, Feb. 2022, doi: 10.23919/JCC.2022.02.020.

[5] J. Liu, R. Luo, T. Huang and C. Meng, "A Load Balancing Routing Strategy for LEO Satellite Network," in IEEE Access, vol. 8, pp. 155136-155144, 2020, doi: 10.1109/ACCESS.2020.3017615.

[6] P. Liu, H. Chen, S. Wei, L. Li, and Z. Zhu, ‘‘Hybrid-traffic-detour based load balancing for onboard routing in LEO satellite networks,’’ China Commun., vol. 15, no. 6, pp. 28–41, Jun. 2018

[7] Yang, Li, and Jing Sun. "Multi-service routing algorithm based on GEO/LEO satellite networks." 2016 International Conference on Network and Information Systems for Computers (ICNISC). IEEE, 2016.

[8] J. Huang, W. Liu, Y. Su, and F. Wang, ‘‘Load balancing strategy and its lookup-table enhancement in deterministic space delay/disruption tolerant networks,’’ Adv. Space Res., vol. 61, no. 3, pp. 811–822, Feb. 2018.

[9] Y. Huang, W. Cao, X. Liu, X. Jiang, J. Yang and F. Yang, "An Adaptive Multipath Routing for LEO Satellite Network," 2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Chongqing, China, 2021, pp. 1536-1541, doi: 10.1109/IMCEC51613.2021.9482379.

[10] Yin, Yabo, et al. "Reinforcement learning-based routing algorithm in satellite-terrestrial integrated networks." Wireless Communications and Mobile Computing 2021 (2021): 1-15.

[11] K. -C. Tsai, T. -J. Yao, P. -H. Huang, C. -S. Huang, Z. Han and L. -C. Wang, "Deep Reinforcement Learning-Based Routing for Space-Terrestrial Networks," 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall), London, United Kingdom, 2022, pp. 1-5, doi: 10.1109/VTC2022-Fall57202.2022.10013028.

[12] Y. Zhao, H. Yao, Z. Qin and T. Mai, "Collaborate Q-learning Aided Load Balance in Satellites Communications," 2021 International Wireless Communications and Mobile Computing (IWCMC), Harbin City, China, 2021, pp. 968-973, doi: 10.1109/IWCMC51323.2021.9498837

[13] T. Li, H. Zhou, H. Luo, W. Quan and S. Yu, "Modeling software defined satellite networks using queueing theory," 2017 IEEE International Conference on Communications (ICC), Paris, France, 2017, pp. 1-6, doi: 10.1109/ICC.2017.7997290.

[14] M. Jarschel, S. Oechsner, D. Schlosser, R. Pries, S. Goll and P. Tran-Gia, "Modeling and performance evaluation of an OpenFlow architecture," 2011 23rd International Teletraffic Congress (ITC), San Francisco, CA, USA, 2011, pp. 1-7.

[15] B. Yang, C. Xi, G. Li, P. Liu and R. Zhu, "Deep Reinforcement Learning-Based Satellite-Ground Links Scheduling for Mega Satellite Constellations," 2022 Asia Communications and Photonics Conference (ACP), Shenzhen, China, 2022, pp. 1284-1288, doi: 10.1109/ACP55869.2022.10089120.

[16] Iridium parameter, https://www.iridium.com/network/

指導教授

胡誌麟(Chih-Lin Hu)

審核日期

2023-8-17

推文