B5G O-RAN 環境下基於 MDP 動態資源分配的混合 式 QoS 資料傳輸方法

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：40

、訪客IP：18.226.248.77

姓名

仇瑞興(Rui-Xing Chou) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

B5G O-RAN 環境下基於 MDP 動態資源分配的混合式 QoS 資料傳輸方法
(Hybrid QoS Data Delivery Based on MDP Dynamic Resource Allocation in B5G O-RAN Environments)

相關論文

★ 非結構同儕網路上以特徵相似度為基準之搜尋方法	★ 以階層式叢集聲譽為基礎之行動同儕網路拓撲架構
★ 線上RSS新聞資料流中主題性事件監測機制之設計與實作	★ 耐延遲網路下具密度感知的路由方法
★ 整合P2P與UPnP內容分享服務之家用多媒體閘道器：設計與實作	★ 家庭網路下簡易無縫式串流影音播放服務之設計與實作
★ 耐延遲網路下訊息傳遞時間分析與高效能路由演算法設計	★ BitTorrent P2P 檔案系統下載端網路資源之可調式配置方法與效能實測
★ 耐延遲網路中利用訊息編碼重組條件之資料傳播機制	★ 耐延遲網路中基於人類移動模式之路由機制
★ 車載網路中以資料匯集技術改善傳輸效能之封包傳送機制	★ 適用於交叉路口環境之車輛叢集方法
★ 車載網路下結合路側單元輔助之訊息廣播機制	★ 耐延遲網路下以靜態中繼節點（暫存盒）最佳化訊息傳遞效能之研究
★ 耐延遲網路下以動態叢集感知建構之訊息傳遞機制	★ 跨裝置影音匯流平台之設計與實作

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2026-8-20以後開放)

摘要(中)

在本論文中，我們研究了在 5G O-RAN 架構下如何高效傳輸混合型資料流
（URLLC、eMBB、mMTC），特別是在考慮到 O-RAN 元件儲存資源有限的情況下，提出了一種基於 Markov decision process (MDP) 和 Duel DQN 的強化學習方法。
此方法結合了網路切片技術和資料預處理，旨在提升頻寬使用率和封包接收率，並降低資料流的丟失率。
研究結果表明，該方法在智慧工廠環境中的模擬結果顯示出顯著的性能提升，尤其是在提升整體頻寬使用率、降低傳輸延遲和丟失率方面，展示了其在實際應用中的有效性。具體而言，本研究解決了兩個關鍵性問題：（1）如何在保證資料流穩定進入系統後被有效執行，提升整體封包接收率；（2）如何找到最佳的空間分配資料流並預留容量
處理緊急資料，以提升整體頻寬使用率。
總結來說，本研究提出的方法在 5G O-RAN 架構下的混合型資料流傳輸問題中展示了良好的應用潛力，對於 5G O-RAN 的未來發展具有重要的指導意義。

摘要(英)

In this thesis, we investigate how to efficiently transmit hybrid data streams (URLLC,
eMBB, mMTC) under the 5G O-RAN architecture, especially considering the limited
storage resources of O-RAN components. We propose a reinforcement learning method
based on Markov Decision Process (MDP) and Duel DQN. This method combines network
slicing techniques and data preprocessing to improve bandwidth utilization and packet
reception rate while reducing data stream loss.
The research results show that this method demonstrates significant performance improvements in a smart factory environment, especially in enhancing overall bandwidth
utilization, reducing transmission delay, and loss rate, demonstrating its effectiveness in
practical applications. Specifically, this study addresses two key issues: (1) how to ensure stable execution of data streams after they enter the system, thereby improving the
overall packet reception rate; and (2) how to find the optimal spatial allocation for data
streams and reserve capacity to handle urgent data, thereby improving overall bandwidth
utilization.
In conclusion, the proposed method demonstrates good application potential in solving
the hybrid data stream transmission problem under the 5G O-RAN architecture, providing
important guidance for the future development of 5G O-RAN.

關鍵字(中)

★ 混合型封包
★ 網路切片
★ 頻寬使用率
★ 智慧工廠

關鍵字(英)

★ O-RAN
★ Markov decision process
★ Duel DQN

論文目次

摘要 i
Abstract ii
圖目錄 vi
表目錄 viii
1 前言 1
1.1 研究背景 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 研究動機與目的 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 論文架構 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 背景與相關文獻探討 5
2.1 5G O-RAN 及未來願景 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.1 5G 網路架構及特性 . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.2 New Radio (NR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.3 Software Defined Network (SDN) . . . . . . . . . . . . . . . . . . . 9
2.1.4 Network Functions Virtualization (NFV) . . . . . . . . . . . . . . . 9
2.2 O-RAN 架構模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 服務管理和編排（SMO） . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.2 Near-RT RIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.3 Non-RT RIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.4 O-RU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.5 O-DU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.6 O-CU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.7 O-CU-UP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.8 O-eNB/O-gNB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.9 O-Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 5G QoS 流量 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.1 High-capacity enhanced mobile broadband (eMBB) . . . . . . . . . 18
2.3.2 Ultra-reliable low-latency communications (URLLC) . . . . . . . . 18
2.3.3 Massive Machine Type Communication (mMTC) . . . . . . . . . . 19
2.4 O-RAN Slicing 的演變 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.1 4G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4.2 5G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4.3 6G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.5 訓練學習 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5.1 DQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5.2 DDQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.5.3 Duel DQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6 排程策略 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.6.1 Proportional Fairness(PF) . . . . . . . . . . . . . . . . . . . . . . . 33
2.6.2 Dominant Resource Fairness（DRF） . . . . . . . . . . . . . . . . 33
2.6.3 Waterfilling (WF) . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.6.4 Round-robin (RR) . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.7 問題與文獻探討 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.7.1 大量封包處理模式 . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.7.2 網路切片工作模式 . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.7.3 訓練模式 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.8 總結 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3 系統架構 44
3.1 系統模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2 封包模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3 通道模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.3.1 頻寬可使用量 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3.2 切片請求 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3.3 路徑選擇 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4 深度強化學習模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.4.1 DQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.4.2 DDQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4.3 Duel DQN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.5 馬可夫決策過程 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.5.1 基於馬可夫決策結合強化學習 . . . . . . . . . . . . . . . . . . . . . 60
4 實驗與結果分析 62
4.1 模擬環境設定 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2 實驗參數設計 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3 模型實驗參數設定 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.4 模型參數設定 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.5 實驗結果 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.5.1 學習率 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.5.2 衰減率 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.5.3 不同情境下對於 Duel DQN 的影響 . . . . . . . . . . . . . . . . . . 70
4.5.4 不同演算法之間頻寬使用率 . . . . . . . . . . . . . . . . . . . . . . 72
4.5.5 不同演算法之間封包丟失率 . . . . . . . . . . . . . . . . . . . . . . 74
5 結論與貢獻 76
6 未來研究方向及議題 77
6.1 前言背景 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.1.1 數位孿生 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.1.2 融合通訊與運算 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.1.3 人工智慧 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.1.4 未來延伸以及改進 . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
參考文獻 80

參考文獻

[1] E. Sarikaya and E. Onur, “Placement of 5g ran slices in multi-tier o-ran 5g networks
with flexible functional splits,” in 2021 17th International Conference on Network
and Service Management (CNSM). IEEE, 2021, pp. 274–282.
[2] S. Kukliński, L. Tomaszewski, and R. Kołakowski, “On o-ran, mec, son and network
slicing integration,” in 2020 IEEE Globecom Workshops (GC Wkshps. IEEE, 2020,
pp. 1–6.
[3] A. 3rd Generation Partnership Project (3GPP), 3GPP Architecture. https://
spectrum.ieee.org/3gpp-release-15-overview: 3rd Generation Partnership Project
(3GPP), 2018.
[4] J. S. Vardakas, K. Ramantas, E. Vinogradov, M. A. Rahman, A. Girycki, S. Pollin,
S. Pryor, P. Chanclou, and C. Verikoukis, “Machine learning-based cell-free support
in the o-ran architecture: an innovative converged optical-wireless solution toward
6g networks,” IEEE Wireless Communications, vol. 29, no. 5, pp. 20–26, 2022.
[5] T. F. Rahman, M. Zhang, and V. Marojevic, “O-ran perspective on industrial internet
of things: A swot analysis,” in 2023 IEEE International Conference on Industrial
Technology (ICIT). IEEE, 2023, pp. 1–6.
[6] R. Joda, S. Naseri, M. Hashemi, and C. Richards, “Ue centric du placement with
carrier aggregation in o-ran using deep q-network algorithm,” in 2023 IEEE 34th
Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC). IEEE, 2023, pp. 1–6.
[7] N. Wu, Z. Ke, and L. Feng, “Stock price forecast based on lstm and ddqn,” in 2022
14th International Conference on Advanced Computational Intelligence (ICACI).
IEEE, 2022, pp. 182–185.
[8] A. Sharma, D. Pantola, S. K. Gupta, and D. Kumari, “Performance evaluation of
dqn, ddqn and dueling dqn in heart disease prediction,” in 2023 Second International
Conference On Smart Technologies For Smart Nation (SmartTechCon). IEEE, 2023,
pp. 5–11.
[9] Y. Jin and M. Hayashi, “Efficiency comparison between proportional fairness and
dominant resource fairness with two different type resources,” in 2016 Annual Conference on Information Science and Systems (CISS). IEEE, 2016, pp. 643–648.
[10] A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and I. Stoica, “Dominant resource fairness: Fair allocation of multiple resource types,” in 8th USENIX
symposium on networked systems design and implementation (NSDI 11), 2011.
[11] T.-Y. Wu, Two Classes of Constrained Optimization Problems with Water-filling
Solutions. https://hdl.handle.net/11296/yznma8: 國立清華大學資訊工程學系,
2023.
[12] A. Mansy, M. Ammar, and E. Zegura, “Deficit round-robin based message ferry
routing,” in 2011 IEEE Global Telecommunications Conference-GLOBECOM 2011.
IEEE, 2011, pp. 1–5.
[13] A. Abd Elkader and K. Eldahshan, “Round robin based scheduling algorithms: A
comparative study,” 2017.
[14] C. Bektas, D. Overbeck, and C. Wietfeld, “Samus: Slice-aware machine learningbased ultra-reliable scheduling,” in ICC 2021-IEEE International Conference on
Communications. IEEE, 2021, pp. 1–6.
[15] H. Gupta, M. Sharma, B. R. Tamma et al., “Apt-ran: A flexible split-based 5g ran to
minimize energy consumption and handovers,” IEEE Transactions on Network and
Service Management, vol. 17, no. 1, pp. 473–487, 2019.
[16] Y. Sun, Y. Wang, H. Yu, B. Guo, and X. Zhang, “A learning-based bandwidth
resource allocation method in sliced 5g c-ran,” in 2019 IEEE Globecom Workshops
(GC Wkshps). IEEE, 2019, pp. 1–5.
[17] C. C. Zhang, K. K. Nguyen, and M. Cheriet, “Joint routing and packet scheduling
for urllc and embb traffic in 5g o-ran,” in ICC 2022-IEEE International Conference
on Communications. IEEE, 2022, pp. 1900–1905.
[18] H. Ko, J. Lee, and S. Pack, “Priority-based dynamic resource allocation scheme
in network slicing,” in 2021 International Conference on Information Networking
(ICOIN). IEEE, 2021, pp. 62–64.
[19] A. Anand, G. De Veciana, and S. Shakkottai, “Joint scheduling of urllc and embb
traffic in 5g wireless networks,” IEEE/ACM Transactions on Networking, vol. 28,
no. 2, pp. 477–490, 2020.
[20] F. Z. Yousaf, M. Gramaglia, V. Friderikos, B. Gajic, D. Von Hugo, B. Sayadi,
V. Sciancalepore, and M. R. Crippa, “Network slicing with flexible mobility and
qos/qoe support for 5g networks,” in 2017 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 2017, pp. 1195–1201.
[21] N. F. Cheng, T. Pamuklu, and M. Erol-Kantarci, “Reinforcement learning based
resource allocation for network slices in o-ran midhaul,” in 2023 IEEE 20th Consumer
Communications & Networking Conference (CCNC). IEEE, 2023, pp. 140–145.
[22] H. Yu, F. Musumeci, J. Zhang, M. Tornatore, and Y. Ji, “Isolation-aware 5g ran slice
mapping over wdm metro-aggregation networks,” Journal of Lightwave Technology,
vol. 38, no. 6, pp. 1125–1137, 2020.
[23] K. H. Mohammadani, R. A. Butt, W. Nawaz, S. Faizullah, and Z. A. Dayo, “Energyefficient sleep-aware slicing-based scheduler (sa-sbs) for multi-operators virtualized
passive optical networks,” IEEE Access, 2023.
[24] B. Lei, J. Zhou, M. Ma, and X. Niu, “Dqn based blockchain data storage in resourceconstrained iot system,” in 2023 IEEE Wireless Communications and Networking
Conference (WCNC). IEEE, 2023, pp. 1–6.
[25] G. O. Boateng, D. Ayepah-Mensah, D. M. Doe, A. Mohammed, G. Sun, and G. Liu,
“Blockchain-enabled resource trading and deep reinforcement learning-based autonomous ran slicing in 5g,” IEEE Transactions on Network and Service Management, vol. 19, no. 1, pp. 216–227, 2021.
[26] A. OPC Foundation, OPC Unified Architecture. http://www.opcfoundation.org/
errata: OPC Foundation, 2015.
[27] A. A. Abdellatif, A. Mohamed, A. Erbad, and M. Guizani, “Dynamic network slicing and resource allocation for 5g-and-beyond networks,” in 2022 IEEE Wireless
Communications and Networking Conference (WCNC). IEEE, 2022, pp. 262–267.
[28] M. Sharara, T. Pamuklu, S. Hoteit, V. Vèque, and M. Erol-Kantarci, “Policygradient-based reinforcement learning for computing resources allocation in o-ran,” in
2022 IEEE 11th International Conference on Cloud Networking (CloudNet). IEEE,
2022, pp. 229–236.
[29] Discrete event simulation for python. https://simpy.readthedocs.io/en/latest/: online, 2020.
[30] Build next-gen apps with openai’s powerful models. https://openai.com/api/: OpenAI, 2022.
[31] M. K. Motalleb, V. Shah-Mansouri, S. Parsaeefard, and O. L. A. López, “Resource
allocation in an open ran system using network slicing,” IEEE Transactions on Network and Service Management, vol. 20, no. 1, pp. 471–485, 2022.
[32] E. Nan, X. Chu, W. Guo, and J. Zhang, “User data traffic analysis for 3g cellular
networks,” in 2013 8th International Conference on Communications and Networking
in China (CHINACOM). IEEE, 2013, pp. 468–472.
[33] O. Adamuz-Hinojosa, L. Zanzi, V. Sciancalepore, A. Garcia-Saavedra, and X. CostaPérez, “Oranus: Latency-tailored orchestration via stochastic network calculus in 6g
o-ran,” arXiv preprint arXiv:2401.03812, 2024.
[34] X. Lin, L. Kundu, C. Dick, and S. Velayutham, “Embracing ai in 5g-advanced toward
6g: A joint 3gpp and o-ran perspective,” IEEE Communications Standards Magazine,
vol. 7, no. 4, pp. 76–83, 2023.

指導教授

胡誌麟(Chih-Lin Hu)

審核日期

2024-8-20

推文