車聯網環境下基於行動邊緣計算卸載和多代理人 強化學習的即時影像及中繼傳輸方法

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：89

、訪客IP：13.58.28.196

姓名

宋柏廷(Bo-Ting Song) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

車聯網環境下基於行動邊緣計算卸載和多代理人強化學習的即時影像及中繼傳輸方法
(Video Streaming and Relaying with MEC Offloading and Multi-Agent Reinforcement Learning in VANETs)

相關論文

★ 非結構同儕網路上以特徵相似度為基準之搜尋方法	★ 以階層式叢集聲譽為基礎之行動同儕網路拓撲架構
★ 線上RSS新聞資料流中主題性事件監測機制之設計與實作	★ 耐延遲網路下具密度感知的路由方法
★ 整合P2P與UPnP內容分享服務之家用多媒體閘道器：設計與實作	★ 家庭網路下簡易無縫式串流影音播放服務之設計與實作
★ 耐延遲網路下訊息傳遞時間分析與高效能路由演算法設計	★ BitTorrent P2P 檔案系統下載端網路資源之可調式配置方法與效能實測
★ 耐延遲網路中利用訊息編碼重組條件之資料傳播機制	★ 耐延遲網路中基於人類移動模式之路由機制
★ 車載網路中以資料匯集技術改善傳輸效能之封包傳送機制	★ 適用於交叉路口環境之車輛叢集方法
★ 車載網路下結合路側單元輔助之訊息廣播機制	★ 耐延遲網路下以靜態中繼節點（暫存盒）最佳化訊息傳遞效能之研究
★ 耐延遲網路下以動態叢集感知建構之訊息傳遞機制	★ 跨裝置影音匯流平台之設計與實作

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-8-19以後開放)

摘要(中)

隨著互聯網的興起，車聯網（Vehicular Ad Hoc Networks, VANET）作為智能交通系統的核心組成部分，正逐漸受到關注。利用無線通信技術和先進的車輛感知器，車聯網實現了汽車間及汽車與基礎設施間的即時數據交換，提升了交通系統的安全性和效率。然而，快速都市化導致車輛數量劇增，交通事故頻發，尤其是追撞事故。為解決此問題，本研究提出基於5G網路的車輛影像協作共享功能，通過無線通信和跳點通訊技術延伸車輛影像傳輸範圍，並使用強化學習選擇最佳傳輸對象，提高影像品質。此外，本文引入多接取邊緣計算（Multi-access Edge Computing, MEC）以應對車
載系統計算能力不足導致的延遲問題。MEC將計算資源部署在靠近用戶車輛的位置，減少數據傳輸延遲並降低網路頻寬需求。
本文通過多代理人強化學習架構，多代理人深度確定性策略梯度（Multi-Agent Deep Deterministic Policy Gradient,MADDPG），結合邊緣計算，預測最佳路徑並選擇最合適的協作車輛，實現最佳傳輸效率，提升整體系統的性能和可靠性。最後進行不同強化學習模型之間的效能比較，證明部署多代理人強化學習能使系統得到長期最大報酬。

摘要(英)

With the rise of the IoT, Vehicular Ad Hoc Networks (VANET) have gradually gained attention as a core component of intelligent transportation systems. Utilizing wire
less communication technologies and advanced vehicle sensors, VANET enables real-time data exchange between vehicles and between vehicles and infrastructure, enhancing the safety and efficiency of transportation systems. However, rapid urbanization has led to a dramatic increase in the number of vehicles and frequent traffic accidents, particularly rear-end collisions. To address this issue, this study proposes a vehicle image cooperative sharing function based on 5G networks. By extending the range of vehicle image transmission through wireless communication and multi-hop transmission , and using reinforcement learning to select the best transmission targets, the quality of the images is improved.
In addition, this paper introduces Multi-access Edge Computing (MEC) to address the issues of insufficient computing ability and transmission delays in onboard systems.MEC deploys computing resources close to the user vehicles, reducing data transmissiondelays and less network bandwidth requirements.This paper employs a multi-agent reinforcement learning framework, Multi-Agent Deep Deterministic Policy Gradient (MADDPG), combining edge computing to predict optimal routes and select the most suitable cooperative vehicles, achieving optimal transmission efficiency and enhancing the overall system performance and reliability. Finally,
a performance comparison between different reinforcement learning models is conducted, demonstrating that deploying multi-agent reinforcement learning can achieve long-term
maximum rewards for the system.

關鍵字(中)

★ 車聯網
★ 跳點傳輸,
★ 多接取邊緣計算
★ 計算卸載
★ 強化學習

關鍵字(英)

★ VANET
★ hop transmission
★ multi-access Edge Computing
★ offloading
★ Reinforcement Learning

論文目次

目錄
摘要i
Abstract ii
圖目錄v
表目錄vi
1簡介1
2研究背景與文獻探討4
2.1車聯網. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1車用行動通訊網路. . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2車輛間的任務邊緣計算與卸載. . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.1多接取邊緣架構. . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.2 MEC之計算卸載. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3機器學習. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.1單代理人強化學習. . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.2多代理人強化學習. . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3系統架構19
3.1影像共享與跳點傳輸路徑. . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2車載環境. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3傳輸能耗. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4數據正規化. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
iii
3.5問題定義. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4強化學習27
4.1單代理人強化學習. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2多代理人強化學習. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5實驗與結果分析37
5.1實驗環境. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2實驗方法. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.2.1實驗參數設計. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.2.2模型參數設計. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.3實驗結果. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.3.1學習率(α)影響. . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.3.2衰減率(γ)影響. . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.3.3環境模擬與效能指標分析. . . . . . . . . . . . . . . . . . . . . . . 44
5.3.4傳輸延遲比較. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.3.5傳輸耗能比較. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6結論與未來研究59
7致謝60
參考文獻61

參考文獻

參考文獻
[1] Y.-C. Chu, “Using stackelberg game and multi-agent reinforcement learning to self
organize relaying groups for real-time video sharing in vehicular networks, https://
hdl.handle.net/11296/v5u5r2,” 2023.
[2] C. S. Evangeline, V. B. Kumaravelu, and A. Joshi, “Safety and driver assistance
in vanets: an experimental approach for v2v,” in 2019 International Conference on
Communication and Electronics Systems (ICCES). IEEE, 2019, pp. 397–402.
[3] P. Gomes, C. Olaverri-Monreal, and M. Ferreira, “Making vehicles transparent
through v2v video streaming,” IEEE Transactions on Intelligent Transportation Sys
tems, vol. 13, no. 2, pp. 930–938, 2012.
[4] X. Xiang, W. Qin, and B. Xiang, “Research on a dsrc-based rear-end collision warning
model,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 3, pp.
1054–1065, 2014.
[5] A. M. Khodayer, A. H. Najim, N. Alkhazraji, A. H. Alkhayyat, and F. H. Abbas,
“Performance evaluation of effective cluster head selection and maintenance for lte
based vehicular ad-hoc networks,” in 2023 6th International Conference on Engi
neering Technology and its Applications (IICETA). IEEE, 2023, pp. 714–719.
[6] J. Hu, S. Chen, L. Zhao, Y. Li, J. Fang, B. Li, and Y. Shi, “Link level performance
comparison between lte v2x and dsrc,” Journal of Communications and Information
Networks, vol. 2, no. 2, pp. 101–112, 2017.
[7] Z. Deng, Z. Cai, and M. Liang, “A multi-hop vanets-assisted offloading strategy in
vehicular mobile edge computing,” IEEE Access, vol. 8, pp. 53062–53071, 2020.
[8] M. M. Hamdi, S. A. Jassim, and B. S. Abdulhakeem, “Successful delivery using
stable multi-hop clustering protocol for energy efficient highway vanets,” in 2023 7th
International Symposium on Multidisciplinary Studies and Innovative Technologies
(ISMSIT). IEEE, 2023, pp. 1–6.
[9] D. Zhang, H. Ge, T. Zhang, Y.-Y. Cui, X. Liu, and G. Mao, “New multi-hop clus
tering algorithm for vehicular ad hoc networks,” IEEE Transactions on Intelligent
Transportation Systems, vol. 20, no. 4, pp. 1517–1530, 2018.
[10] K. Abboud, H. A. Omar, and W. Zhuang, “Interworking of dsrc and cellular network
technologies for v2x communications: A survey,” IEEE transactions on vehicular
technology, vol. 65, no. 12, pp. 9457–9470, 2016.
[11] S. A. Ahmad, A. Hajisami, H. Krishnan, F. Ahmed-Zaid, and E. Moradi-Pari, “V2v
system congestion control validation and performance,” IEEE Transactions on Ve
hicular Technology, vol. 68, no. 3, pp. 2102–2110, 2019.
[12] D. K. Nayak, S. V. Reddy, and S. D. Roy, “Performance evaluation of cv2x communi
cation,” in 2022 IEEE North Karnataka Subsection Flagship International Conference
(NKCon). IEEE, 2022, pp. 1–5.
[13] M. A. Palash and D. Wijesekera, “Adaptive traffic signal control using cv2x,” in 2023
IEEE 98th Vehicular Technology Conference (VTC2023-Fall). IEEE, 2023, pp. 1–7.
[14] Y. Tang, N. Cheng, W. Wu, M. Wang, Y. Dai, and X. Shen, “Delay-minimization
routing for heterogeneous vanets with machine learning based mobility prediction,”
IEEE Transactions on Vehicular Technology, vol. 68, no. 4, pp. 3967–3979, 2019.
[15] J. He, L. Cai, J. Pan, and P. Cheng, “Delay analysis and routing for two-dimensional
vanets using carry-and-forward mechanism,” IEEE Transactions on Mobile Comput
ing, vol. 16, no. 7, pp. 1830–1841, 2016.
[16] Y. Su, H. Cai, and J. Shi, “An improved realistic mobility model and mechanism
for vanet based on sumo and ns3 collaborative simulations,” in 2014 20th IEEE
International Conference on Parallel and Distributed Systems (ICPADS). IEEE,
2014, pp. 900–905.
[17] C.-M. Huang, M.-S. Chiang, D.-T. Dao, W.-L. Su, S. Xu, and H. Zhou, “V2v data
offloading for cellular network based on the software defined network (sdn) inside
mobile edge computing (mec) architecture,” IEEE Access, vol. 6, pp. 17741–17755,
2018.
[18] F. Liu, J. Chen, Q. Zhang, and B. Li, “Online mec offloading for v2v networks,”
IEEE Transactions on Mobile Computing, vol. 22, no. 10, pp. 6097–6109, 2022.
[19] Y. Wang, X. Hu, L. Guo, and Z. Yao, “Research on v2i/v2v hybrid multi-hop edge
computing offloading algorithm in iov environment,” in 2020 IEEE 5th International
Conference on Intelligent Transportation Engineering (ICITE). IEEE, 2020, pp.
336–340.
[20] K. Wang, X. Wang, X. Liu, and A. Jolfaei, “Task offloading strategy based on rein
forcement learning computing in edge computing architecture of internet of vehicles,”
IEEE Access, vol. 8, pp. 173779–173789, 2020.
[21] H. Wang, T. Lv, Z. Lin, and J. Zeng, “Energy-delay minimization of task migration
based on game theory in mec-assisted vehicular networks,” IEEE Transactions on
Vehicular Technology, vol. 71, no. 8, pp. 8175–8188, 2022.
[22] L. Luo, L. Sheng, H. Yu, and G. Sun, “Intersection-based v2x routing via rein
forcement learning in vehicular ad hoc networks,” IEEE Transactions on Intelligent
Transportation Systems, vol. 23, no. 6, pp. 5446–5459, 2021.
[23] J. Huang, J. Peng, H. Xiang, L. Li, and Y. Yang, “Hybrid spectrum access for v2v
heterogeneous networks with deep reinforcement learning,” in 2022 14th International
Conference on Wireless Communications and Signal Processing (WCSP). IEEE,
2022, pp. 1091–1095.
[24] S.-H. Wu, R.-H. Hwang, C.-Y. Wang, and C.-H. Chou, “Deep reinforcement learning
based resource allocation for 5g v2v groupcast communications,” in 2023 Interna
tional Conference on Computing, Networking and Communications (ICNC). IEEE,
2023, pp. 1–6.
[25] X. Zhang, M. Peng, S. Yan, and Y. Sun, “Deep-reinforcement-learning-based mode
selection and resource allocation for cellular v2x communications,” IEEE Internet of
Things Journal, vol. 7, no. 7, pp. 6380–6391, 2019.
[26] T. Li, K. Zhu, N. C. Luong, D. Niyato, Q. Wu, Y. Zhang, and B. Chen, “Applications
of multi-agent reinforcement learning in future internet: A comprehensive survey,”
IEEE Communications Surveys & Tutorials, vol. 24, no. 2, pp. 1240–1279, 2022.
[27] C. Lu, Z. Wang, W. Ding, G. Li, S. Liu, and L. Cheng, “Marvel: Multi-agent rein
forcement learning for vanet delay minimization,” China Communications, vol. 18,
no. 6, pp. 1–11, 2021.
[28] O. Urmonov, H. Aliev, and H. Kim, “Multi-agent deep reinforcement learning for
enhancement of distributed resource allocation in vehicular network,” IEEE Systems
Journal, vol. 17, no. 1, pp. 491–502, 2022.
[29] N. Hammami and K. K. Nguyen, “Multi-agent actor-critic for cooperative resource
allocation in vehicular networks,” in 2022 14th IFIP Wireless and Mobile Networking
Conference (WMNC). IEEE, 2022, pp. 93–100.
[30] T. Lyu, H. Xu, F. Liu, M. Li, L. Li, and Z. Han, “Two layer stackelberg game
based resource allocation in cloud-network convergence service computing,” IEEE
Transactions on Cognitive Communications and Networking, 2024.
[31] X. Chen, L. Jiao, W. Li, and X. Fu, “Efficient multi-user computation offloading
for mobile-edge cloud computing,” IEEE/ACM transactions on networking, vol. 24,
no. 5, pp. 2795–2808, 2015.
[32] S.-W. Kim, B. Qin, Z. J. Chong, X. Shen, W. Liu, M. H. Ang, E. Frazzoli, and
D. Rus, “Multivehicle cooperative driving using cooperative perception: Design and
experimental validation,” IEEE Transactions on Intelligent Transportation Systems,
vol. 16, no. 2, pp. 663–680, 2014.

指導教授

胡誌麟(Chih-Lin Hu)

審核日期

2024-8-20

推文