毫米波無細胞大規模多輸入多輸出系統中使用深度強化學習技術應用於用戶選擇及功率分配

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：38

、訪客IP：3.146.105.194

姓名

吳玲萱(Lin-Hsuan Wu) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

毫米波無細胞大規模多輸入多輸出系統中使用深度強化學習技術應用於用戶選擇及功率分配
(User Selection and Power Allocation by Using Deep Reinforcement Learning in Millimeter Wave Cell Free Massive MIMO Systems)

相關論文

★ 利用手持式手機工具優化行動網路系統於特殊型活動環境	★ 穿戴裝置動態軌跡曲線演算法設計
★ 石英諧振器之電極面設計對振盪頻率擾動之溫度相依性研究	★ 股票開盤價漲跌預測
★ 感知無線電異質網路下以不完美頻譜偵測進行資源配置之探討	★ 大數量且有限天線之多輸入多輸出系統效能分析
★ 具有元學習分類權重轉移網路生成遮罩於少樣本圖像分割技術	★ 具有注意力機制之隱式表示於影像重建三維人體模型
★ 使用對抗式圖形神經網路之物件偵測張榮	★ 基於弱監督式學習可變形模型之三維人臉重建
★ 以非監督式表徵分離學習之邊緣運算裝置低延遲樂曲中人聲轉換架構	★ 基於序列至序列模型之 FMCW雷達估計人體姿勢
★ 基於多層次注意力機制之單目相機語意場景補全技術	★ 應用於3GPP WCDMA-FDD上傳鏈路系統的遞迴最小平方波束合成犛耙式接收機
★ 調適性遠時程瑞雷衰退通道預測演算法設計與性能比較	★ 智慧型天線之複合式到達方位-時間延遲估測演算法及Geo-location應用

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-8-29以後開放)

摘要(中)

無細胞大規模MIMO系統是一項具有潛力的技術，被提出為5G和6G的關鍵技術之一。不同於傳統的蜂巢式結構，在無細胞大規模MIMO系統中具有一個中央控制器及大量的無線存取點(AP)在覆蓋範圍內，並且每個無線存取點都具備大量的服務天線，能夠同時為覆蓋範圍內的所有用戶進行聯合傳輸。一個關鍵挑戰在於當無線存取點受到流量限制時，要如何選擇服務用戶及功率分配，使所有使用者能夠獲得最佳的資料傳輸率。在本篇論文中，使用深度強化學習(Deep reinforcement learning)技術應用在毫米波無細胞大規模MIMO系統中的用戶選擇及功率分配，透過放入適當的環境資訊和設定回饋方法，並且經過有效的訓練，來達到最適合的多用戶選擇及無線存取點的功率分配。我們的環境資訊包括所有無線存取點對於所有用戶的路徑損耗和通道狀態資訊，獎勵的方法設定為所有用戶的最大頻譜效率，透過隨機分布的無線存取點和用戶來做為訓練的輸入，在訓練結束後，將測試環境放入訓練好的神經網路，就能獲得連續動作，相當於用戶選擇及功率分配。最後根據深度強化學習的結果來計算頻譜效率，能夠證明此方法是具有優勢的。

摘要(英)

The cell-free massive MIMO system is a potential technology and has been proposed as one of the key technologies for 5G and 6G. Different from the traditional cellular structure, in a cell-free massive MIMO system there is a central controller and a number of wireless access points within the coverage area, and each access point has a large number of serving antennas. The system is capable of joint transmission for all user equipments within the coverage area at the same time. A key challenge is how to select service user equipments and allocate power so that all user equipments can obtain the better transmission data rate when the wireless access point is limited by traffic load. In this paper, deep reinforcement learning technique is applied to user selection and power allocation in millimeter wave cell-free massive MIMO systems. By putting in the appropriate channel state information and setting the reward method. After effective training, the optimal multi-user selection and power allocation of the access point can be achieved. Our environmental information includes the path loss and the channel state information for all access points for all user equipments. The reward method is set to the maximum spectral efficiency of all UEs. A random distribution of access points and user equipments is used as training data. After training, put the test environment into the trained neural network, and we can get continuous action, which is equivalent to user selection and power allocation. Finally, the spectral efficiency is calculated according to the results of deep reinforcement learning, which can prove the advantage of this method.

關鍵字(中)

★ 毫米波
★ 無細胞大規模多輸入多輸出系統
★ 用戶選擇
★ 功率分配
★ 深度強化學習技術

關鍵字(英)

★ Millimeter-wave
★ Cell Free Massive MIMO Systems
★ User Selection
★ Power Allocation
★ Deep Reinforcement Learning

論文目次

論文摘要 i
Abstract ii
致謝 iv
Contents v
List of Figures vii
List of Tables viii
Chpater 1. Introduction 1
1.1. Millimeter-Wave Frequency 1
1.2. Cell Free Massive MIMO 2
1.3. User Selection Under AP Constraints 3
1.4. Power Allocation 4
1.5. Deep Reinforcement Learning 5
1.6. Related Work 8
1.7. Contributions 10
1.8. Organization 11
1.9. Abbreviations 12
1.10. Notation 13
Chpater 2. System model 15
2.1. Channel Model 15
2.2. AP Constraints 18
2.3. Problem Formulation 19
Chpater 3. Propose Scheme 21
3.1. User-Centric Method Under AP Constraints 21
3.2. Water Filling Algorithm 24
3.3. Deep Neural Network 25
3.4. Stochastic Gradient Descent 27
3.5. DDPG-based Approach 28
Chpater 4. Simulation Results 35
4.1. Scenario 35
4.2. Hyper-Parameter Selection 38
4.3. Results 40
Chpater 5. Conclusion 47
Reference 48

參考文獻

[1] Zhouyue Pi and F. Khan, "An introduction to millimeter-wave mobile broadband systems," IEEE Communications Magazine, vol. 49, pp. 101-107, 2011.
[2] S. Rangan, T. S. Rappaport, and E. Erkip, "Millimeter-Wave Cellular Wireless Networks: Potentials and Challenges," Proceedings of the IEEE, vol. 102, no. 3, pp. 366-385, 2014.
[3] L. Lu, G. Y. Li, A. L. Swindlehurst, A. Ashikhmin, and R. Zhang, "An Overview of Massive MIMO: Benefits and Challenges," IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 5, pp. 742-758, 2014.
[4] J. Zhang, S. Chen, Y. Lin, J. Zheng, B. Ai, and L. Hanzo, "Cell-Free Massive MIMO: A New Next-Generation Paradigm," IEEE Access, vol. 7, pp. 99878-99888, 2019.
[5] S. Elhoushy, M. Ibrahim, and W. Hamouda, "Cell-Free Massive MIMO: A Survey," IEEE Communications Surveys & Tutorials, vol. 24, no. 1, pp. 492-523, 2022.
[6] S. Biswas and P. Vijayakumar, "AP selection in Cell-Free Massive MIMO system using Machine Learning Algorithm," presented at the 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), 2021.
[7] Y. Lin, R. Zhang, L. Yang, and C. L. a. L. Hanzo, "User Centric Clustering for Designing Ultradense Networks.," IEEE Vehicular Technology Magazine, vol. 14, no. 3, pp. 107-114, 2019.
[8] Aditya Akella, Glenn Judd, Srinivasan Seshan, and P. Steenkiste, "Self-Management in Chaotic Wireless Deployments," Proc. ACM MOBICOM, 2005.
[9] Qilin Qi, Andrew Minturn, and Y. L. Yang, "An efficient water filling algorithm for power allocation in OFDM-based cognitive radio systems," ICSAI, 2012.
[10] P.Mangayarkarasi, M.Ramya, and S.Jayashri, "Analysis of various power allocation algorithms for wireless networks," International Conference on Communication and Signal Processing, 2012.
[11] Timothy P. Lillicrap et al., "Continuous control with deep reinforcement learning," ICLR,arXiv preprint arXiv:1509.02971, 2015.
[12] Wang Qiang and Z. Zhongli, "Reinforcement learning model algorithms and its application," In 2011 International Conference on Mechatronic Science, Electric Engineering and Computer, pp. 1143-1146, 2011.
[13] S. Bhatnagar and S. Kumar, "A Simultaneous Perturbation Stochastic Approximation-Based Actor–Critic Algorithm for Markov Decision Processes," IEEE Transactions on Automatic Control, vol. 49, no. 4, pp. 592-598, 2004.
[14] F. Q. Lauzon, "An introduction to deep learning," In 2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA) pp. 1438-1439, 2012.
[15] R. Sutton, D. McAllester, S. Singh, and Y. Mansour, "Policy Gradient Methods for Reinforcement Learning with Function Approximation," Proceedings of the 9th Yale Workshop on Adaptive and Learning Systems, vol. 12, pp. 1057-1063, 2000.
[16] S.-F. H. Ronald Y. Chang, and Feng-Tsun Chien, "Reinforcement Learning-Based Joint Cooperation Clustering and Content Caching in Cell-Free Massive MIMO Networks," IEEE 94th Vehicular Technology Conference, 2021.
[17] Y. Zhao, I. G. Niemegeers, and S. M. H. De Groot, "Dynamic Power Allocation for Cell-Free Massive MIMO: Deep Reinforcement Learning Methods," IEEE Access, vol. 9, pp. 102953-102965, 2021.
[18] Yasser Al-Eryani, Mohamed Akrout, and E. Hossain, "Antenna Clustering for Simultaneous Wireless Information and Power Transfer in a MIMO Full-Duplex System: A Deep Reinforcement Learning-Based Design," IEEE Transactions on Communications, vol. 69, no. 4, pp. 2331-2345, 2021.
[19] F. S. Liang Chen, Kai Li, Ruiqing Chen, Yang Yang, Jun Wang, "Deep Reinforcement Learning for Resource Allocation in Massive MIMO," In 2021 29th European Signal Processing Conference (EUSIPCO),IEEE, pp. 1611-1615, 2021.
[20] H. Huang, Y. Yang, H. Wang, Z. Ding, H. Sari, and F. Adachi, "Deep Reinforcement Learning for UAV Navigation Through Massive MIMO Technique," IEEE Transactions on Vehicular Technology, vol. 69, no. 1, pp. 1117-1121, 2020.
[21] Vijay R. Konda and J. N. Tsitsiklis, "Actor-Critic Algorithms," NIPS Proceedings, vol. 42, pp. 1143-1166, 1999.
[22] E. G. Larsson, O. Edfors, F. Tufvesson, and T. L. Marzetta, "Massive MIMO for next generation wireless systems," IEEE Communications Magazine, vol. 52, no. 2, pp. 186-195, 2014.
[23] R. Irmer et al., "Coordinated multipoint Concepts performance and field trial results," IEEE Communications Magazine, vol. 49, no. 2, pp. 102-111, 2011.
[24] Z. Pi and F. Khan, "An introduction to millimeter wave mobile broadband systems," IEEE Communications Magazine, vol. 49, pp. 101-107, 2011.
[25] P. Zhu, H. Mao, J. Li, and X. You, "Energy efficient joint energy cooperation and power allocation in multiuser distributed antenna systems with hybrid energy supply," IET Communications, vol. 13, no. 2, pp. 153-161, 2019.
[26] T. C. Mai, H. Q. Ngo, and T. Q. Duong, "Downlink Spectral Efficiency of Cell-Free Massive MIMO Systems With Multi-Antenna Users," IEEE Transactions on Communications, vol. 68, no. 8, pp. 4803-4815, 2020.
[27] S. Buzzi and C. D′Andrea, "Cell-Free Massive MIMO: User-Centric Approach," IEEE Wireless Communications Letters, vol. 6, no. 6, pp. 706-709, 2017.
[28] H. T. Dao and S. Kim, "Power Allocation and User-AP Connection in Distributed Massive MIMO Systems," IEEE Communications Letters, vol. 25, no. 2, pp. 565-569, 2021.
[29] J. Wang, B. Wang, J. Fang, and H. Li, "Millimeter Wave Cell-Free Massive MIMO Systems: Joint Beamforming and AP-User Association," IEEE Wireless Communications Letters, vol. 11, no. 2, pp. 298-302, 2022.
[30] O. E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R. W. Heath, "Spatially Sparse Precoding in Millimeter Wave MIMO Systems," IEEE Transactions on Wireless Communications, vol. 13, no. 3, pp. 1499-1513, 2014.
[31] Mario Alonzo and S. Buzzi, "Cell-free and user-centric massive MIMO at millimeter wave frequencies," In 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC),IEEE, pp. 1-5, 2017.
[32] Stefano Buzzi, Carmen D’Andrea, and C. D’Elia, "User-Centric Cell-Free Massive MIMO with Interference Cancellation and Local ZF Downlink Precoding," In 2018 15th International Symposium on Wireless Communication Systems (ISWCS),IEEE, pp. 1-5, 2018.
[33] Lucas Claudino and T. Abrao, "Efficient ZF-WF Strategy for Sum-Rate Maximization of MU-MISO Cognitive Radio Networks," AEU-International Journal of Electronics and Communications, vol. 84, pp. 366-374, 2018.
[34] P. V. D. Tse, "Fundamentals Wireless Communications," Cambridge University Press, 2005. Cambridge University Press.
[35] V. Mnih, Kavukcuoglu, Koray, Silver, David,, A. Graves, Antonoglou,Ioannis,stra, Daan, Wierstra, and M. Riedmiller, "Playing Atari with Deep Reinforcement Learning," arXiv preprint arXiv:1312.5602, 2013.
[36] G. L. David Silver, Nicolas Heess, Thomas Degris, Daan Wierstra, Martin Riedmiller, "Deterministic Policy Gradient Algorithms," In International conference on machine learning, pp. 387-395, 2014.
[37] V. Mnih, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A, Veness, Joel, Bellemare, and G. Marc G, Alex, Riedmiller, Martin, Fidjeland, Andreas K, Ostrovski, Georg, et al., "Human-level control through deep reinforcement learning," Nature, pp. 529-533, 2015. Natrue.
[38] "3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Study on channel model for frequencies from 0.5 to 100 GHz (Release 16)."

指導教授

陳永芳(Yung-Fang Chen)

審核日期

2022-8-29

推文