使用深度強化學習在大規模多輸入多輸出正交分頻多重進接及混合波束成型系統之子載波配置

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：60

、訪客IP：18.225.92.25

姓名

李日維(Jih-Wei Li) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

使用深度強化學習在大規模多輸入多輸出正交分頻多重進接及混合波束成型系統之子載波配置
(Subchannel Allocation in Massive MIMO-OFDMA and Hybrid Beamforming Systems with Deep Reinforcement Learning)

相關論文

★ 利用手持式手機工具優化行動網路系統於特殊型活動環境	★ 穿戴裝置動態軌跡曲線演算法設計
★ 石英諧振器之電極面設計對振盪頻率擾動之溫度相依性研究	★ 股票開盤價漲跌預測
★ 感知無線電異質網路下以不完美頻譜偵測進行資源配置之探討	★ 大數量且有限天線之多輸入多輸出系統效能分析
★ 具有元學習分類權重轉移網路生成遮罩於少樣本圖像分割技術	★ 具有注意力機制之隱式表示於影像重建三維人體模型
★ 使用對抗式圖形神經網路之物件偵測張榮	★ 基於弱監督式學習可變形模型之三維人臉重建
★ 以非監督式表徵分離學習之邊緣運算裝置低延遲樂曲中人聲轉換架構	★ 基於序列至序列模型之 FMCW雷達估計人體姿勢
★ 基於多層次注意力機制之單目相機語意場景補全技術	★ 應用於3GPP WCDMA-FDD上傳鏈路系統的遞迴最小平方波束合成犛耙式接收機
★ 調適性遠時程瑞雷衰退通道預測演算法設計與性能比較	★ 智慧型天線之複合式到達方位-時間延遲估測演算法及Geo-location應用

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-8-29以後開放)

摘要(中)

結合類比和數位元件的混合式波束成型架構，是為了減少在毫米波 (mmWave) 頻段附有大規模多輸出多輸入 (MIMO) 系統中的硬體成本和功率消耗的一個替代方案。然而，由於通訊系統越來越複雜且伴隨著傳輸數據量大，人工智慧 (AI) 是一個有希望的輔助工具被用來協助我們解決高維和複雜的問題。在這篇論文中，我們著重的是藉由基於AI的子通道分配方法來最大化資料傳輸率同時考慮所有使用者的服務品質（QoS），而且我們認為射頻鏈 (RF chain) 的數量在實際的混合波束成型的架構中很稀少。這不只讓子通道分配對於大規模MIMO附加正交分頻多址 (MIMO-OFDMA) 和mmWave系統很重要，而且能夠讓系統在一個訊框中服務更多使用者。與傳統基於迭代的子載波配置方法不同，我們使用深度強化學習 (DRL) 演算法去解決即時策略問題。更進一步，我們提出基於競爭深度雙Q網路 (Dueling-DDQN) 去實行動態的子通道分配。
數值結果顯示提出方法的性能在混合式波束成型架構的測試集中隨著訓練漸漸趨近於貪婪法的性能，而且平均總和傳輸率和每位使用者的平均頻譜效益在合理的中斷率變化範圍內也都有提升。

摘要(英)

Hybrid beamforming, which combines analog and digital components, is an alternative for less power and hardware cost consumption in millimeter-wave (mmWave) with massive multiple-input multiple-output (MIMO) systems. However, because communication systems become more sophisticated and come with explosive transmission data, artificial intelligence (AI) is a promising auxiliary approach for assisting us to solve high-dimensional and complex problems. In this paper, we emphasize that the maximum sum rate is reached by AI-based subchannel allocation while considering all users’ quality of service (QoS) for data rate, and we assume that the number of radio frequency (RF) chains is rare in practical hybrid beamforming architecture. This practical assumption not only makes the subchannel allocation important for hybrid beamforming in the massive MIMO with Orthogonal frequency division multiple access (MIMO-OFDMA) and mmWave systems but also enables the system to serve more users at a time slot. Different from the conventional subcarrier allocation algorithms, we use a deep reinforcement learning (DRL) algorithm to solve real-time decision-making problems. Further, we propose the dueling double deep Q-network (Dueling-DDQN) to implement the dynamic subchannel allocation.
Simulation results show that the rate performance of the proposed algorithm in the testing set for hybrid beamforming architecture is gradually close to the greedy method with training. Moreover, the average sum rate and the average spectral efficiency of each user are also raised on the reasonable change of outage probability range.

關鍵字(中)

★ 深度強化學習
★ 資源配置
★ 毫米波
★ 大型多輸入多輸出
★ 混合式波束成型

關鍵字(英)

★ Deep Reinforcement Learning
★ Resource Allocation
★ mmWave
★ Massive-MIMO
★ Hybrid Beamforming

論文目次

論文摘要 i
Abstract iii
致謝 v
Contents vi
List of Figures vii
List of Tables viii
Chapter 1. Introduction 1
1.1. Millimeter-Wave 1
1.2. Massive MIMO 2
1.3. Hybrid Beamforming Architecture 3
1.4. Resource Allocation 6
1.5. Deep Reinforcement Learning 7
1.6. Related Works 8
1.7. Contributions 10
1.8. Organization 11
1.9. Abbreviations 11
1.10. Notation 14
Chapter 2. System Model 15
2.1. Signal Model 16
2.2. Channel Model 17
2.3. Problem Formulation 20
Chapter 3. Dynamic Subchannel Allocation with DRL-Based Method 23
3.1 Basic Concept of the DRL and RL 23
3.2 DRL Formulation for Subchannel Allocation 25
3.3 Dueling-DDQN Algorithm and Network structure 30
3.4 Hybrid Beamforming Design 36
Chapter 4. Computational Complexity 41
Chapter 5. Simulation Results & Training Details 45
Chapter 6. Conclusion 58
References 59

參考文獻

[1]nK. Mouratidis, A. Papagiannakis, “COVID-19, internet, and mobility: The rise of telework, telehealth, e-learning, and e-shopping,’’ in Sustainable Cities and Society,74 (2021) 103182.
[2]nM. Kamal, A. Aljohani, and E. Alanazi, ‘‘IoT meets COVID-19: Status, challenges, and opportunities,’’ 2020, arXiv:2007.12268. [Online]. Available: http://arxiv.org/abs/2007.12268.
[3] J. G. Andrews et al., ‘‘What Will 5G Be?,’’ in IEEE Journal on Selected Areas in Communications, vol. 32, no. 6, pp. 1065-1082, June 2014.
[4] T. S. Rappaport et al., ‘‘Millimeter Wave Mobile Communications for 5G Cellular: It Will Work!” in IEEE Access, vol. 1, pp. 335-349, May 2013.
[5] S. K. Yong, C. C. Chong, “An overview of multigigabit wireless through millimeter wave technology: Potentials and technical challenges,” EURASIP Journal on Wireless Communications and Networking, vol. 2007, no. 1, pp. 50–50, 2006.
[6] S. A. Hoseini, M. Ding, and M. Hassan, “Massive MIMO performance comparison of beamforming and multiplexing in the Terahertz band,” in Proc. IEEE Globecom Workshops, Dec. 2017, pp. 1–6.
[7] A. Swindlehurst, E. Ayanoglu, P. Heydari, and F. Capolino, “Millimeter-wave massive MIMO: The next wireless revolution?” IEEE Communication Magazine, vol. 52, no. 9, pp. 56–62, Sep. 2014.
[8] L. Lu, G. Y. Li, A. L. Swindlehurst, A. Ashikhmin and R. Zhang, ‘‘An Overview of Massive MIMO: Benefits and Challenges,” in IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 5, pp. 742-758, Oct. 2014.
[9] H. Q. Ngo, E. G. Larsson, and T. L. Marzetta, “Energy and Spectral Efficiency of Very Large Multiuser MIMO Systems,” IEEE Trans. Commun., vol. 61, Apr. 2013, pp. 1436–1449.
[10] O. El Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R. W. Heath, “Spatially sparse precoding in millimeter wave MIMO systems,” IEEE Trans. Wireless Commun., vol. 13, no. 3, pp. 1499–1513, Mar. 2014.
[11] T. E. Bogale, L. B. Le, A. Haghighat, and L. Vandendorpe, “On the number of RF chains and phase shifters, and scheduling design with hybrid analog–digital beamforming,” IEEE Trans. Wireless Commun., vol. 15, no. 5, pp. 3311–3326, May 2016.
[12] J. Zhang, X. Yu and K. B. Letaief, ‘‘Hybrid Beamforming for 5G and Beyond Millimeter-Wave Systems: A Holistic View,” in IEEE Open Journal of the Communications Society, vol. 1, pp. 77-91, 2020.
[13] A. F. Molisch et al., “Hybrid beamforming for massive MIMO: A survey,” IEEE Commun. Mag., vol. 55, no. 9, pp. 134–141, Sep. 2017.
[14] T. F. Maciel and A. Klein, ‘‘A resource allocation strategy for SDMA/OFDMA systems,” IST Mobile and Wireless Communications Summit, 1-5 July 2007.
[15] C. F. Tsai, C. J. Tsang, F.-C. Ren, and C.-M. Yen, “Adaptive radio resource allocation for downlink OFDMA/SDMA systems,” in Proc. IEEE ICC, Glasgow, U.K., 2007, pp. 5683–5688.
[16] Y. F. Chen and J. W. Chen, “A fast subcarrier, bit, and power allocation algorithm for multiuser OFDM-based systems,” in IEEE Trans. Veh. Technol., vol. 57, no. 2, pp. 873–881, Mar. 2008.
[17] Li, Suo Ping, X. F. Tian, and M. Li, “Study on Subcarrier and Power Allocation Algorithm for Downlink Transmission in Multiuser OFDM System,” in International Symposium on Computer Science and Computational Technology, 2008, pp. 295-298.
[18] Y. Yang, “Analysis of the Impact of Artificial Intelligence Development on Employment,” in International Conference on Computer Engineering and Application (ICCEA), 2020, pp. 324-327.
[19] S. Cheng, B. Wang, “An overview of Publications on Artificial Intelligence Research: A Quantitative Analysis of Recent Papers,” in Fifth International Joint Conference on Computational Sciences and Optimization, June 2012, pp. 683-686.
[20] C. Luo, L. Zhou, Q. Wei, “Identification of Research Fronts in Artificial Intelligence,” in 2017 2nd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), 2017, pp. 104-108.
[21] R. Li et al., “Intelligent 5G: When cellular networks meet artificial intelligence,” IEEE Wireless Commun., vol. 24, no. 5, pp. 175–183, Oct. 2017.
[22] R. Shafin, L. Liu, V. Chandrasekhar, H. Chen, J. Reed, and J. C. Zhang, “Artificial intelligence-enabled cellular networks: A critical path to beyond-5G and 6G,” IEEE Wireless Commun., vol. 27, no. 2, pp. 212–217, Apr. 2020.
[23] C. Zhang, P. Patras, and H. Haddadi, “Deep learning in mobile and wireless networking: A survey,” IEEE Commun. Surveys Tuts., vol. 21, no. 3, pp. 2224–2287, 3rd Quart, 2019.
[24] J. Liu, X. Ma, W. Han, L. Wang, “Resource Allocation in OFDMA Networks with Deep Reinforcement Learning,” IEEE 8th International Conference on Information, Communication and Network (ICICN), pp.111-117, 2020.
[25] R. Dong, C. She, W. Hardjawana, Y. Li, B. Vucetic, “Deep learning for radio resource allocation with diverse quality-of-service requirements in 5G,” IEEE Trans. Wireless Commun., vol. 20, no. 4, pp. 2309–2324, April 2021.
[26] Y. Yuan, Z. Yuan, and L. Tian, “5G Non-Orthogonal Multiple Access Study in 3GPP,” IEEE Commun. Mag., vol. 58, no. 7, July 2020, pp. 90–96.
[27] W. Shin, M. Vaezi, B. Lee, D. J. Love, J. Lee, H. V. Poor, “Non-Orthogonal Multiple Access in Multi-Cell Networks: Theory, Performance, and Practical Challenges,” IEEE Communications Magazine, vol. 55, no. 10, pp. 176-183, Oct. 2017.
[28] Y. Liu, Z. Qin, M. Elkashlan, Z. Ding, A. Nallanathan, and L. Hanzo, “Nonorthogonal multiple access for 5G and beyond,” Proceedings of the IEEE, vol. 105, no. 12, pp. 2347–2381, Dec. 2017.
[29] S. M. Tseng, Y. F. Chen, C. S. Tsai, W. D. Tsai, “Deep-Learning-Aided Cross-Layer Resource Allocation of OFDMA/NOMA Video Communication Systems,” IEEE Access, vol. 07, pp. 157730-157740, Oct. 2019.
[30] Y. Zhang, X. Wang, Y. Xu, “Energy-Efficient Resource Allocation in Uplink NOMA Systems with Deep Reinforcement Learning,” 2019 11th International Conference on Wireless Communications and Signal Processing (WCSP), 2019.
[31] C. He, Y. Hu, Y. Chen, B. Zeng, “Joint Power Allocation and Channel Assignment for NOMA with Deep Reinforcement Learning,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 10, pp. 2200-2210, Oct. 2019.
[32] W. Saetan, S. Thipchaksurat, “Power Allocation for Sum Rate Maximization in 5G NOMA System with Imperfect SIC: A Deep Learning Approach,” 2019 4th International Conference on Information Technology (InCIT), pp. 195-198, 2019.
[33] M. Liu, T. Song, G. Gui, “Deep Cognitive Perspective: Resource Allocation for NOMA-Based Heterogeneous IoT With Imperfect SIC,” IEEE Internet of Things Journal, vol. 6, no. 2, pp. 2885-2894, Apr. 2019.
[34] J. Xu, B. Ai, Y. Sun, Y. Chen, “Power Allocation for Millimeter-Wave Railway Systems with Multi-Agent Deep Reinforcement Learning,” 2020 IEEE Global Communications Conference, 2020.
[35] I. Ahmed, H. Khammari, “Joint machine learning based resource allocation and hybrid beamforming design for massive MIMO systems,” in Proc. IEEE Globecom Workshops, Dec. 2018.
[36] E. G. Larsson, O. Edfors, F. Tufvesson and T. L. Marzetta, ‘‘Massive MIMO for next generation wireless systems,” in IEEE Communications Magazine, vol. 52, no. 2, pp. 186-195, February 2014.
[37] F. Sohrabi and W. Yu, “Hybrid Analog and Digital Beamforming for mmWave OFDM Large-Scale Antenna Arrays,” IEEE J. Sel. Areas Commun., vol. 35, no. 7, pp. 1432–1443, Jul. 2017.
[38] Z. Pi and F. Khan, “An introduction to millimeter-wave mobile broadband systems,” IEEE Commun. Mag., vol. 49, no. 6, pp. 101–107, Jun. 2011.
[39] R. W. Heath, N. González-Prelcic, S. Rangan, W. Roh, and A. M. Sayeed, “An overview of signal processing techniques for millimeter wave MIMO systems,” IEEE J. Sel. Topics Signal Process., vol. 10, no. 3, pp. 436–453, Apr. 2016.
[40] 3GPP Technical Report 38.901 v16.1.0, Study on channel model for frequencies from 0.5 to 100 GHz (Release 16), 2020.
[41] V. Mnih, K. Kavukcuoglu, D. Silver, et al., ‘‘Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529-533, Feb.2015.
[42] 3GPP Technical specification 38.211 v15.2.0, Physical channels and modulation (Release 15), 2018.
[43] 3GPP Technical specification 38.214 v15.3.0, Physical layer procedures for data (Release 15), 2018.
[44] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp.533-536,1986.
[45] Z. Wang, N. de Freitas, and M. Lanctot, “Dueling Network Architectures for Deep Reinforcement Learning,” In Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016.
[46] A. F. Agarap, “Deep learning using rectified linear units (relu),” arXiv preprint arXiv:1803.08375, 2018.
[47] H. van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double Q-learning,” in Proc. Association for the Advancement of Artificial Intelligence, pp. 2094–2100, 2016.
[48] T. Hastie, R. Tibshirani, and J. Friedman, “The Elements of Statistical Learning,” ser. Springer Series in Statistics. New York, NY, USA: Springer New York Inc., 2001.
[49] V. N. Ha, D. H. N. Nguyen, and J. Frigon, “Subchannel allocation and hybrid precoding in millimeter-wave OFDMA systems,” IEEE Trans. Wireless Commun., vol.17, no.9, pp.5900-5914, Sep. 2018.
[50] Y. Kwon, J. Chung, and Y. Sung, “Hybrid beamformer design for mmWave wideband multiuser MIMO-OFDM systems: (Invited paper),” in Proc. IEEE 18th Int. Workshop Signal Process. Adv. Wireless Commun. (SPAWC), Sapporo, Japan, Jul. 2017, pp. 1–5.
[51] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[52] 3GPP Technical Specification 38.104 v16.5.0, Base Station (BS) radio transmission and reception (Release 16), 2020.
[53] X. Wan, “Influence of feature scaling on convergence of gradient iterative algorithm,” J. Physics: Conf. Series, vol. 1213, no. 3, pp. 1-5, 2019.
[54] R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” In ICML, 2013.

指導教授

陳永芳(Yung-Fang Chen)

審核日期

2022-9-12

推文