結合類比和數位元件的混合式波束成型架構，是為了減少在毫米波 (mmWave) 頻段附有大規模多輸出多輸入 (MIMO) 系統中的硬體成本和功率消耗的一個替代方案。然而，由於通訊系統越來越複雜且伴隨著傳輸數據量大，人工智慧 (AI) 是一個有希望的輔助工具被用來協助我們解決高維和複雜的問題。在這篇論文中，我們著重的是藉由基於AI的子通道分配方法來最大化資料傳輸率同時考慮所有使用者的服務品質 （QoS），而且我們認為射頻鏈 (RF chain) 的數量在實際的混合波束成型的架構中很稀少。這不只讓子通道分配對於大規模MIMO附加正交分頻多址 (MIMO-OFDMA) 和mmWave系統很重要，而且能夠讓系統在一個訊框中服務更多使用者。與傳統基於迭代的子載波配置方法不同，我們使用深度強化學習 (DRL) 演算法去解決即時策略問題。更進一步，我們提出基於競爭深度雙Q網路 (Dueling-DDQN) 去實行動態的子通道分配。 數值結果顯示提出方法的性能在混合式波束成型架構的測試集中隨著訓練漸漸趨近於貪婪法的性能，而且平均總和傳輸率和每位使用者的平均頻譜效益在合理的中斷率變化範圍內也都有提升。 ;Hybrid beamforming, which combines analog and digital components, is an alternative for less power and hardware cost consumption in millimeter-wave (mmWave) with massive multiple-input multiple-output (MIMO) systems. However, because communication systems become more sophisticated and come with explosive transmission data, artificial intelligence (AI) is a promising auxiliary approach for assisting us to solve high-dimensional and complex problems. In this paper, we emphasize that the maximum sum rate is reached by AI-based subchannel allocation while considering all users’ quality of service (QoS) for data rate, and we assume that the number of radio frequency (RF) chains is rare in practical hybrid beamforming architecture. This practical assumption not only makes the subchannel allocation important for hybrid beamforming in the massive MIMO with Orthogonal frequency division multiple access (MIMO-OFDMA) and mmWave systems but also enables the system to serve more users at a time slot. Different from the conventional subcarrier allocation algorithms, we use a deep reinforcement learning (DRL) algorithm to solve real-time decision-making problems. Further, we propose the dueling double deep Q-network (Dueling-DDQN) to implement the dynamic subchannel allocation. Simulation results show that the rate performance of the proposed algorithm in the testing set for hybrid beamforming architecture is gradually close to the greedy method with training. Moreover, the average sum rate and the average spectral efficiency of each user are also raised on the reasonable change of outage probability range.
