| 摘要: | 隨著行動通訊快速發展，第三代合作夥伴計畫 (3GPP) 於第17 版標準 (Rel-17) 中，於
第五代行動通訊 (5G New Radio，NR) 要容納數以萬計之設備 (User Equipment，UE)，考量
未來行動通訊系統 (Beyond 5G 或 6G) 將同時支援更大量之終端設備，且服務模式由現行下
行主導 (DL dominant) 逐漸改變為上行主導 (UL dominant)，因此需更多頻譜資源傳送資料。
高頻段如FR2 符合頻寬需求。然而，高頻段之特性需使用更大子載波間距 (Sub-Carrier Spacing，
SCS) 抵抗相位噪聲（phase noise）。提高子載波間距也將造成每個時隙 (Slot) 時間縮短，進
一步壓縮資源排程器演算法所需運算時間之上限。若基站無法及時完成上下行資源排程，將
造成資源浪費，違背增加頻寬之初衷。
針對上行傳輸，此研究提出一種新穎之上行排程器—基於深度強化式學習之輕量化排程
器 (DRL-based Lightweight Scheduler，DRL-LS)，依據排隊中封包之資料特徵 (如剩餘延遲預
算 (Remaining Packet Delay Budget，R-PDB) 與服務等級等) 決定上行之優先等級。考慮排程
器演算法運算時間受限，資源排程器僅排程高優先等級封包；針對尚未排程之上行通道資源，
導入基於競爭 (Contention-based) 之通訊協定，使符合條件之上行封包 (亦即較不急迫之封包)
可於此資源內進行隨機競爭，競爭失敗之封包將於後續上行資源進行重傳。當上行封包隨時
間提高優先等級時，將改以傳統排程模式(亦即免競爭 (Contention-free) 模式) 進行排程，以
符合封包延遲預算。此 DRL-LS 透過 DRL 學習如何與通道環境互動於達成最高吞吐量與控
制排程器演算法運算時間之間取得平衡。
最後，為驗證DRL-LS與競爭上行行為之可行性，本研究將上述DRL-LS之代理人 (Agent)
部署於RIC 內，並且使用OpenAirInterface5G (OAI) 與Mosaic5G FlexRIC 開源軟體進行實際
驗證，觀察UEs 進行競爭之行為。;With the rapid development of mobile communications, the 3rd Generation Partnership Project
(3GPP) in Release 17 specification has aimed to accommodate numerous User Equipments (UE) in
5G New Radio (NR) network. Considering the future mobile communication systems (beyond 5G or
6G), an even larger number of devices should be supported. Furthermore, the transmission behavior
is gradually shifting from downlink-dominant (DL dominant) to uplink-dominant (UL dominant).
This shift necessitates more frequency resources, namely bandwidth (BW), for UL data transmission.
Higher frequency bands, such as FR2, can fulfill the bandwidth requirements. However, the
characteristics of higher frequency bands require larger Sub-Carrier Spacing (SCS) to counteract
phase noise (PN). Increasing the SCS also leads to shorter slot duration, which further shortens the
computational time budget of scheduler. If the base station (gNB) fails to accomplish scheduling in
time, it will result in resource waste consequently.
For uplink transmission, this study proposes a novel uplink scheduler named the Deep
Reinforcement Learning-based Lightweight Scheduler (DRL-LS). The DRL-LS determines the
priority of UL data in the queue by its characteristic, such as the Remaining Packet Delay Budget (RPDB)
and the Quality of Service (QoS). Due to the limited time budget for scheduling, the scheduler
only schedules data of high priority in a contention-free manner. For those unscheduled UL data, a
contention-based (CB) UL transmission protocol is proposed, allowing eligible uplink data (i.e., less
urgent data) to access CB resources in a contention manner. Data packets that experience contention
failure will be retransmitted in subsequent UL resources. To meet the packet delay budget, the priority
of queueing UL data increases over time. Additionally, the DRL-LS would learn how to interact with
the environment to achieve a balance between the maximum throughput and media access protocol.
Finally, to validate the performance of the proposed DRL-LS, this study deploys the agent in the
RAN Intelligence Controller (RIC), which is built over the open platform including the
OpenAirInterface5G (OAI) and Mosaic5G FlexRIC open source software. |
