感知無線電網路之多通道交會問題:強化學習應用

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：46

、訪客IP：3.133.87.156

姓名

徐媺晴(Mei-Ching HSU) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

感知無線電網路之多通道交會問題:強化學習應用
(Multi-channel Rendezvous in Cognitive Radio Network: a Reinforcement Learning Approach)

相關論文

★ 運用SIFT特徵進行光學影像目標識別	★ 語音關鍵詞辨識擷取系統
★ 適用於筆記型電腦之WiMAX天線研究	★ 應用於凱氏天線X頻段之低雜訊放大器設計
★ 適用於802.11a/b/g WLAN USB dongle曲折型單極天線設計改良	★ 應用於行動裝置上的雙頻(GPS/BT)天線
★ SDH設備單體潛伏性障礙效能分析與維運技術	★ 無風扇嵌入式觸控液晶平板系統小型化之設計
★ 自動化RFID海關通關系統設計	★ 發展軟體演算實現線性調頻連續波雷達測距系統之設計
★ 近場通訊之智慧倉儲管理	★ 在Android 平台上實現NFC 室內定位
★ Android應用程式開發之電子化設備巡檢	★ 鏈路預算估測預期台灣衛星通訊的發展
★ 在中上衰落通道中分集結合技術之二階統計特性	★ 先進長程演進系統中載波聚合技術的初始同步

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文為探討如何使用強化學習解決在感知無線電網路中多通道交會問題。感知無線電網路中多通道交會問題是指兩個二級用戶如何在有限的時間內選擇跳到同個通道並成功交換彼此的訊息。本篇論文是在一個對稱性、同步、同質性以及全局共同標號的假設下進行研究，同時為了更貼近實際情況，我們假設每個通道有著不同的通道狀態，並且無法讓用戶得知該通道的狀態情況，某些狀態可能會導致雖然兩個用戶在同個通道卻無法成功通訊。此種無法得知通道狀態的情況下的交會問題稱作盲交會。我們在上述假設下提出了一個快速強化學習演算法，讓兩個用戶學習在不同的狀態下成功交會的交會策略。我們提出的快速強化學習演算法能大幅增加通道選擇策略學習的效率，並且在收斂後學到一組能與已有最佳解的特定通道狀態相比擬的平均交會時間（ETTR），與在沒有最佳解的狀態下有著最佳的效能（最低的ETTR）。

摘要(英)

In this thesis, we consider the multichannel rendezvous problem in cognitive radio networks (CRNs) where the probability that two users hopping on the same channel have a successful rendezvous is a function of channel states. The channel states are modelled by stochastic processes with joint distributions known to users. However, the exact state of a channel at any time is not observable. We derived that the lower bound of the ETTR of the general channel model is the ETTR of the fast time-varying channel model and the upper bound is the ETTR of the slow time-varying channel model. By formulating such a multichannel rendezvous problem as an adversarial bandit problem, we propose using a reinforcement learning approach to learn the channel selection probabilities pi(t), i = 1; 2; : : : ;N. Our experimental results show that the reinforcement learning approach is very effective and yields comparable ETTRs when comparing to various approximation policies in the literature.

關鍵字(中)

★ 強化學習
★ 多通道交會
★ 無線感知系統

關鍵字(英)

論文目次

目　　　　錄
中文提要 ……………………………………………………………… i
英文摘要 ……………………………………………………………… ii
誌謝 ……………………………………………………………… iii
目錄 ……………………………………………………………… iv
圖目錄 ……………………………………………………………… v
表目錄 ……………………………………………………………… vi
符號說明 ……………………………………………………………… vii
一、緒論………………………………………………………… 1
二、系統模型…………………………………………………… 11
三、相關論文回顧……………………………………………… 13
3-1 廣義時變通道模型………………………………………… 13
3-2 兩種狀態的馬可夫通道模型……………………………… 15
3-3 正相關馬可夫鍊中的通道交會時間期望值下界………… 16
3-4 將多通道交會問題視為強化學習問題…………………… 18
四、用於解決感知無線電網路多通道交會問題之快速收斂強化學習演算法……………………………………………… 20
五、實驗與綜合比較…………………………………………… 25
六、結論………………………………………………………… 36
參考文獻 ……………………………………………………………… 38

圖目錄　List of Figures
圖一在ω=0.5時，快速強化學習演算法根據不同ρ所學到的通道選擇策略……………………………………………… 27
圖二在ω=0.5時，〔36〕演算法根據不同ρ所學到的通道選擇策略（直接採用〔36〕的圖一）……………………… 29
圖三快速強化學習演算法的學習曲線………………………… 34

表目錄　List of Tables
表一在ρ=0.1時，不同ω情況下的ETTR…………………… 31
表二在ρ=0.5時，不同ω情況下的ETTR…………………… 32
表三在ρ=0.9時，不同ω情況下的ETTR…………………… 32

參考文獻

［1］Kolodzy, Paul, and Interference Avoidance. "Spectrum policy task force." Federal Commun. Comm., Washington, DC, Rep. ET Docket 40.4 (2002): 147-158.
［2］Mitola, Joseph, and Gerald Q. Maguire. "Cognitive radio: making software radios more personal." IEEE personal communications 6.4 (1999): 13-18.
［3］Chen, Tao, et al. "CogMesh: A cluster-based cognitive radio network." 2007 2nd IEEE international symposium on new frontiers in dynamic spectrum access networks. IEEE, 2007.
［4］Le, Long, and Ekram Hossain. "A MAC protocol for opportunistic spectrum access in cognitive radio networks." 2008 IEEE Wireless Communications and Networking Conference. IEEE, 2008.
［5］Zhang, Xi, and Hang Su. "CREAM-MAC: Cognitive radio-enabled multi-channel MAC protocol over dynamic spectrum access networks." IEEE Journal of Selected Topics in Signal Processing 5.1 (2010): 110-123.
［6］Zhao, Jun, Haitao Zheng, and Guang-Hua Yang. "Distributed coordination in dynamic spectrum allocation networks." First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005.. IEEE, 2005.
［7］Chang, Guey-Yun, and Jen-Feng Huang. "A fast rendezvous channel-hopping algorithm for cognitive radio networks." IEEE Communications Letters 17.7 (2013): 1475-1478.
［8］Shih, C-F., Tsung Ying Wu, and Wanjiun Liao. "DH-MAC: A dynamic channel hopping MAC protocol for cognitive radio networks." 2010 IEEE International Conference on Communications. IEEE, 2010.
［9］Bahl, Paramvir, Ranveer Chandra, and John Dunagan. "SSCH: slotted seeded channel hopping for capacity improvement in IEEE 802.11 ad-hoc wireless networks." Proceedings of the 10th annual international conference on Mobile computing and networking. 2004.
［10］Bian, Kaigui. "Maximizing rendezvous diversity in rendezvous protocols for decentralized cognitive radio networks." IEEE transactions on Mobile Computing 12.7 (2012): 1294-1307.
［11］Bian, Kaigui, Jung-Min Park, and Ruiliang Chen. "A quorum-based framework for establishing control channels in dynamic spectrum access networks." Proceedings of the 15th annual international conference on Mobile computing and networking. 2009.
［12］Chang, Guey-Yun, et al. "Novel channel-hopping schemes for cognitive radio networks." IEEE Transactions on Mobile Computing 13.2 (2012): 407-421.
［13］DaSilva, Luiz A., and Igor Guerreiro. "Sequence-based rendezvous for dynamic spectrum access." 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks. IEEE, 2008.
［14］Hou, Fen, et al. "Asynchronous multichannel MAC design with difference-set-based hopping sequences." IEEE Transactions on Vehicular Technology 60.4 (2011): 1728-1739.
［15］Kondareddy, Yogesh Reddy, and Prathima Agrawal. "Synchronized MAC protocol for multi-hop cognitive radio networks." 2008 IEEE International Conference on Communications. IEEE, 2008.
［16］Lin, Zhiyong, et al. "Jump-stay based channel-hopping algorithm with guaranteed rendezvous for cognitive radio networks." 2011 Proceedings IEEE INFOCOM. IEEE, 2011.
［17］Mo, Jeonghoon, Hoi-Sheung Wilson So, and Jean Walrand. "Comparison of multichannel MAC protocols." IEEE Transactions on mobile computing 7.1 (2007): 50-65.
［18］Shin, Jongmin, Dongmin Yang, and Cheeha Kim. "A channel rendezvous scheme for cognitive radio networks." IEEE Communications Letters 14.10 (2010): 954-956.
［19］Wang, Jen-Hung, et al. "A Reinforcement Learning Approach for the Multichannel Rendezvous Problem." 2019 IEEE Globecom Workshops (GC Wkshps). IEEE, 2019.
［20］Yang, D., J. Shin, and C. Kim. "Deterministic rendezvous scheme in multichannel access networks." Electronics Letters 46.20 (2010): 1402-1404.
［21］Zhang, Yifan, et al. "ETCH: Efficient channel hopping for communication rendezvous in dynamic spectrum access networks." 2011 Proceedings IEEE INFOCOM. IEEE, 2011.
［22］Bian, Kaigui, Jung-Min Park, and Ruiliang Chen. "A quorum-based framework for establishing control channels in dynamic spectrum access networks." Proceedings of the 15th annual international conference on Mobile computing and networking. 2009.
［23］Yang, D., J. Shin, and C. Kim. "Deterministic rendezvous scheme in multichannel access networks." Electronics Letters 46.20 (2010): 1402-1404.
［24］Theis, Nick C., Ryan W. Thomas, and Luiz A. DaSilva. "Rendezvous for cognitive radios." IEEE transactions on mobile computing 10.2 (2010): 216-227.
［25］Lin, Zhiyong, et al. "Jump-stay based channel-hopping algorithm with guaranteed rendezvous for cognitive radio networks." 2011 Proceedings IEEE INFOCOM. IEEE, 2011.
［26］Gu, Zhaoquan, et al. "Nearly optimal asynchronous blind rendezvous algorithm for cognitive radio networks." 2013 IEEE international conference on sensing, communications and networking (SECON). IEEE, 2013.
［27］Chang, Guey-Yun, and Jen-Feng Huang. "A fast rendezvous channel-hopping algorithm for cognitive radio networks." IEEE Communications Letters 17.7 (2013): 1475-1478.
［28］Chang, Guey-Yun, et al. "Novel channel-hopping schemes for cognitive radio networks." IEEE Transactions on Mobile Computing 13.2 (2012): 407-421.
［29］Gu, Zhaoquan, Qiang-Sheng Hua, and Weiguo Dai. "Fully distributed algorithms for blind rendezvous in cognitive radio networks." Proceedings of the 15th ACM international symposium on Mobile ad hoc networking and computing. 2014.
［30］Chang, Cheng-Shang, et al. "Efficient encoding of user IDs for nearly optimal expected time-to-rendezvous in heterogeneous cognitive radio networks." IEEE/ACM Transactions on Networking 25.6 (2017): 3323-3337.
［31］Li, Guyue, et al. "Deterministic distributed rendezvous algorithms for multi-radio cognitive radio networks." Proceedings of the 17th ACM international conference on Modeling, analysis and simulation of wireless and mobile systems. 2014.
［32］Yu, Lu, et al. "Multiple radios for fast rendezvous in cognitive radio networks." IEEE Transactions on Mobile Computing 14.9 (2014): 1917-1931.
［33］ Yu, Lu, et al. "Adjustable rendezvous in multi-radio cognitive radio networks." 2015 IEEE global communications conference (GLOBECOM). IEEE, 2015.
［34］Chang, Yeh-Cheng, Cheng-Shang Chang, and Jang-Ping Sheu. "An enhanced fast multi-radio rendezvous algorithm in heterogeneous cognitive radio networks." IEEE Transactions on Cognitive Communications and Networking 4.4 (2018): 847-859.
［35］Chang, Cheng-Shang, et al. "ETTR Bounds and Approximation Solutions of Blind Rendezvous Policies in Cognitive Radio Networks with Random Channel States." arXiv preprint arXiv:1906.10424 (2019).
［36］Wang, Jen-Hung, et al. "A Reinforcement Learning Approach for the Multichannel Rendezvous Problem." 2019 IEEE Globecom Workshops (GC Wkshps). IEEE, 2019.
［37］Ross, Sheldon M. "Stochastic Processes. John Wiley & Sons." New York (1996).
［38］Chang, Cheng-Shang, XiuLi Chao, and Michael Pinedo. "Integration of discrete-time correlated Markov processes in a TDM system." Probability in the Engineering and Informational Sciences 4.1 (1990): 29-56.
［39］Marshall, Albert W., Ingram Olkin, and Barry C. Arnold. Inequalities: theory of majorization and its applications. Vol. 143. New York: Academic press, 1979.
［40］Tchen, André H. "Inequalities for distributions with given marginals." The Annals of Probability 8.4 (1980): 814-827.
［41］Rolski, Tomasz. "Upper bounds for single server queues with doubly stochastic Poisson arrivals." Mathematics of Operations Research 11.3 (1986): 442-450.
［42］Auer, Peter, et al. "The nonstochastic multiarmed bandit problem." SIAM journal on computing 32.1 (2002): 48-77.

指導教授

林嘉慶(Jia-Chin Lin)

審核日期

2020-7-14

推文