SD-WAN中一維卷積自編碼之流量分類與強化學習之服務導向多路徑路由

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：35

、訪客IP：3.16.212.44

姓名

邱楷程(Kai-Cheng Chiu) 查詢紙本館藏

畢業系所

軟體工程研究所

論文名稱

SD-WAN中一維卷積自編碼之流量分類與強化學習之服務導向多路徑路由
(Reinforcement Learning-based Service-oriented Multipath Routing with 1D Convolutional Autoencoder Traffic Classification in SD-WAN)

相關論文

★ 基於SVM之訊務分類機制及其於SDN網路之應用	★ 無特徵圖神經網路分類模型以偵測Android惡意軟體為例
★ 基於 Sidecar 的異質函數鏈無伺服器平台	★ 無線行動隨意網路上穩定品質服務路由機制之研究
★ 應用多重移動式代理人之網路管理系統	★ 應用移動式代理人之網路協同防衛系統
★ 鏈路狀態資訊不確定下QoS路由之研究	★ 以訊務觀察法改善光突發交換技術之路徑建立效能
★ 感測網路與競局理論應用於舒適性空調之研究	★ 以搜尋樹為基礎之無線感測網路繞徑演算法
★ 基於無線感測網路之行動裝置輕型定位系統	★ 多媒體導覽玩具車
★ 以Smart Floor為基礎之導覽玩具車	★ 行動社群網路服務管理系統－應用於發展遲緩兒家庭
★ 具位置感知之穿戴式行動廣告系統	★ 調適性車載廣播

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 ( 永不開放)

摘要(中)

為符合使用者越來越高的網路服務品質，網際網路從傳統基本的路由機制發展至今，逐漸演變為服務導向（service-oriented）的架構。為使網路更加彈性，許多新興的網路虛擬化技術，如網路功能虛擬化（Network Function Virtualization，NFV）與軟體定義網路（Software Defined Network，SDN）因此應運而生。而在服務導向的架構下，快速地、有效地在網路中分類封包便是一項重要的技術。藉由SDN中的控制器（controller）可程式化、中樞管理的優勢，網路管理者可在controller中撰寫程式，部署封包分類機制與路由決策。而以SDN的技術為基礎，廣域軟體定義網路（Software Defined Wide Area Network，SD-WAN）近年逐漸成為主流架構。在SD-WAN中，多種服務有多條路由可以選擇，且不同的服務對其網路品質的要求也有所不同，因此服務的流量分類（traffic classification）與路由決策（routing decision）是SD-WAN中不可或缺的部分。本論文為了在SD-WAN架構中達到流量分類與路由決策，提出兩個主要機制：CAPC，基於近年發展快速的卷積神經網路（Convolutional Neural Network，CNN）與自編碼（Deep Autoencoder，DAE）技術有效分類自行錄製與公開加密資料集的服務，分別達到優於其他深度學習模型的99.98%與97.42%的最高正確率； RED-STAR，基於深度強化學習（Deep Reinforcement Learning，DRL）技術，將CAPC所分類出的服務類型與機制中所正規化的網路指標作為DRL中神經網路的輸入值，逐漸學習並定期計算出每個服務應被導向何路由才能取得最大獎勵值（reward），並在實際服務流量情境下得到高於其他路徑分配演算法的1.7327和1.8642平均reward。

摘要(英)

To meet users’ requirements for higher service quality, the Internet has been evolving from the traditional routing mechanism to the current service-oriented architecture. To make networks more flexible, numerous emerging network virtualization technology, such as Network Function Virtualization (NFV) and Software Defined Network (SDN) were proposed. In the service-oriented architecture, instantly and effectively classifying packets during transmission is essential. With the advantages of programmability and centralized control of SDN controller, network managers can program within the controller, and deploy packet classification and routing mechanism. Based on SDN, Software Defined Wide Area Network (SD-WAN) has become the main stream of network architecture in recent years. In SD-WAN, there are plenty of network services having to be routed to several paths, and different services might even have different service quality demands. As a result, traffic classification and routing decision are indispensable in SD-WAN. To classify traffic and route service traffic, two mechanisms have been proposed: CAPC, which classifies packets of different kinds of services based on the recent fast-growing Convolutional Neural Network (CNN) and Deep Autoencoder (DAE), with the highest accuracies of 99.98% and 97.42% on the self-captured traffic and the open encrypted dataset; RED-STAR, based on Deep Reinforcement Learning, takes the classification results from CAPC and the regularized network metrics as input values for DRL, and learns to decide the path to which a service should be routed for the maximum reward value, with the average reward values of 1.7327 and 1.8642 in the real-case scenarios.

關鍵字(中)

★ 軟體定義廣域網路
★ 服務導向
★ 流量分類
★ 路由決策
★ 卷積神經網路
★ 自編碼
★ 網路指標
★ 深度強化學習

關鍵字(英)

★ Software Defined Wide Area Network
★ Service-Oriented
★ Traffic Classification
★ Routing Decision
★ Convolutional Neural Network
★ Deep Autoencoder
★ Network Metrics
★ Deep Reinforcement Learning

論文目次

摘要 i
Abstract ii
誌謝 iv
目錄 v
圖目錄 ix
表目錄 xv
第一章緒論 1
1.1. 概要 1
1.2. 研究動機 2
1.3. 研究目的 4
1.4. 章節架構 4
第二章背景知識與相關研究 5
2.1. 軟體定義廣域網路（SD-WAN） 5
2.1.1. 軟體定義網路（SDN） 5
2.1.2. 軟體定義廣域網路（SD-WAN） 7
2.2. 流量分類（Traffic Classification） 11
2.2.1. 基於埠號（Port-based） 12
2.2.2. 基於負載（Payload-based） 12
2.2.3. 基於機器學習（Machine Learning-based） 13
2.3. 深度學習（Deep Learning） 15
2.3.1. 深度神經網路（DNN） 15
2.3.2. 卷積神經網路（CNN） 18
2.4. 強化學習（RL） 23
2.4.1. Q-Learning 25
2.4.2. Deep Q-Learning 27
2.5. 相關研究 30
2.5.1. 網路流量分類 30
2.5.2. 多路徑路由 32
第三章研究方法 34
3.1. 服務流量處理與CAPC模型訓練 34
3.1.1. 網路服務流量錄製（Network Service Traffic Recording） 35
3.1.2. 資料前處理（Data Preprocessing） 38
3.1.3. CAPC模型（CAPC Model） 41
3.2. 多路徑SD-WAN與RED-STAR路由決策機制 47
3.2.1. 鏈接探索（Link-layer Discovery） 53
3.2.2. 網路狀態觀察（Network State Observation） 57
3.2.3. SMR機制 63
3.2.4. 網路指標常態化（Network State Normalization） 65
3.2.5. 路徑選擇（Path Decision） 68
3.2.6. 獎勵值規則（Reward Policy） 71
3.2.7. RED-STAR強化學習運作流程 74
3.3. 系統實作 79
3.3.1. CAPC系統實作 79
3.3.2. RED-STAR系統實作 81
第四章實驗與討論 84
4.1. 情境一：CAPC流量分類成效 84
4.1.1. 實驗一：CPU與GPU訓練速度比較 84
4.1.2. 實驗二：Fine-grained服務流量分類 87
4.1.3. 實驗三：Coarse-grained服務流量分類 93
4.1.4. 實驗四：開放VPN資料集fine-grained服務流量分類 95
4.1.5. 實驗五：分類成果穩定性比較 98
4.2. 情境二：RED-STAR運行環境功能驗證 100
4.2.1. 實驗六：網路指標正當性 100
4.2.2. 實驗七：CAPC於RED-STAR系統中之分類機制 107
4.2.3. 實驗八：動態路徑決策 108
4.3. 情境三：RED-STAR多重路由決策 111
4.3.1. 實驗九：獎勵值機制 112
4.3.2. 實驗十：不同服務的組合 116
4.3.3. 實驗十一：不同路由決策機制 121
4.3.4. 實驗十二：實際服務流量 124
第五章結論與未來研究方向 127
5.1. 結論 127
5.2. 未來研究 128
參考文獻 131

參考文獻

[1] Kreutz, Diego, et al. "Software-defined networking: A comprehensive survey." Proceedings of the IEEE 103.1 (2014): 14-76.
[2] Mijumbi, Rashid, et al. "Network function virtualization: State-of-the-art and research challenges." IEEE Communications surveys & tutorials 18.1 (2015): 236-262.
[3] Silver Peak Systems, Inc., “SD-WAN Explained”, 2020. Accessed on: April 19, 2020. [Online]. Available: https://www.silver-peak.com/sd-wan/sd-wan-explained
[4] Alpaydin, Ethem. Introduction to machine learning. MIT press, 2020.
[5] Beale, Hagan Demuth, Howard B. Demuth, and M. T. Hagan. "Neural network design." Pws, Boston (1996).
[6] LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436-444.
[7] Kaelbling, Leslie Pack, Michael L. Littman, and Andrew W. Moore. "Reinforcement learning: A survey." Journal of artificial intelligence research 4 (1996): 237-285.
[8] Mnih, Volodymyr, et al. "Playing atari with deep reinforcement learning." arXiv preprint arXiv:1312.5602 (2013).
[9] Ruder, Sebastian. "An overview of gradient descent optimization algorithms." arXiv preprint arXiv:1609.04747 (2016).
[10] McKeown, Nick, et al. "OpenFlow: enabling innovation in campus networks." ACM SIGCOMM Computer Communication Review 38.2 (2008): 69-74.
[11] Caraguay, Ángel Leonardo Valdivieso, et al. "SDN/NFV Architecture for IoT Networks." WEBIST. 2018.
[12] Davie, Bruce S., and Yakov Rekhter. MPLS: technology and applications. Morgan Kaufmann Publishers Inc., 2000.
[13] Dahlman, Erik, Stefan Parkvall, and Johan Skold. 4G: LTE/LTE-advanced for mobile broadband. Academic press, 2013.
[14] Cisco System, Inc., “Cisco Annual Internet Report (2018–2023) White Paper”, 2020. Accessed on April 20, 2020. [Online]. Available: https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html
[15] Wang, Pan, et al. "A survey of techniques for mobile service encrypted traffic classification using deep learning." IEEE Access 7 (2019): 54024-54033.
[16] Yan, Jinghua, and Jing Yuan. "A survey of traffic classification in software defined networks." 2018 1st IEEE International Conference on Hot Information-Centric Networking (HotICN). IEEE, 2018.
[17] IANA. Internet Assigned Numbers Authority, “Internet Assigned Numbers Authority”, 2020. Accessed on: April 9, 2020. [Online]. Available: https://www.iana.org/
[18] Joe, Touch, et al. “Service Name and Transport Protocol Port Number Registry”, 2020. Accessed on: April 9, 2020. [Online]. Available: https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml
[19] Moore, Andrew W., and Konstantina Papagiannaki. "Toward the accurate identification of network applications." International Workshop on Passive and Active Network Measurement. Springer, Berlin, Heidelberg, 2005.
[20] Finsterbusch, Michael, et al. "A survey of payload-based traffic classification approaches." IEEE Communications Surveys & Tutorials 16.2 (2013): 1135-1156.
[21] ClearCenter, “L7-Filter | ClearFoundation | Deep Packet Inspection”, 2020. Accessed on April 9, 2020. [Online]. Available: https://l7-filter.clearos.com/
[22] ntop, “nDPI – ntop”, 2020. Accessed on: April 9, 2020. [Online]. Available: https://www.ntop.org/products/deep-packet-inspection/ndpi/
[23] Gringoli, Francesco, et al. "Gt: picking up the truth from the ground for internet traffic." ACM SIGCOMM Computer Communication Review 39.5 (2009): 12-18.
[24] Dainotti, Alberto, Antonio Pescape, and Kimberly C. Claffy. "Issues and future directions in traffic classification." IEEE network 26.1 (2012): 35-40.
[25] Da Silva, Anderson Santos, et al. "Identification and selection of flow features for accurate traffic classification in SDN." 2015 IEEE 14th International Symposium on Network Computing and Applications. IEEE, 2015.
[26] Liu, Chien-Chang, et al. "Svm-based classification mechanism and its application in sdn networks." 2018 10th International Conference on Communication Software and Networks (ICCSN). IEEE, 2018.
[27] Qazi, Zafar Ayyub, et al. "Application-awareness in SDN." Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM. 2013.
[28] Zaki, Fatimah Audah Md, and Tan Saw Chin. "FWFS: Selecting Robust Features Towards Reliable and Stable Traffic Classifier in SDN." IEEE Access 7 (2019): 166011-166020.
[29] Xu, Jun, et al. "Deep neural networks for application awareness in SDN-based network." 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, 2018.
[30] Parsaei, Mohammad Reza, et al. "Network traffic classification using machine learning techniques over software defined networks." International Journal of Advanced Computer Science and Applications 8.7 (2017): 220-225.
[31] Zhou, Huiyi, et al. "A method of improved CNN traffic classification." 2017 13th International Conference on Computational Intelligence and Security (CIS). IEEE, 2017.
[32] Wang, Pan, et al. "Datanet: Deep learning based encrypted network traffic classification in sdn home gateway." IEEE Access 6 (2018): 55380-55391.
[33] Wang, Wei, et al. "End-to-end encrypted traffic classification with one-dimensional convolution neural networks." 2017 IEEE International Conference on Intelligence and Security Informatics (ISI). IEEE, 2017.
[34] Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." IEEE transactions on pattern analysis and machine intelligence 35.8 (2013): 1798-1828.
[35] ISHA SALIAN, https://blogs.nvidia.com/blog/2018/08/02/supervised-unsupervised-learning/
[36] Dudani, Sahibsingh A. "The distance-weighted k-nearest-neighbor rule." IEEE Transactions on Systems, Man, and Cybernetics 4 (1976): 325-327.
[37] Noble, William S. "What is a support vector machine?." Nature biotechnology 24.12 (2006): 1565-1567.
[38] Safavian, S. Rasoul, and David Landgrebe. "A survey of decision tree classifier methodology." IEEE transactions on systems, man, and cybernetics 21.3 (1991): 660-674.
[39] Liaw, Andy, and Matthew Wiener. "Classification and regression by randomForest." R news 2.3 (2002): 18-22.
[40] Krishna, K., and M. Narasimha Murty. "Genetic K-means algorithm." IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 29.3 (1999): 433-439.
[41] Johnson, Stephen C. "Hierarchical clustering schemes." Psychometrika 32.3 (1967): 241-254.
[42] Smyth, Padhraic. "Model selection for probabilistic clustering using cross-validated likelihood." Statistics and computing 10.1 (2000): 63-72.
[43] Watkins, Christopher JCH, and Peter Dayan. "Q-learning." Machine learning 8.3-4 (1992): 279-292.
[44] Sprague, Nathan, and Dana Ballard. "Multiple-goal reinforcement learning with modular sarsa (0)." (2003).
[45] Sutton, Richard S., et al. "Policy gradient methods for reinforcement learning with function approximation." Advances in neural information processing systems. 2000.
[46] Konda, Vijay R., and John N. Tsitsiklis. "Actor-critic algorithms." Advances in neural information processing systems. 2000.
[47] Mnih, Volodymyr, et al. "Playing atari with deep reinforcement learning." arXiv preprint arXiv:1312.5602 (2013).
[48] Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529-533.
[49] University of New Brunswick, “VPN-nonVPN dataset (ISCXVPN2016)”, 2016. Accessed on April 23, 2020. [Online]. Available: https://www.unb.ca/cic/datasets/vpn.html
[50] Zhou, Huiyi, et al. "A method of improved CNN traffic classification." 2017 13th International Conference on Computational Intelligence and Security (CIS). IEEE, 2017.
[51] Chiang, Yi-Rou, et al. "A multipath transmission scheme for the improvement of throughput over SDN." 2017 International Conference on Applied System Innovation (ICASI). IEEE, 2017.
[52] Yahya, Widhi, et al. "Improving End-to-End Network Throughput Using Multiple Best Paths Routing in Software Defined Networking." 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE). IEEE, 2018.
[53] Hossain, Md Billal, and Jin Wei. "Reinforcement Learning-Driven QoS-Aware Intelligent Routing for Software-Defined Networks." 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, 2019.
[54] Oliveira, Alexandre T., et al. "SDN-based architecture for providing QoS to high performance distributed applications." 2018 IEEE Symposium on Computers and Communications (ISCC). IEEE, 2018.
[55] Bi, Yuanguo, et al. "Intelligent QoS-aware Traffic Forwarding for SDN/OSPF Hybrid Industrial Internet." IEEE Transactions on Industrial Informatics (2019).
[56] VideoLAN, “Official download of VLC media player, the best Open Source player - VideoLAN”, 2020. Accessed on: April 23, 2020. [Online]. Available: https://www.videolan.org/vlc/
[57] Wikipedia, “pcap”, 2020, Accessed on: April 23, 2020. [Online]. Available: https://en.wikipedia.org/wiki/Pcap
[58] libpcap, “the-tcpdump-group/libpcap: the LIBpcap interface to various kernel packet capture mechanism”, 2020, Accessed on April 23, 2020. [Online]. Available: https://github.com/the-tcpdump-group/libpcap
[59] Riverbed Technology, “WinPcap – Home”, 2018, Accessed on: April 23, 2020 [Online]. Available: https://www.winpcap.org/
[60] nmap, “Npcap: Windows Packet Capture Library & Driver”, 2019, Accessed on: April 23, 2020 [Online]. Available: https://nmap.org/npcap/
[61] Wireshark, “Wireshark · Go Deep.”, 2020, Accessed on: April 23, 2020. [Online]. Available: https://www.wireshark.org/
[62] Prabhav Jain, “Feature Scaling | CommonLounge”, 2019. Accessed on: April 11, 2020 [Online]. Available: https://www.commonlounge.com/discussion/232c0df22e884e119eb1518f4c23372e/history
[63] Linux Foundation, “Open vSwitch”, 2020. Accessed on: April 24, 2020. [Online]. Available: https://www.openvswitch.org/
[64] Congdon, Paul. "Link layer discovery protocol and MIB." V1. 0 May 20.2002 (2002): 1-20.
[65] Zimmermann, Hubert. "OSI reference model-the ISO model of architecture for open systems interconnection." IEEE Transactions on communications 28.4 (1980): 425-432.
[66] Azzouni, Abdelhadi, et al. "Limitations of OpenFlow topology discovery protocol." 2017 16th Annual Mediterranean Ad Hoc Networking Workshop (Med-Hoc-Net). IEEE, 2017.
[67] Phemius, Kevin, and Mathieu Bouet. "Monitoring latency with openflow." Proceedings of the 9th International Conference on Network and Service Management (CNSM 2013). IEEE, 2013.
[68] Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980 (2014).
[69] NVIDIA Corporation, “CUDA Toolkit”, 2020, May 7, 2020. [Online]. Available: https://developer.nvidia.com/cuda-toolkit
[70] NVIDIA Corporation, “cuDNN Download”, 2020, May 7, 2020. [Online]. Available: https://developer.nvidia.com/rdp/cudnn-download
[71] Abadi, Martín, et al. "Tensorflow: Large-scale machine learning on heterogeneous distributed systems." arXiv preprint arXiv:1603.04467 (2016).
[72] Fred Klassen, and AppNeta, “tcprewrite”, 2018, Accessed on May 7, 2020. [Online]. Available: https://tcpreplay.appneta.com/wiki/tcprewrite-man.html
[73] Ryu SDN Framework Community, “Ryu SDN Framework”, 2017, Accessed on May 7, 2020. [Online]. Available: https://osrg.github.io/ryu/
[74] Mininet org., “Mininet, An Instant Virtual Network on your Laptop (or other PC)”, 2018, Accessed on May 7, 2020 [Online]. Available: http://mininet.org/
[75] Addy Yeow Chin Heng, “Bit-Twist Libpcap-based Ethernet packet generator”, 2012, Accessed on: May 7, 2020. [Online]. Available: http://bittwist.sourceforge.net/
[76] Fred Klassen, and AppNeta, “tcpreplay”, 2018, Accessed on May 7, 2020. [Online]. Available: https://tcpreplay.appneta.com/wiki/tcpreplay-man.html
[77] A. H. Lashkari, G. Draper-Gil, M. S. I. Mamun, and A. A. Ghorbani, ‘‘Characterization of encrypted and VPN trafﬁc using time-related features,’’ in Proc. Int. Conf. Inf. Syst. Secur. Privacy (ICISSP), Feb. 2016, pp. 407–414.
[78] Jon Dugan, et al. “iPerf - The ultimate speed test tool for TCP, UDP and SCTP,” 2013, Accessed on: May 12, 2020. [Online]. Available: https://iperf.fr/

指導教授

周立德(Li-Der Chou)

審核日期

2020-7-24

推文