以Deep Q Network為基礎之雲端工作排程最佳化

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：31

、訪客IP：3.145.37.211

姓名

梁藝鐘(Yi-Chung Liang) 查詢紙本館藏

畢業系所

資訊管理學系在職專班

論文名稱

以Deep Q Network為基礎之雲端工作排程最佳化
(A NOVEL SCHEDULING OPTIMIZATION BASED ON DEEP Q NETWORK)

相關論文

★ 台灣50走勢分析：以多重長短期記憶模型架構為基礎之預測	★ 以多重遞迴歸神經網路模型為基礎之黃金價格預測分析
★ 增量學習用於工業4.0瑕疵檢測	★ 遞回歸神經網路於電腦零組件銷售價格預測之研究
★ 長短期記憶神經網路於釣魚網站預測之研究	★ 基於深度學習辨識跳頻信號之研究
★ Opinion Leader Discovery in Dynamic Social Networks	★ 深度學習模型於工業4.0之機台虛擬量測應用
★ A Novel NMF-Based Movie Recommendation with Time Decay	★ 以類別為基礎sequence-to-sequence模型之POI旅遊行程推薦
★ A DQN-Based Reinforcement Learning Model for Neural Network Architecture Search	★ Neural Network Architecture Optimization Based on Virtual Reward Reinforcement Learning
★ 生成式對抗網路架構搜尋	★ 以漸進式基因演算法實現神經網路架構搜尋最佳化
★ Enhanced Model Agnostic Meta Learning with Meta Gradient Memory	★ 遞迴類神經網路結合先期工業廢水指標之股價預測研究

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-7-1以後開放)

摘要(中)

根據國外研究報告，欲提高企業競爭性，可在公司內部積極引進數位轉型，就學生從事的產業電信業而言，數位轉型是有助於提高客戶數及降低單位成本，公司也在這兩年大力推動雲端 (Azure)、AI及大數據 (Databricks平台)等技術落地，我們單位在這方面的具體作法，是全面導入倉儲資料上雲，對比於地端的資料倉儲系統架構 (Teradata)，機器租借/授權使用費用昂貴，且容量缺乏彈性擴充，雲端架構則有建置容易，可依需求用量擴充等優勢。
在導入過程，需依據選用的 PaaS平台 (Databricks)，學習開發新的ETL Job並依據雲端特性，重新規劃其工作排程，預計每天會有數以千隻ETL Job被運作，需在公司制定的KPI要求的時間跑完，最後要能做到成本/效益最佳化，雲端工作排程這個議題，也就變成極具有挑戰性，在排程過程中，考慮平台的特性，需針對資源分配 (VM)、工作優先級 (Job) 等多個因素，思考並做出最佳解決方案。
為了解決這個問題，本研究評估了不同的組合方法，後續進行了實驗和比較，希望能收集不同組合方法在不同情況下的表現，進而找出最適合的方法進行工作排程優化。實驗分成四個部分：
實驗一，自訂CNN驗證了不同的資料集，矩陣大小不一樣
實驗二，評估了同一資料集，三種不同的DNN (CNN、resNet-18、mobileNet)
實驗三，resNet-18驗證不同的資料集，矩陣大小不一樣
實驗四，實驗三基礎上加入Dueling DQN，驗證不同的資料集，矩陣大小不一樣
使用workflow和VM的相關資源特徵值，可減少狀態空間的複雜度，使得後續強化學習演算法更加高效和穩定。本研究使用的DDDQN演算法是增強式學習的一種，Agent使用端到端的強化學習直接從高維度感官輸入中學習成功的策略，是將深度卷積網路跟強化學習結合起來的架構。而傳統的工作排程問題中，常是使用啟發式算法或是基於規則的方法進行優化，這些方法需要手動設計決策規則，無法適應不斷變化的生產環境。

摘要(英)

According to international research reports, to enhance corporate competitiveness, businesses can actively promote digital transformation within the company. In the case of the telecommunications industry that students are engaged in, digital transformation helps increase the number of customers and reduce unit costs. In the past two years, the company has been vigorously promoting the implementation of technologies such as cloud (Azure), AI, and big data (Databricks platform). Our unit′s specific approach is to fully migrate warehouse data to the cloud. Compared to the on-premises data warehouse system architecture (Teradata), which has high machine rental/license usage fees and lacks flexible capacity expansion, the cloud architecture offers advantages such as easy deployment and scalability according to demand.
During the implementation process, it is necessary to learn and develop new ETL Jobs based on the selected PaaS platform (Databricks) and redesign the job scheduling according to cloud characteristics. It is estimated that thousands of ETL Jobs will be run daily, which need to be completed within the company′s established KPI requirements. The ultimate goal is to optimize cost-effectiveness, making cloud job scheduling a highly challenging topic. During the scheduling process, considering the platform′s characteristics, the best solution must be found by focusing on multiple factors such as resource allocation (VM) and job priority (Job).
To solve this problem, this study evaluated different combination methods, followed by experiments and comparisons, hoping to collect the performance of different combination methods under various scenarios and find the most suitable method for optimizing job scheduling. The experiments were divided into four parts:
Experiment 1: Custom CNN validated different datasets with different matrix sizes.
Experiment 2: Evaluated three different DNNs (CNN, ResNet-18, MobileNet) on the same dataset.
Experiment 3: ResNet-18 validated different datasets with different matrix sizes.
Experiment 4: Added Dueling DQN, validating different datasets with different matrix sizes.
By using Job and Cluster′s resource-related feature values, this approach can reduce the complexity of the state space, making the deep reinforcement learning algorithm in the second stage more efficient and stable. The DDDQN (Dueling Double Deep Q Network) algorithm is a type of reinforcement learning. The agent uses end-to-end reinforcement learning to learn successful strategies directly from high-dimensional sensory inputs, combining deep convolutional networks with reinforcement learning architecture. In traditional job scheduling problems, heuristic algorithms or rule-based methods are often used for optimization. These methods often require manual design of decision rules and cannot adapt to the ever-changing production environment.

關鍵字(中)

★ 雲端工作排程
★ 強化學習
★ DDDQN
★ CNN
★ 數位轉型

關鍵字(英)

★ Cloud Job Scheduling
★ Reinforcement Learning
★ Dueling Double DQN
★ CNN
★ Digital Transformation

論文目次

1 第一章緒論 1
1.1 研究背景 2
1.2 研究動機 2
1.3 研究目的 3
1.4 論文架構 6
2 第二章文獻探討 7
2.1 Microsoft Azure 介紹 7
2.2 Databricks 介紹 7
2.3 雲端工作排程 8
2.4 Job-Shop Scheduling Problem(JSSP) 10
3 第三章研究方法 13
3.1 資料前處理&資料建檔 14
3.2 模型建構 16
3.3 超參數設定 22
4 第四章研究結果 26
4.1 資料描述 26
4.2 環境說明 28
4.3 實驗一結果(自訂 CNN) 31
4.4 實驗二結果(DNN 比較) 33
4.5 實驗三結果(resNet-18) 35
4.6 實驗四結果(add Dueling) 36
5 第五章結論建議 39
5.1 研究結論及貢獻 39
5.2 研究限制 40
5.3 未來建議 41
6 參考文獻 42

參考文獻

6參考文獻
參考書目
[1] Valdez-de-Leon, Omar. 「A Digital Maturity Model for Telecommunications Service Providers」. Technology Innovation Management Review 6, 期 8 (2016年): 19–32.
[2] Ivančić, Lucija, Vesna Vukšić及Mario Spremić. 「Mastering the Digital Transformation Process: Business Practices and Lessons Learned」. Technology Innovation Management Review 9, 期 2 (2019年): 36–50. https://doi.org/10.22215/timreview/1217.
[3] Snežana, Radukić, Mastilo Zoran及Kostić Zorana. 「Effects of Digital Transformation and Network Externalities in the Telecommunication Markets」. ECONOMICS 7, 期 2 (2019年12月1日): 31–42. https://doi.org/10.2478/eoik-2019-0019.
[5] Mezmaz, M., N. Melab, Y. Kessaci, Y. C. Lee, E. -G. Talbi, A. Y. Zomaya及D. Tuyttens. 「A Parallel Bi-Objective Hybrid Metaheuristic for Energy-Aware Scheduling for Cloud Computing Systems」. Journal of Parallel and Distributed Computing 71, 期 11 (2011年11月1日): 1497–1508. https://doi.org/10.1016/j.jpdc.2011.04.007.
[6] Armbrust, Michael, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, 等. 「A view of cloud computing」. Communications of the ACM 53, 期 4 (2010年4月1日): 50–58. https://doi.org/10.1145/1721654.1721672.
[7] Dikaiakos, Marios D., Dimitrios Katsaros, Pankaj Mehra, George Pallis及Athena Vakali. 「Cloud Computing: Distributed Internet Computing for IT and Scientific Research」. IEEE Internet Computing 13, 期 5 (2009年9月): 10–13. https://doi.org/10.1109/MIC.2009.103.
[8] Li, Qiang, 及Yike Guo. 「Optimization of Resource Scheduling in Cloud Computing」. 收入 2010 12th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, 315–20, 2010. https://doi.org/10.1109/SYNASC.2010.8.
[9] Buyya, Rajkumar. 「Market-Oriented Cloud Computing: Vision, Hype, and Reality of Delivering Computing as the 5th Utility」. 收入 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, 1–1, 2009. https://doi.org/10.1109/CCGRID.2009.97.
[10] 「NIST Cloud Computing Program - NCCP」. NIST. 引見於 2023年4月18日. https://www.nist.gov/programs-projects/nist-cloud-computing-program-nccp.
[16] Wang, Yuanbin, Kangjie Hong, Jun Zou, Tao Peng及Huayong Yang. 「A CNN-Based Visual Sorting System With Cloud-Edge Computing for Flexible Manufacturing Systems」. IEEE Transactions on Industrial Informatics 16, 期 7 (2020年7月): 4726–35. https://doi.org/10.1109/TII.2019.2947539.
[17] Wang, Xin, 及Hong Shen. 「A Scalable Deep Reinforcement Learning Model for Online Scheduling Coflows of Multi-Stage Jobs for High Performance Computing」. arXiv, 2021年12月21日. https://doi.org/10.48550/arXiv.2112.11055.
[18] Peng, Yanghua, Yixin Bao, Yangrui Chen, Chuan Wu及Chuanxiong Guo. 「Optimus: an efficient dynamic resource scheduler for deep learning clusters」. 收入 Proceedings of the Thirteenth EuroSys Conference, 1–14. EuroSys ’18. New York, NY, USA: Association for Computing Machinery, 2018. https://doi.org/10.1145/3190508.3190517.
[19] Chang, Bao, Hsiu-Fen Tsai及Yu-Chieh Lin. 「Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches」. Computer Modeling in Engineering & Sciences 134, 期 2 (2022年): 783–815. https://doi.org/10.32604/cmes.2022.020128.
[20] Hadjar, Karim, 及Ahmed Jedidi. 「A New Approach for Scheduling Tasks and/or Jobs in Big Data Cluster」. 收入 2019 4th MEC International Conference on Big Data and Smart City (ICBDSC), 1–4, 2019. https://doi.org/10.1109/ICBDSC.2019.8645613.
[21] Swarup, Shashank, Elhadi M. Shakshuki及Ansar Yasar. 「Task Scheduling in Cloud Using Deep Reinforcement Learning」. Procedia Computer Science, The 12th International Conference on Ambient Systems, Networks and Technologies (ANT) / The 4th International Conference on Emerging Data and Industry 4.0 (EDI40) / Affiliated Workshops, 184 (2021年1月1日): 42–51. https://doi.org/10.1016/j.procs.2021.03.016.
[22] Ye, Yufei, Xiaoqin Ren, Jin Wang, Lingxiao Xu, Wenxia Guo, Wenqiang Huang及Wenhong Tian. 「A New Approach for Resource Scheduling with Deep Reinforcement Learning」. arXiv, 2018年6月21日. https://doi.org/10.48550/arXiv.1806.08122.
[23] Saraswathi, A. T., Y. R. A. Kalaashri及S. Padmavathi. 「Dynamic Resource Allocation Scheme in Cloud Computing」. Procedia Computer Science, Graph Algorithms, High Performance Implementations and Its Applications ( ICGHIA 2014 ), 47 (2015年1月1日): 30–36. https://doi.org/10.1016/j.procs.2015.03.180.
[24] Alizadeh. 「Learning scheduling algorithms for data processing clusters」. 收入 Proceedings of the ACM Special Interest Group on Data Communication, 270–88. SIGCOMM ’19. New York, NY, USA: Association for Computing Machinery, 2019. https://doi.org/10.1145/3341302.3342080.
[25] Xu, Jianqiao, Zhuohan Xu及Bing Shi. 「Deep Reinforcement Learning Based Resource Allocation Strategy in Cloud-Edge Computing System」. Frontiers in Bioengineering and Biotechnology 10 (2022年). https://www.frontiersin.org/articles/10.3389/fbioe.2022.908056.
[26] Lillicrap, Timothy P., Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver及Daan Wierstra. 「Continuous control with deep reinforcement learning」. arXiv, 2019年7月5日. https://doi.org/10.48550/arXiv.1509.02971.
[27] Szegedy, Christian, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke及Andrew Rabinovich. 「Going Deeper with Convolutions」. arXiv, 2014年9月16日. https://doi.org/10.48550/arXiv.1409.4842.
[28] Howard, Andrew G., Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto及Hartwig Adam. 「MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications」. arXiv, 2017年4月16日. https://doi.org/10.48550/arXiv.1704.04861.
[29] Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, 等. 「Human-Level Control through Deep Reinforcement Learning」. Nature 518, 期 7540 (2015年2月): 529–33. https://doi.org/10.1038/nature14236.
[30] Hasselt, Hado van, Arthur Guez及David Silver. 「Deep Reinforcement Learning with Double Q-learning」. arXiv, 2015年12月8日. https://doi.org/10.48550/arXiv.1509.06461.
[31] Wang, Ziyu, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot及Nando de Freitas. 「Dueling Network Architectures for Deep Reinforcement Learning」. arXiv, 2016年4月5日. https://doi.org/10.48550/arXiv.1511.06581.
[32] Han, Bao-An, 及Jian-Jun Yang. 「Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN」. IEEE Access 8 (2020年): 186474–95. https://doi.org/10.1109/ACCESS.2020.3029868.
[33] Gu, Shixiang, Timothy Lillicrap, Ilya Sutskever及Sergey Levine. 「Continuous Deep Q-Learning with Model-based Acceleration」. arXiv, 2016年3月2日. https://doi.org/10.48550/arXiv.1603.00748.
[34] Gu, Shixiang, Timothy Lillicrap, Ilya Sutskever及Sergey Levine. 「Continuous Deep Q-Learning with Model-based Acceleration」. arXiv, 2016年3月2日. https://doi.org/10.48550/arXiv.1603.00748.
參考網站
[4] https://www.techbang.com/posts/103904-far-eastone-and-microsoft-form-a-strategic-alliance
[11] https://learn.microsoft.com/zh-tw/azure/cost-management-billing/reservations/exchange-and-refund-azure-reservations
[12] https://zh.wikipedia.org/zh-tw/Microsoft_Azure
[13] https://learn.microsoft.com/zh-tw/azure/cost-management-billing/reservations/manage-reserved-vm-instance#change-optimize-setting-for-reserved-vm-instances
[14] https://zh.wikipedia.org/wiki/Databricks
[15] https://learn.microsoft.com/zh-tw/azure/databricks/introduction/
https://azure.microsoft.com/zh-tw/pricing/details/virtual-machines/linux/#pricing
https://azure.microsoft.com/en-us/pricing/details/databricks/

指導教授

陳以錚(Yi-Cheng Chen)

審核日期

2023-7-4

推文