LMM：Kubernetes橫向移動攻擊中基於強化學習的緩解機制

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：9

、訪客IP：18.97.14.86

姓名

洪琪懿(Ci-Yi Hung) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

LMM：Kubernetes橫向移動攻擊中基於強化學習的緩解機制
(LMM: A Reinforcement-Learning-Based Mitigation Mechanism of Lateral Movement in Kubernetes)

相關論文

★ 無線行動隨意網路上穩定品質服務路由機制之研究	★ 應用多重移動式代理人之網路管理系統
★ 應用移動式代理人之網路協同防衛系統	★ 鏈路狀態資訊不確定下QoS路由之研究
★ 以訊務觀察法改善光突發交換技術之路徑建立效能	★ 感測網路與競局理論應用於舒適性空調之研究
★ 以搜尋樹為基礎之無線感測網路繞徑演算法	★ 基於無線感測網路之行動裝置輕型定位系統
★ 多媒體導覽玩具車	★ 以Smart Floor為基礎之導覽玩具車
★ 行動社群網路服務管理系統－應用於發展遲緩兒家庭	★ 具位置感知之穿戴式行動廣告系統
★ 調適性車載廣播	★ 車載網路上具預警能力之車輛碰撞避免機制
★ 應用於無線車載網路上之合作式交通資訊傳播機制以改善車輛擁塞	★ 智慧都市中應用車載網路以改善壅塞之調適性虛擬交通號誌

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 ( 永不開放)

摘要(中)

隨著微服務架構（Microservices Architecture）與容器技術日益普及，主流的容器編排平台Kubernetes 雖提供多種安全模組，仍因其高度互通的網路架構與權限設定常遭錯誤設置，導致橫向移動攻擊（Lateral Movement）的風險攀升。本論文提出 Lateral Movement Mitigation（LMM）之機制，整合事件追蹤、風險評估（Risk Assessment）與強化學習（Reinforcement Learning，RL），以強化 Kubernetes 應對橫向移動攻擊的能力。LMM 利用搭配自定義規則的 Falco 擷取容器事件資料，並以為基礎高階馬可夫鏈（High-Order Markov Chain）建構轉移矩陣（Transition Probability Matrix）估算容器指令序列的轉移機率（Transition Probability）以作風險評估，進一步作為 RL 代理之輸入狀態。RL 代理採用根據 MITRE ATT&CK 緩解建議設計的緩解行動，如套用 NetworkPolicy、減少 Service Account 權限或套用 Pod Security Admission（PSA）規則，以動態增強 Kubernetes 之內建安全模組。實驗結果顯示，LMM 在命名空間繞道攻擊中Accuracy 較 Warp 提升 24.73%，F1-score 較 ADA 提升 29.09%；在 RBAC 錯誤設置情境中則分別較 Warp 提升 21.61% 與 31.51%。在緩解所需時間上，LMM 採取的行動中最快可減少 Warp 98.54% 與 ADA 98.38%，展現高效且具即時性的緩解能力。綜上所述，LMM 結合即時追蹤、風險評估與自動決策，提供一套兼具效率與準確性的 Kubernetes 橫向移動攻擊主動防禦解決方案。

摘要(英)

With the growing adoption of microservices architecture and container technologies, Kubernetes—while offering a variety of built-in security modules—remains vulnerable to lateral movement due to its highly interconnected network architecture and frequent misconfigurations in permission settings. This study proposes the Lateral Movement Mitigation (LMM) mechanism, which integrates event tracking, risk assessment, and reinforcement learning (RL) to enhance Kubernetes′ defense against lateral movement. LMM leverages Falco with custom rules to capture container-level event data and utilizes a high-order Markov chain to construct a transition probability matrix for estimating the likelihood of command sequences. These transition probabilities are then used for risk assessment and provided as input states to the RL agent. The RL agent selects mitigation actions—such as applying NetworkPolicy, reducing Service Account privileges, or enforcing Pod Security Admission (PSA) rules—based on recommendations from the MITRE ATT&CK framework, thereby dynamically strengthening Kubernetes′ native security modules. Experiments show that LMM improves accuracy by 24.73% over Warp and F1-score by 29.09% over ADA in the Kubernetes namespace bypass scenario. In the RBAC misconfiguration scenario, LMM outperforms Warp by 21.61% in accuracy and 31.51% in F1-score. In terms of mitigation latency, LMM achieves up to 98.54% and 98.38% faster response times compared to Warp and ADA, respectively, demonstrating its effectiveness and real-time responsiveness. In summary, LMM combines real-time monitoring, risk modeling, and automated decision-making to deliver an efficient and accurate proactive defense solution against lateral movement in Kubernetes.

關鍵字(中)

★ 橫向移動攻擊
★ Kubernetes 安全
★ 強化學習
★ 馬可夫鏈
★ 事件追蹤
★ 動態防禦

關鍵字(英)

★ Lateral Movement
★ Kubernetes Security
★ Reinforcement Learning
★ Markov Chain
★ Event Tracking
★ Dynamic Defense

論文目次

摘要 i
Abstract ii
誌謝 iii
圖目錄 vii
表目錄 ix
第一章緒論 1
1.1. 研究動機 2
1.2. 研究目的 3
1.3. 章節架構 4
第二章背景知識與相關研究 5
2.1. Kubernetes 5
2.2. 橫向移動攻擊 Lateral Movement 8
2.3. 強化學習 Reinforcement Learning 10
2.4. 高階馬可夫鏈 High Order Markov Chain 13
2.5. 相關研究 14
第三章提出的LMM系統 17
3.1. 系統架構與設計 17
3.2. 系統運作流程 19
3.2.1. 容器事件蒐集 19
3.2.2. 轉移機率矩陣之計算 22
3.2.3. 風險評估與風險分數之計算 27
3.2.4. 橫向移動攻擊緩解問題之 MDP 建模 31
3.2.5. 緩解行動之設計 33
第四章實驗與討論 43
4.1. 系統環境與軟硬體規格 43
4.2. 情境一：不同學習機制與 RL 模型之學習成效 45
4.2.1. 實驗一：不同 RL 模型之學習成效 47
4.2.2. 實驗二：分析 CL 對模型之影響 49
4.3. 情境二：超參數對模型效能之影響 52
4.3.1. 實驗三：不同 timestep 模型之緩解效能分析 53
4.3.2. 實驗四：分析 reward shaping 之閾值設定對模型之影響 57
4.3.3. 實驗五：分析馬可夫鏈階數對模型之影響 60
4.3.4. 實驗六：分析 binary reward 設定對模型之影響 62
4.4. 情境三：LMM、Warp 與 ADA 緩解效能之比較 64
4.4.1. 實驗七：緩解 Kubernetes 命名空間繞道攻擊之效能 64
4.4.2. 實驗八：緩解容器逃逸攻擊之效能 67
4.4.3. 實驗九：緩解RBAC錯誤設置攻擊之效能 70
4.5. 情境四：系統開銷分析 73
4.5.1. 實驗十： Warp、ADA和LMM之CPU使用量 73
4.5.2. 實驗十一：Warp、ADA和LMM之記憶體使用量 74
4.5.3. 實驗十二：Warp、ADA和LMM緩解行動所需時間 75
第五章結論與未來研究方向 77
5.1. 結論 77
5.2. 研究限制 78
5.3. 未來研究 79
參考文獻 81

參考文獻

[1] N. Alshuqayran, N. Ali and R. Evans, “A systematic mapping study in microservice architecture,” 2016 IEEE 9th International Conference on Service-Oriented Computing and Applications (SOCA), pp. 44-51, 2016, doi: 10.1109/SOCA.2016.15.
[2] M. Ait Said, A. Ezzati, S. Mihi, and L. Belouaddane, “Microservices adoption: an industrial inquiry into factors influencing decisions and implementation strategies,” International Journal of Computing and Digital Systems, vol. 15, no. 1, pp. 1417-1432, 2024, doi: 10.12785/ijcds/1501100.
[3] Gartner, “Microservices Architecture: Have Engineering Organizations Found Success?” Accessed on Jun 18, 2025. [Online]. Available: https://www.gartner.com/peer-community/oneminuteinsights/omi-microservices-architecture-have-engineering-organizations-found-success-u6b.
[4] C. Pahl, A. Brogi, J. Soldani, and P. Jamshidi, “Cloud container technologies: a state-of-the-art review,” IEEE Transactions on Cloud Computing, vol. 7, no. 3, pp. 677-692, 2017, doi: 10.1109/TCC.2017.2702586.
[5] E. Casalicchio, “Container orchestration: a survey,” Systems Modeling: Methodologies and Tools, pp. 221-235, 2019, doi: 10.1007/978-3-319-92378-9_14.
[6] The Linux Foundation, “With Kubernetes, the U.S. department of defense is enabling DevSecOps on F-16s and battleships,” Accessed on Jun 18, 2025. [Online]. Available: https://www.cncf.io/blog/2020/05/07/with-kubernetes-the-u-s-department-of-defense-is-enabling-devsecops-on-f-16s-and-battleships/
[7] The Kubernetes Authors, “Kubernetes user case studies,” Accessed on Jun 18, 2025. [Online]. Available: https://kubernetes.io/case-studies/
[8] A. A. Tripathi, “Attacking and defending kubernetes, ” Dublin Business School, 2024.
[9] S. Krishnapriya and S. Singh, “A Comprehensive Survey on Advanced Persistent Threat (APT) Detection Techniques,” Computers, Materials & Continua, vol. 80, no. 2, 2024.
[10] A. D. Kent, “Comprehensive, Multi-Source Cyber-Security Events Data Set,” 2015, doi: 10.17021/1179829.
[11] B. A. Powell, “Role-based lateral movement detection with unsupervised learning,” arXiv preprint arXiv:2108.02713, 2021, doi: 10.48550/arXiv.2108.02713.
[12] I. J. King and H. H. Huang, “Euler: Detecting network lateral movement via scalable temporal link prediction,” ACM Transactions on Privacy and Security, vol. 26, no. 3, pp. 1-36, 2023, doi: 10.1145/3588771.
[13] D. Kushwaha et al., “Lateral movement detection using user behavioral analysis,” arXiv preprint arXiv:2208.13524, 2022, doi: 10.48550/arXiv.2208.13524.
[14] Z. Tian et al., “Real-Time Lateral Movement Detection Based on Evidence Reasoning Network for Edge Computing Environment, ” in IEEE Transactions on Industrial Informatics, vol. 15, no. 7, pp. 4285-4294, July 2019, doi: 10.1109/TII.2019.2907754.
[15] The Linux Foundation, “Cloud native security microsurvey: more than 80% of organizations want to build modern security systems with open source software,” Accessed on Jun 18, 2025. [Online]. Available: https://www.cncf.io/blog/2021/10/12/cloud-native-security-microsurvey-more-than-80-of-organizations-want-to-build-modern-security-systems-with-open-source-software/
[16] B. Gajbhiye, O. Goel, and P. K. Gopalakrishna Pandian, “Security challenges in the container cloud,” in 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), pp. 137-145, 2021, doi: 10.36676/jqst.v1.i2.16.
[17] K. German and O. Ponomareva, “An overview of container security in a kubernetes cluster,” in 2023 IEEE Ural-Siberian Conference on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT), pp. 283-285, 2023, doi: 10.1109/USBEREIT58508.2023.10158865.
[18] A. Virkud, M. A. Inam, A. Riddle, J. Liu, G. Wang, and A. Bates, “How does endpoint detection use the MITRE ATT&CK framework?,” in 33rd USENIX Security Symposium (USENIX Security 24), pp. 3891-3908, 2024.
[19] The Linux Foundation, “Falco,” Accessed on Jun 18, 2025. [Online]. Available: https://falco.org/
[20] Falco, “On host anomaly detection framework - new anomalydetection Plugin,” Accessed on Jun 18, 2025. [Online]. Available: https://github.com/falcosecurity/falco/blob/master/proposals/20230620-anomaly-detection-framework.md.
[21] W.-K. Ching et al., “Higher-order markov chains,” Markov Chains: Models, Algorithms and Applications, pp. 141-176, 2013, doi: 10.1007/978-1-4614-6312-2_6.
[22] A. K. Shakya, G. Pillai, and S. Chakrabarty, “Reinforcement learning algorithms: A brief survey,” Expert Systems with Applications, vol. 231, p. 120495, 2023, doi: 10.1016/j.eswa.2023.120495.
[23] C. Liu, C.-K. Tan, Y.-S. Fang, and T.-S. Lok, “The security risk assessment methodology,” Procedia Engineering, vol. 43, pp. 600-609, 2012.
[24] S. Katz, “Estimation of probabilities from sparse data for the language model component of a speech recognizer,” in IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 35, no. 3, pp. 400-401, 1987, doi: 10.1109/TASSP.
[25] The MITRE Corporation, “Lateral movement,” Accessed on Jun 18, 2025. [Online]. Available: https://attack.mitre.org/tactics/TA0008/
[26] The Kubernetes Authors, “Cluster architecture,” Accessed on Jun 18, 2025. [Online]. Available: https://kubernetes.io/docs/concepts/architecture/.
[27] R. Kumar and M. C. Trivedi, “Networking analysis and performance comparison of Kubernetes CNI plugins,” in Advances in Computer, Communication and Computational Sciences: Proceedings of IC4S 2019, pp. 99-109, 2021, doi: 10.1007/978-981-15-4409-5_9.
[28] C. Carrión, “Kubernetes scheduling: taxonomy, ongoing issues and challenges,” ACM Computing Surveys, vol. 55, no. 7, pp. 1-37, 2022, doi: 10.1145/3539606.
[29] G. Budigiri, C. Baumann, E. Truyen, and W. Joosen, “Elastic cross-layer orchestration of network policies in the Kubernetes stack,” IEEE Transactions on Network and Service Management, vol. 22, no. 2, pp. 2031-2058, 2025, doi: 10.1145/3539606.
[30] M. S. I. Shamim, F. A. Bhuiyan, and A. Rahman, “Xi commandments of kubernetes security: a systematization of knowledge related to kubernetes security practices,” 2020 IEEE Secure Development (SecDev), pp. 58-64, 2020, doi: 10.1109/SecDev45635.2020.00025.
[31] R. K. Malviya, “Securing Kubernetes for enterprise-scale deployments: challenges, best practices, and future directions,” Best Practices, and Future Directions, 2024, doi: 10.2139/ssrn.5055443.
[32] C. Canella, M. Werner, D. Gruss, and M. Schwarz, “Automating seccomp filter generation for linux applications,” in Proceedings of the 2021 on Cloud Computing Security Workshop, pp. 139-151, 2021, doi: 10.1145/3474123.3486762.
[33] Y.-C. Yu, C.-Y. Hung, and L.-D. Chou, “Kernel-level hidden rootkit detection based on eBPF,” Computers & Security, p. 104582, 2025, doi: 10.1016/j.cose.2025.104582.
[34] B. Gajbhiye, O. Goel, and P. K. Gopalakrishna Pandian, “Managing vulnerabilities in containerized and Kubernetes environments,” Available at SSRN 4982847, 2024, doi: 10.36676/jqst.v1.i2.16.
[35] C. Smiliotopoulos, G. Kambourakis, and C. Kolias, “Detecting lateral movement: A systematic survey,” Heliyon, vol. 10, no. 4, 2024, doi: 10.1016/j.heliyon.2024.e26317.
[36] S. Kampa, “Navigating the landscape of Kubernetes security threats and challenges,” Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), vol. 3, no. 4, pp. 274-281, 2024, doi: 10.60087/jklst.v3.n4.p274.
[37] N. Kodakandla, “Securing cloud-Native infrastructure with zero trust architecture,” Journal of Current Science and Research Review, vol. 2, no. 02, pp. 18-28, 2024.
[38] A. Tiwari, “Supervised learning: from theory to applications,” in Artificial intelligence and machine learning for EDGE computing, pp. 23-32, 2022, doi: 10.1016/B978-0-12-824054-0.00026-5.
[39] C.-C. Liu and L.-D. Chou, “5g/b5g network slice management via staged reinforcement learning,” IEEE Access, vol. 11, pp. 72272-72280, 2023, doi: 10.1109/ACCESS.2023.3294281.
[40] A. G. Barto, “Reinforcement learning: an introduction. by richard’s sutton,” SIAM Rev, vol. 6, no. 2, p. 423, 2021, doi: 10.1137/21N975254.
[41] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: a brief survey,” IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26-38, 2017, doi: 10.1109/MSP.2017.2743240.
[42] D. Dutta and S. R. Upreti, “A survey and comparative evaluation of actor‐critic methods in process control,” The Canadian Journal of Chemical Engineering, vol. 100, no. 9, pp. 2028-2056, 2022.
[43] J. Fan, Z. Wang, Y. Xie, and Z. Yang, “A theoretical analysis of deep Q-learning,” in Learning for dynamics and control, 2020: PMLR, pp. 486-489.
[44] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning, pp. 1889-1897, 2015.
[45] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:2505.23805, 2017, doi: 10.48550/arXiv.1707.06347.
[46] Stable Baselines3, “Recurrent PPO,” Accessed on Jun 18, 2025. [Online]. Available: https://sb3-contrib.readthedocs.io/en/master/modules/ppo_recurrent.html
[47] M. J. Hausknecht and P. Stone, “Deep recurrent Q-learning for partially observable MDPs,” in AAAI fall symposia, vol. 45, p. 141, 2015.
[48] R. Yu et al., "Reward Models in Deep Reinforcement Learning: A Survey," arXiv preprint arXiv:2506.15421, 2025.
[49] A. Gupta, A. Pacchiano, Y. Zhai, S. Kakade, and S. Levine, “Unpacking reward shaping: understanding the benefits of reward engineering on sample complexity,” Advances in Neural Information Processing Systems, vol. 35, pp. 15281-15295, 2022.
[50] X. Wang, Y. Chen, and W. Zhu, “A survey on curriculum learning,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 9, pp. 4555-4576, 2021, doi: 10.1109/TPAMI.2021.3069908.
[51] M. E. Taylor and P. Stone, “Transfer learning for reinforcement learning domains: A survey,” Journal of Machine Learning Research, vol. 10, no. 7, 2009.
[52] S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone, “Curriculum learning for reinforcement learning domains: a framework and survey,” Journal of Machine Learning Research, vol. 21, no. 181, pp. 1-50, 2020.
[53] W.-K. Ching and M. K. Ng, “Markov chains,” Models, algorithms and applications, vol. 650, pp. 111-139, 2006, doi: 10.1007/978-1-4614-6312-2.
[54] R. L. Tweedie, “Markov chains: structure and applications,” Handbook of statistics, vol. 19, pp. 817-851, 2001, doi: 10.1016/S0169-7161(01)19025-5.
[55] W. K. Ching, E. S. Fung, and M. K. Ng, “Higher‐order Markov chain models for categorical data sequences,” Naval Research Logistics (NRL), vol. 51, no. 4, pp. 557-574, 2004, doi: 10.1002/nav.20017.
[56] W.-K. Ching, M. K. Ng, and E. S. Fung, “Higher-order multivariate Markov chains and their applications,” Linear Algebra and its Applications, vol. 428, no. 2-3, pp. 492-507, 2008, doi: 10.1016/j.laa.2007.05.021.
[57] R. L. Tweedie, “Markov chains: structure and applications,” Handbook of statistics, vol. 19, pp. 817-851, 2001, doi: 10.1016/S0169-7161(01)19025-5.
[58] S. Bagheri, H. Kermabon-Bobinnec, S. Majumdar, Y. Jarraya, L. Wang, and M. Pourzandi, “Warping the defence timeline: non-disruptive proactive attack mitigation for kubernetes clusters,” in ICC 2023-IEEE International Conference on Communications, pp. 777-782, 2023, doi: 10.1109/ICC45041.2023.10278632.
[59] N. K. Kitson, A. C. Constantinou, Z. Guo, Y. Liu, and K. Chobtham, “A survey of bayesian network structure learning,” Artificial Intelligence Review, vol. 56, no. 8, pp. 8721-8814, 2023, doi: 10.1007/s10462-022-10351-w.
[60] A. Halinen, “Security risks for sidecar containers in Kubernetes,” Master’s thesis, Aalto University, Espoo, Finland, 2024.
[61] K. Poeng and L. Schumacher, “Lateral movement identification in cross-cloud deployment,” in 2024 20th International Conference on Network and Service Management (CNSM), pp. 1-4, 2024, doi: 10.23919/CNSM62983.2024.10814442.
[62] A. Sheriff, K. Huang, Z. Nemeth, and M. Nakhjiri, “ADA: automated moving target defense for AI workloads via ephemeral infrastructure-native rotation in Kubernetes,” arXiv preprint arXiv:2505.23805, 2025, doi: 10.48550/arXiv.2505.23805.
[63] G.-l. Cai, B.-s. Wang, W. Hu, and T.-z. Wang, “Moving target defense: state of the art and characteristics,” Frontiers of Information Technology & Electronic Engineering, vol. 17, no. 11, pp. 1122-1153, 2016, doi: 10.1631/FITEE.1601321.
[64] M. A. R. A. Amin, S. Shetty, L. Njilla, D. K. Tosh and C. Kamhoua, “Hidden markov model and cyber deception for the prevention of adversarial lateral movement,” in IEEE Access, vol. 9, pp. 49662-49682, 2021, doi: 10.1109/ACCESS.2021.3069105.
[65] M. van Haren, P. Van Aubel, H. Vranken, N. Keyaerts, and B. Sue, “Identification and prevention of lateral movement in Kubernetes,” Master’s thesis, Radboud University Nijmegen, Nijmegen, Netherlands, 2024.
[66] Madhu Akula, “Kubernetes goat,” Accessed on Jun 18, 2025. [Online]. Available: https://madhuakula.com/kubernetes-goat/.
[67] NumFOCUS Inc, “Pandas”, Accessed on Jun 18, 2025. [Online]. Available: https://pandas.pydata.org/
[68] NumPy, “Numpy”, Accessed on Jun 18, 2025. [Online]. Available: https://numpy.org/citing-numpy/
[69] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, "Stable-baselines3: reliable reinforcement learning implementations," Journal of machine learning research, vol. 22, no. 268, pp. 1-8, 2021.
[70] M. Towers et al., “Gymnasium: a standard interface for reinforcement learning environments,” arXiv preprint arXiv:2407.17032, 2024, doi: 10.48550/arXiv.2407.17032.
[71] Gulshan Kumar, “Evaluation metrics for intrusion detection systems-a study,” Evaluation, vol. 2, no. 11, pp. 11-7, 2014.
[72] Jay Beale, “Kubernetes penetration tool,” Accessed on Jun 18, 2025. [Online]. Available: https://www.inguardians.com/peirates/
[73] H. Lee, S. Kwon, and J.-H. Lee, “Experimental analysis of security attacks for Docker container communications,” Electronics, vol. 12, no. 4, p. 940, 2023, doi: 10.3390/electronics12040940.
[74] R. Ahmad, I. Alsmadi, W. Alhamdani, and L. a. Tawalbeh, “Zero-day attack detection: a systematic literature review,” Artificial Intelligence Review, vol. 56, no. 10, pp. 10733-10811, 2023.
[75] Shengnan Zhang, Yan Hu, and Guangrong Bian, “Research on string similarity algorithm based on levenshtein distance,” in 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), pp. 2247-2251, 2017, doi: 10.1109/IAEAC.2017.8054419.
[76] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu, “A comprehensive survey on graph neural networks,” IEEE transactions on neural networks and learning systems, vol. 32, no. 1, pp. 4-24, 2020, doi: 10.1109/TNNLS.2020.2978386.
[77] S. J. Johnson, M. R. Murty, and I. Navakanth, “A detailed review on word embedding techniques with emphasis on word2vec,” Multimedia Tools and Applications, vol. 83, no. 13, pp. 37979-38007, 2024, doi: 10.1007/s11042-023-17007-z.
[78] N. M. Gardazi, A. Daud, M. K. Malik, A. Bukhari, T. Alsahfi, and B. Alshemaimri, “BERT applications in natural language processing: a review,” Artificial Intelligence Review, vol. 58, no. 6, pp. 1-49, 2025, doi: 10.1007/s10462-025-11162-5.
[79] M. Hutsebaut-Buysse, K. Mets, and S. Latré, “Hierarchical reinforcement learning: A survey and open research challenges,” Machine Learning and Knowledge Extraction, vol. 4, no. 1, pp. 172-221, 2022.

指導教授

周立德(Li-Der Chou)

審核日期

2025-8-18

推文