基於Kubernetes和Kafka的微服務高可用性機制

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：13

、訪客IP：3.16.147.87

姓名

曹恕查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於Kubernetes和Kafka的微服務高可用性機制
(High-Availability Mechanism for Microservice System based on Kubernetes and Kafka)

相關論文

★ 以伸展樹為基礎的Android Binder Driver	★ 應用增量式學習於多種農作物判釋之研究
★ 應用分類重建學習偵測航照圖幅中的新穎坵塊	★ 用於輔助工業零件辨識之尺寸估算系統
★ 使用無紋理之3D CAD工業零件模型結合長度檢測實現細粒度真實工業零件影像分類	★ 一個建立在平行工作系統上的動態全球計算平台
★ 用權重參照計數演算法執行主動物件垃圾收集	★ 一個動態負載平衡之最大可能性估算計算架構
★ 利用多項系統負載資訊進行動態P2P系統重組的策略研究	★ 基於Hadoop系統的雲端應用程式特徵擷取與計算監測架構
★ 適用於大型動態分散式系統的調適性計算模型	★ 一個提供彈性虛擬資料中心的雲端服務平台
★ 雲端彈性虛擬機房服務平台之資源控管中心	★ 一個適用於自動供應雲端系統的動態調適計算架構
★ 線性相關工作與非相關工作的探索式排程策略	★ 適用於大資料集高效率的分散式階層分群演算法

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-9-1以後開放)

摘要(中)

現今物聯網(IoT)、人工智慧(AI)、大數據等等技術的興起，對於數據處理、自動化的需求也越來越高，因此雲霧協作架構(Cloud Fog Collaboration)也成為其中不可或缺的技術。然而由於其中雲端以及邊緣端在計算環境有很大的差異性，這導致了服務需要在不同的平台上部署與轉移，但是傳統的部署需要在各個環境中進行調整，其中的耦合性導致了部署的難度上升，而微服務架構(Microservices)將應用拆分成多個獨立的輕量化服務，有低耦合度的特性，解決了上述難題。容器化技術則為微服務架構的實現提供了一個適合的方案。然而，透過容器化的形式將微服務部署於雲霧協作環境下，如何維持微服務的高可用性成為了一個重要的議題，雖然現今的容器編排工具如Kubernetes、Docker Swarm都提供了高可用性機制，而Kubernetes針對節點故障、服務故障、服務過載等高可用性機制常面臨問題皆有相關對應手段，但仍存在一些不足，無法很好地滿足微服務的高可用性，除此之外，微服務架構的無狀態(Stateless)特性也給高可用性機制帶來了一定挑戰。因此，本研究設計出一套系統架構，基於Kubernetes和Apache Kafka資料串流平台，透過偵測不同情境執行對應的解決手段，改善Kubernetes的高可用性機制，優化節點、服務故障時的服務恢復時間，並且再過載發生時執行自動擴展以恢復服務的QoS，並且在上述的處理手段中不會影響到原先的任務進度運行。

摘要(英)

The rise of technologies such as the Internet of Things (IoT), Artificial Intelligence (AI), and Big Data has led to increasing demands for data processing and automation. As a result, Cloud Fog Collaboration has become an indispensable technology. However, there is significant disparity in the computing environments between the cloud and the edge, leading to the need for services to be deployed and migrated across different platforms. Traditional deployment methods require adjustments in each environment, increasing the complexity of deployment due to coupling. Microservices architecture addresses these challenges by breaking down applications into multiple independent and lightweight services with low coupling. Containerization technology provides a suitable solution for implementing the Microservices architecture.Deploying Microservices in the Cloud Fog Collaboration environment through containerization raises an important issue of how to maintain high availability. Although container orchestration tools like Kubernetes and Docker Swarm provide high availability mechanisms for node failures, service failures, and service overloads, they still have some limitations. Additionally, the stateless nature of Microservices poses challenges to high availability mechanisms. Therefore, in this research, we design a system architecture based on Kubernetes and Apache Kafka data streaming platform. We detect different scenarios and execute corresponding solutions to improve Kubernetes′ high availability mechanism, optimize service recovery time during node and service failures, and perform automatic scaling during service overloads to restore Quality of Service (QoS).

關鍵字(中)

★ 雲霧協作
★ 微服務
★ Kubernetes
★ Kafka
★ 高可用性

關鍵字(英)

★ Cloud Fog Collaboration
★ Microservice
★ Kubernetes
★ Kafka
★ High Availability

論文目次

摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 vi
表目錄 vii
一、緒論 1
1-1 研究背景 1
1-2 研究動機與目的 2
1-3 論文架構 5
2-1 Kubernetes 6
2-1-1 Kubernetes原生Pod重啟機制 8
2-1-2 Kubernetes原生Pod轉移機制 8
2-2 Apache Kafka 8
2-2-3 Kafka Consumer Lag 10
2-3 Prometheus 11
2-4 MongoDB 12
三、相關研究 13
3-1 利用Kafka Message Queue令微服務間彼此溝通 13
3-2 微服務的Auto Scaling相關研究 14
3-3 Kubernetes高可用性機制的相關研究 14
四、機制與架構設計 16
4-1 系統架構 16
4-2 偵測機制設計 19
4-2-1 節點故障 19
4-2-2 服務故障 21
4-2-3 服務過載 24
五、實驗 27
5-1 實驗環境 27
5-1-1 硬體配置 27
5-1-2 任務定義及系統配置 28
5-1-3 前置準備 28
5-1-4 實驗情境 29
5-2 節點故障模擬 31
5-2-1 至邊緣節點 31
5-2-2 至雲端節點 33
5-3 服務故障模擬 35
5-4 任務過載模擬 37
5-4-1 長期過載 38
5-4-2 短期過載 41
5-5 結果分析 43
六、結論與未來研究方向 45
參考文獻 47

參考文獻

[1] J. Jiang, Z. Li, Y. Tian, and N. Al-Nabhan, "A review of techniques and methods for IoT applications in collaborative cloud-fog environment," Security and Communication Networks, vol. 2020, pp. 1-15, 2020.
[2] L. Qian, Z. Luo, Y. Du, and L. Guo, "Cloud computing: An overview," in Cloud Computing: First International Conference, CloudCom 2009, Beijing, China, December 1-4, 2009. Proceedings 1, 2009: Springer, pp. 626-631.
[3] K. Cao, Y. Liu, G. Meng, and Q. Sun, "An overview of edge computing research," IEEE Access, vol. 8, pp. 85714-85728, 2020.
[4] N. Alshuqayran, N. Ali, and R. Evans, "A systematic mapping study in microservice architecture," in 2016 IEEE 9th International Conference on Service-Oriented Computing and Applications (SOCA), 2016: IEEE, pp. 44-51.
[5] Z. Sofia, "Container technologies," Hypatia, vol. 15, no. 2, pp. 181-201, 2000.
[6] V. Singh and S. K. Peddoju, "Container-based microservice architecture for cloud applications," in 2017 International Conference on Computing, Communication and Automation (ICCCA), 2017: IEEE, pp. 847-852.
[7] W. Lloyd, S. Ramesh, S. Chinthalapati, L. Ly, and S. Pallickara, "Serverless computing: An investigation of factors influencing microservice performance," in 2018 IEEE International Conference on Cloud Engineering (IC2E), 2018: IEEE, pp. 159-169.
[8] L. A. Vayghan, M. A. Saied, M. Toeroe, and F. Khendek, "Deploying microservice-based applications with Kubernetes: Experiments and lessons learned," in 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), 2018: IEEE, pp. 970-973.
[9] N. Marathe, A. Gandhi, and J. M. Shah, "Docker swarm and Kubernetes in cloud computing environment," in 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), 2019: IEEE, pp. 179-184.
[10] A. Javed, K. Heljanko, A. Buda, and K. Främling, "CEFIoT: A fault-tolerant IoT architecture for edge and cloud," in 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), 2018: IEEE, pp. 813-818.
[11] "Kubernetes." https://kubernetes.io/docs/home/. (accessed 2023).
[12] V. Medel, R. Tolosana-Calasanz, J. Á. Bañares, U. Arronategui, and O. F. Rana, "Characterising resource management performance in Kubernetes," Computers & Electrical Engineering, vol. 68, pp. 286-297, 2018.
[13] H. Loo, A. Yeo, K. Yip, and T. Liu, "Live Pod Migration in Kubernetes," University of British Columbia, Vancouver, Canada. [Online]. Available at, 2018.
[14] "Apache Kafka." https://kafka.apache.org/ (accessed 2023).
[15] S. Ehrenstein, "Scalability benchmarking of Kafka Streams applications," Kiel University, 2020.
[16] "Prometheus." https://prometheus.io/ (accessed 2023).
[17] "MongoDB." https://www.mongodb.com/ (accessed 2023).
[18] K. Bantia, N. Govardhan, and M. Anala, "Inter-Service Communication among Microservices using Kafka Connect," in 2022 IEEE 13th International Conference on Software Engineering and Service Science (ICSESS), 2022: IEEE, pp. 43-47.
[19] Y. Wang, F. Zhang, and S. U. Khan, "HCA Operator: A Hybrid Cloud Auto-scaling Tooling for Microservice Workloads," in 2022 18th International Conference on Mobility, Sensing and Networking (MSN), 2022: IEEE, pp. 885-890.
[20] J. Kreps, N. Narkhede, and J. Rao, "Kafka: A distributed messaging system for log processing," in Proceedings of the NetDB, 2011, vol. 11, no. 2011: Athens, Greece, pp. 1-7.

指導教授

王尉任(Wei-Jen Wang)

審核日期

2023-7-31

推文