在Kubernetes與Kafka上實作基於Consumer lag之微服務動態轉移機制;Consumer lag aware microservice relocation mechanism over Kubernetes and Kafka

NCUIR > College of Electrical Engineering & Computer Science > Software Engineer > Electronic Thesis & Dissertation > Item 987654321/83799

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/83799

Title:	在Kubernetes與Kafka上實作基於Consumer lag之微服務動態轉移機制;Consumer lag aware microservice relocation mechanism over Kubernetes and Kafka
Authors:	許家繻;Hsu, Chia-Ju
Contributors:	軟體工程研究所
Keywords:	Kubernetes;Kafka;微服務;動態轉移;Kubernetes;Kafka;microservice;dynamic relocation
Date:	2020-08-17
Issue Date:	2020-09-02 17:07:53 (UTC+8)
Publisher:	國立中央大學
Abstract:	近年來容器化技術發展迅速，許多軟體服務開始以容器和微服務的形式部署運行，而開源容器管理系統Kubernetes為容器自動部署和管理提供強大的功能，使容器部署變得更加彈性且可靠，許多雲端供應商也提供Kubernetes代管服務，使在雲端叢集中部署容器變得更加普遍。同時，一些物聯網和邊緣運算的應用情境也開始嘗試使用Kubernetes部署工作，這些工作可能會面臨邊緣端環境變化導致的性能下降、節點計算能力不均或工作量動態變化等問題導致工作量堆積。然而，Kubernetes的計算資源管理依賴管理者的估算，難以適應動態的變化。因此，本研究設計出一套Kubernetes擴充功能，以Kafka訊息串流平台作為容器之間的資料傳遞管道，透過監控其Consumer Lag以支援在Kubernetes上實現根據容器的工作進度對容器進行轉移，並設計轉移流程以保證轉移後不會有資料遺失或重複處理的狀況。;Container technology has developed rapidly in recent years, and many software services are deployed as containers and microservices. Kubernetes, an open-source container orchestration system that provides powerful features to automatically deploy and manage containerized applications, makes container deployment more flexible and reliable. So many cloud vendors provide managed Kubernetes service make it more popular to deploy containers on a cloud cluster. Some scenarios of the Internet of Things or edge computing also use Kubernetes to deploy their jobs. These jobs may face some problems, such as degradation of work node performance caused by edge environment variety, uneven computing power between work nodes, or dynamic workload. However, managing resources for containers in Kubernetes is rely on the manager’s estimation, and it is difficult to adjust to dynamic workload changes. Therefore, this thesis proposes a set of extension functions for Kubernetes, make it possible to relocate containers according to the work progress, which is achieved by using Kafka as message passing channel and monitoring Consumer Lag of each job. This thesis also proposes a relocation control flow to avoid data loss or data re-process after container relocation.
Appears in Collections:	[Software Engineer] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	250	View/Open

社群 sharing

Loading...