在Kubernetes與Kafka上實作基於Consumer lag之微服務動態轉移機制;Consumer lag aware microservice relocation mechanism over Kubernetes and Kafka

NCU Institutional Repository > 資訊電機學院 > 軟體工程研究所 > 博碩士論文 > Item 987654321/83799

jsp.display-item.identifier=請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/83799

题名:	在Kubernetes與Kafka上實作基於Consumer lag之微服務動態轉移機制;Consumer lag aware microservice relocation mechanism over Kubernetes and Kafka
作者:	許家繻;Hsu, Chia-Ju
贡献者:	軟體工程研究所
关键词:	Kubernetes;Kafka;微服務;動態轉移;Kubernetes;Kafka;microservice;dynamic relocation
日期:	2020-08-17
上传时间:	2020-09-02 17:07:53 (UTC+8)
出版者:	國立中央大學
摘要:	近年來容器化技術發展迅速，許多軟體服務開始以容器和微服務的形式部署運行，而開源容器管理系統Kubernetes為容器自動部署和管理提供強大的功能，使容器部署變得更加彈性且可靠，許多雲端供應商也提供Kubernetes代管服務，使在雲端叢集中部署容器變得更加普遍。同時，一些物聯網和邊緣運算的應用情境也開始嘗試使用Kubernetes部署工作，這些工作可能會面臨邊緣端環境變化導致的性能下降、節點計算能力不均或工作量動態變化等問題導致工作量堆積。然而，Kubernetes的計算資源管理依賴管理者的估算，難以適應動態的變化。因此，本研究設計出一套Kubernetes擴充功能，以Kafka訊息串流平台作為容器之間的資料傳遞管道，透過監控其Consumer Lag以支援在Kubernetes上實現根據容器的工作進度對容器進行轉移，並設計轉移流程以保證轉移後不會有資料遺失或重複處理的狀況。;Container technology has developed rapidly in recent years, and many software services are deployed as containers and microservices. Kubernetes, an open-source container orchestration system that provides powerful features to automatically deploy and manage containerized applications, makes container deployment more flexible and reliable. So many cloud vendors provide managed Kubernetes service make it more popular to deploy containers on a cloud cluster. Some scenarios of the Internet of Things or edge computing also use Kubernetes to deploy their jobs. These jobs may face some problems, such as degradation of work node performance caused by edge environment variety, uneven computing power between work nodes, or dynamic workload. However, managing resources for containers in Kubernetes is rely on the manager’s estimation, and it is difficult to adjust to dynamic workload changes. Therefore, this thesis proposes a set of extension functions for Kubernetes, make it possible to relocate containers according to the work progress, which is achieved by using Kafka as message passing channel and monitoring Consumer Lag of each job. This thesis also proposes a relocation control flow to avoid data loss or data re-process after container relocation.
显示于类别:	[軟體工程研究所 ] 博碩士論文

文件中的档案:

档案	描述	大小	格式	浏览次数
index.html		0Kb	HTML	353	检视/开启

在NCUIR中所有的数据项都受到原著作权保护.

社群 sharing

数据加载中.....