運用硬體壓縮技術改善NCU MFTVM容錯系統回應時間

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：43

、訪客IP：18.117.154.177

姓名

陳冠丞(Guan-Cheng Chen) 查詢紙本館藏

畢業系所

軟體工程研究所

論文名稱

運用硬體壓縮技術改善NCU MFTVM容錯系統回應時間
(Improving Response Time of NCU MFTVM Fault-Tolerant System based on Hardware Compression Technology)

相關論文

★ 坵塊分離、合併檢測系統	★ 使用雲林、彰化地區航空影像建立水稻判釋模型之研究
★ 以伸展樹為基礎的Android Binder Driver	★ 應用增量式學習於多種農作物判釋之研究
★ 應用分類重建學習偵測航照圖幅中的新穎坵塊	★ 用於輔助工業零件辨識之尺寸估算系統
★ 使用無紋理之3D CAD工業零件模型結合長度檢測實現細粒度真實工業零件影像分類	★ 一個建立在平行工作系統上的動態全球計算平台
★ 用權重參照計數演算法執行主動物件垃圾收集	★ 一個動態負載平衡之最大可能性估算計算架構
★ 利用多項系統負載資訊進行動態P2P系統重組的策略研究	★ 基於Hadoop系統的雲端應用程式特徵擷取與計算監測架構
★ 適用於大型動態分散式系統的調適性計算模型	★ 一個提供彈性虛擬資料中心的雲端服務平台
★ 雲端彈性虛擬機房服務平台之資源控管中心	★ 一個適用於自動供應雲端系統的動態調適計算架構

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2026-9-1以後開放)

摘要(中)

隨著雲端運算與虛擬化技術的普及，利用虛擬機器提供服務，將多個不同伺服器或服務整合至一台實體機器上，以彈性分配和利用硬體資源，但相對於實體機器，虛擬機更易因硬體設備或人為因素發生故障，造成服務中斷，因此必須使用高可用性技術，確保其可用性，基於虛擬化技術的容錯系統能在虛擬機器因某些因素發生故障時保持虛擬機之運行狀態，使服務不中斷，提升系統可用性。本研究基於中央大學平行與分散計算實驗室開發的NCU MFTVM容錯系統上繼續進行開發。NCU MFTVM屬於持續同步架構，持續同步技術不間斷地將主要虛擬機的狀態同步到備援虛擬機上，可透過外部管理介面控制的容錯系統，並基於容錯系統的準則，保持兩者對外輸出的一致性。本研究在虛擬機容錯系統中加入Intel® QAT硬體壓縮功能，改善了容錯系統的壓縮機制，在虛擬機容錯系統在同步大量資料時，降低NCU MFTVM容錯系統的回應時間，提升容錯系統的可靠性以及可用性。

摘要(英)

As cloud computing and virtualization technologies become more widespread, utilizing virtual machines to provide services and consolidating multiple different servers or services onto a single physical machine allows for flexible allocation and utilization of hardware resources. However, compared to physical machines, virtual machines are more prone to failures due to hardware issues or human errors, leading to service corruption. Therefore, it is necessary to employ high availability technologies to ensure their availability. Fault-tolerant systems based on virtualization technology can maintain the operational state of virtual machines when they encounter failures, preventing service disruptions and enhancing system availability.
Our research is based on the NCU MFTVM fault-tolerant system, which belongs to a continuous checkpointing architecture and can be controlled through an external management interface. Continuous checkpointing ensures constant synchronization of the state from the primary virtual machine to the backup virtual machine, adhering to fault-tolerant system criteria to maintain output consistency across both VMs. Ours research incorporates Intel® QAT hardware compression feature into the virtual machine fault-tolerant system, improving the compression mechanism of NCU MFTVM fault-tolerant system and reducing the response time during the synchronization of large amounts of data, thereby enhancing both its reliability and availability.

關鍵字(中)

★ QEMU-KVM
★ 虛擬機器
★ 容錯系統
★ 持續同步
★ 壓縮
★ Intel® QuickAssist Technology

關鍵字(英)

★ QEMU-KVM
★ Virtual Machine
★ Fault-Tolerance
★ Continuous Checkpointing
★ Compression
★ Intel® QuickAssist Technology

論文目次

第一章緒論 1
1-1研究背景 1
1-2研究動機與問題 1
1-3論文貢獻 3
1-4論文架構 4
第二章背景知識與相關研究 5
2-1 QEMU-KVM 5
2-2容錯系統類型 5
2-2-1 Lock-Stepping 5
2-2-2 Continuous Checkpointing 6
2-2-3 Hybrid 6
2-3 Network Output Buffer time 7
2-4 NCU MFTVM 容錯系統 8
2-5壓縮技術 9
2-5-1 XBZRLE ( XOR binary RLE) 9
2-5-2 Zstd 10
2-5-3 Intel® QuickAssist Technology 10
2-6 相關研究 11
第三章系統架構 12
3-1 主要架構 12
3-1-1 Checkpointing and Messaging 12
3-1-2 Autopilot 13
3-1-3 Compression 13
3-2 Intel® QAT module 13
3-3 Intel® QAT used in NCU MFTVM 14
3-3-1 Intel® QAT 初始化 14
3-3-2 Intel® QAT的使用限制 15
3-4 NCU MFTVM Data Flow 17
3-4-1 Checkpoint模組初始化 17
3-4-2 Checkpointing Data Flow with Intel® QAT 17
3-5 NCU MFTVM Compress module 21
3-6 軟硬體壓縮切換機制 22
3-6-1動機與分析方法 22
3-6-2 Checkpoint time and dirty pages in video streaming 23
3-6-3 Checkpoint time and dirty pages in kernel compile 25
3-6-4 Mix Compressor module 26
第四章效能改進 28
4-1實驗環境 28
4-2實驗設計 30
4-3實驗結果 31
4-3-1 Kernel Compile 31
4-3-2 Video application 36
4-3-3 Fio 41
4-4壓縮器效益比較 45
4-4-1 Compressor comparison in kernel compile 45
4-4-2 Compressor comparison in video streaming 47
4-4-3 Compressor comparison in fio 49
第五章結論與未來研究方向 53
5-1結論 53
5-2未來研究方向 53
參考文獻 54

參考文獻

[1] Y.-L. Lee, S. N. Arizky, Y.-R. Chen, D. Liang, and W.-J. Wang, "High-Availability Computing Platform with Sensor Fault Resilience," Sensors, vol. 21, no. 2, p. 542, Jan. 2021.
[2] Mesbahi, M. R., Rahmani, A. M., & Hosseinzadeh, M., "Reliability and high availability in cloud computing environments: a reference roadmap, " Human-centric Computing and Information Sciences, vol. 8, pp. 1-31, 2018.
[3] P. Kumari and P. Kaur, "A survey of fault tolerance in cloud computing," Journal of King Saud University-Computer and Information Sciences, vol. 33, no. 10, pp. 1159-1176, 2021.
[4] Jiann-Wen Wang, "An Adaptive Continuous Checkpointing Fault-Tolerant Virtual Machine System based on QEMU-KVM with libvirt, " 2020. Master Thesis. National Central University.
[5] G. Singh and A. K. Singh, "Optimizing multi-VM migration by allocating transfer and compression rate using geometric programming," Simulation Modelling Practice and Theory, vol. 106, pp. 102201, 2021.
[6] P. Svard, J. Tordsson, E. Elmroth, and B. Hudzia, "Evaluation of delta compression techniques for efficient live migration of large virtual machines," in Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Newport Beach, CA, USA, pp. 111-120, 2011.
[7] F. Bellard, "QEMU, a fast and portable dynamic translator," in USENIX Annual Technical Conference, FREENIX Track, vol. 41, no. 46, pp. 10-5555, Apr. 2005.
[8] Tiwei Bie, Heqing Zhu, "VirtIO, " in: Heqing Zhu (Ed.), "Data Plane Development Kit (DPDK): A Software Optimization Guide to the User Space-Based Network Applications, " Boca Raton, 2020, pp. 229-250.
[9] Tiwei Bie, Heqing Zhu, "Vhost-User, " in: Heqing Zhu (Ed.), "Data Plane Development Kit (DPDK): A Software Optimization Guide to the User Space-Based Network Applications, " Boca Raton, 2020, pp. 251-262.
[10] A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori, "kvm: the Linux virtual machine monitor, " in Proceedings of the Linux Symposium, vol. 1, no. 8, pp. 225-230, July 2007.
[11] T. Bressoud and F. Schneider, "Hypervisor-Based Fault Tolerance," ACM Transactions on Computer Systems, vol. 14, pp. 80–107, Feb. 1996, doi: 10.1145/224056.224058.
[12] Y. Tamura, K. Sato, S. Kihara, and S. Moriai, "Kemari: Virtual machine synchronization for fault tolerance," in Proc. USENIX Annu. Tech. Conf. (Poster Session), June 2008, pp. 1-2.
[13] B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson, and A. Warfield, "Remus: High availability via asynchronous virtual machine replication," in Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation, April 2008, pp. 161-174.
[14] "Features/MicroCheckpointing - QEMU." https://wiki.qemu.org/Features/MicroCheckpointing (accessed Jun. 12, 2024)
[15] "Cuju: An Open Source Project for Virtualization-Based Fault Tolerance," GitHub. https://github.com/Cuju-ft/Cuju, 2019, (accessed Jun. 2024).
[16] Y. Dong, W. Ye, Y. Jiang, I. Pratt, S. Ma, J. Li, and H. Guan, "Colo: Coarse-grained lock-stepping virtual machines for non-stop service," in Proceedings of the 4th Annual Symposium on Cloud Computing, October 2013, pp. 1-16.
[17] "qemu git repository: docs/xbzrle.txt," GitHub. https://github.com/qemu/qemu
(accessed Jun. 15, 2024).
[18] Li, Liang Zhang, and Yang, "KVM live migration optimization," presented at the KVM Forum, Intel Open Source Center, Aug. 2015. [Online]. Available: http://events17.linuxfoundation.org/sites/events/files/slides/kvm%20live%20migraion%20optimization-kvm%20forum%202015.pdf. (Accessed: Jul. 15, 2024)
[19] "Zstandard - Fast real-time compression algorithm, " GitHub. https://github.com/facebook/zstd (accessed Jul. 15, 2024).
[20] Intel, "Intel® QuickAssist Technology Use Cases" Intel, [Online]. Available: https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/what-is-intel-qat.html. (Accessed: Jul. 10, 2024).
[21] J. Zhang, L. Liang, and D. Wang, "Optimizing VNF live migration via para-virtualization driver and QuickAssist technology," in 2017 IEEE International Conference on Communications (ICC), IEEE, 2017, pp. 1-6.
[22] X. Hu, F. Wang, W. Li, J. Li, and H. Guan, "QZFS: QAT Accelerated Compression in File System for Application Agnostic and Cost Efficient Data Storage," in 2019 USENIX Annual Technical Conference (USENIX ATC 19), Renton, WA, USA, 2019, pp. 163-176.
[23] Microsoft, "SQL Server 2022: Acceleration Offloading with Intel QuickAssist Technology (QAT) - Data Exposed," [Online]. Available:
https://learn.microsoft.com/en-us/shows/data-exposed/sql-server-2022-acceleration-offloading-with-intel-quickassist-technology-qat-data-exposed. (Accessed: Jun. 9, 2024).
[24] Ching Chi, "A Fault-Tolerant QEMU-KVM System with Block Replication based on QEMU Block Migration, " 2021. Master Thesis. National Central University.
[25] Phoronix-Test-Suite, "Phoronix Test Suite, " [Online]. Available:
https://www.phoronix-test-suite.com/. (Accessed: Jun. 17, 2024).
[26] Wei-Lun Dai, "Fault-Tolerant VM with Event-Driven Architecture and virtial hard disk write replication, " 2022. Master Thesis. National Central University.

指導教授

王尉任(Wei-Jen Wang)

審核日期

2024-7-18

推文