| 摘要: | 近期，隨著 ChatGPT 的問世，大型語言模型的開發逐漸蓬勃發展，但現實世界的應用往往較為複雜，無法由單一的 agent 完成，因此開始有人探索如何透過多個 agent協作以解決更複雜的任務，並由此衍生出各種多 agent 對話框架。其中，AutoGen 作為Microsoft 開發的框架，受到廣泛關注，其特色為使用者可以輕鬆自定義 agent 及任務，並且支援 Python 的協程 (coroutine) 並行處理 I/O 密集型任務。
然而，協程本質上是運行在單執行緒上，因此當執行的任務需要大量的計算資源時，無法利用多核 CPU 的效能，實現真正的平行執行，例如，在處理大規模數據時，這類任務往往會將數據分批 (Batch) 進行處理，而 AutoGen 的架構意味著這些任務必須sequential 的執行，導致處理時間增加，限制了其在真實場景中的效能與擴展性。
此外，由於 AutoGen 的執行模式受限於單執行緒架構，整個系統無法擴展到多台計算資源。例如，現代分散式系統中，可以將不同任務分散到多台機器上同時執行以充分利用資源。但 AutoGen 無法實現這樣的處理，進一步限制了其處理複雜任務的能力以及應用範圍。
AutoGen 系統缺乏容錯能力。由於系統為單執行緒運行，一旦執行緒執行過程發生錯誤或未捕獲的例外錯誤，整個系統將立刻停止運行，導致運行中斷，缺乏穩定性和可用性，讓使用 LLM 服務所投入的資源和成本付諸流水。
因此，為了應對上述挑戰，本研究提出了 AsyncGen，一種基於分散式系統改進的多 agent 對話框架，將 AutoGen 和分散式系統框架 Ray 相結合，透過 Ray 的資源調度，將計算任務分散至多台節點進行平行處理。實驗結果顯示，經改進的框架得以在多台機器上同時運行，顯著降低了執行時間，提升了容錯性和擴展能力，展現了基於分散式系統擴展多 agent 框架的潛力與價值。;Recently, with the emergence of ChatGPT, the development of large language models has been flourishing. However, real-world applications are often more complex and cannot be han-
dled by a single agent alone. As a result, there has been growing interest in exploring how multiple agents can collaborate to solve more complex tasks, leading to the creation of various multi-agent conversation frameworks. Among them, AutoGen, developed by Microsoft, has garnered significant attention for allowing users to easily customize agents and tasks. It also supports Python coroutines, enabling concurrent handling of I/O-intensive tasks.
However, coroutines inherently run on a single thread. When tasks require substantial computational resources, they cannot leverage the performance of multi-core CPUs to achieve true parallel execution. For instance, in processing large-scale data, tasks are often divided into batches, but AutoGen’s architecture necessitates sequential execution of these tasks, resulting in increased processing time and limiting its performance and scalability in real-world scenarios.
Moreover, AutoGen’s execution model is constrained by its single-threaded architecture, preventing the system from scaling across multiple computing resources. In modern distributed systems, tasks can be distributed across multiple machines to maximize resource utilization.
However, AutoGen is unable to implement such processing, further restricting its ability to handle complex tasks and its range of applications. Additionally, the AutoGen system lacks fault tolerance. Since it operates on a single thread,
any errors or uncaptured exceptions during execution will cause the entire system to halt immediately. This results in interruptions, undermining its stability and reliability, and wasting the resources and costs invested in using LLM services.
To address these challenges, this study proposes AsyncGen, an improved multi-agent conversation framework based on distributed systems. By integrating AutoGen with the distributed system framework Ray, AsyncGen utilizes Ray’s resource scheduling to distribute computational tasks across multiple nodes for parallel processing. Experimental results show that the improved framework can operate simultaneously across multiple machines, significantly reducing execution time, enhancing fault tolerance, and improving scalability, demonstrating the potential and value of extending multi-agent frameworks using distributed systems. |
