Combine Internal Test Oracles and Capture/Replay GUI Testing for Precise Replay and Higher Validation Power

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/61026

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/61026

Title:	Combine Internal Test Oracles and Capture/Replay GUI Testing for Precise Replay and Higher Validation Power
Authors:	潘韋成;Pan,Wei-Cheng
Contributors:	資訊工程學系
Keywords:	測試自動化;錄製重播;圖形介面測試;Test Automation;Capture Replay;GUI Testing
Date:	2013-07-24
Issue Date:	2013-08-22 12:10:06 (UTC+8)
Publisher:	國立中央大學
Abstract:	由於圖形使用者界面（Graphical User Interface, GUI）應用程式的種類非常複雜多變，為它們實施測試自動化相當困難，而有關GUI應用程式的測試方法之研究已經行之有年，其中較廣為使用的即是Capture-and-Replay方法。Capture-and-Replay方法可分為非侵入式與侵入式兩種。非侵入式方法是以模擬作業系統的操作資訊記錄操作過程，因此不需要修改待測程式；而侵入式方法則相反，是在待測程式內部插入額外程式碼記錄操作訊息。非侵入式的實作方法非常適合錄製與重現視窗操作，MIT實驗室的Sikuli即屬此類。它能夠撰寫腳本找尋畫面上的視窗元件位置，並模擬滑鼠與鍵盤操作。但因為此作法通常需要在作業系統插入hook以攔截滑鼠與鍵盤等輸入事件，若操作內容有頻繁的拖曳事件則很容易影響到軟體的效能，導致重播失敗，甚至可能一開始就無法錄製。這個問題在近期逐漸興起的多點觸控行動裝置應用程式等需要大量拖曳操作的程式又更顯重要。若要避免效能上的損失，在待測程式內部插入程式碼記錄輸入流程則是另一種可行的方式，然而若需要插入大量程式碼，容易對開發人員造成負擔。此外，在重播過程中的test oracle也是測試自動化時相當重要，卻變化很大的環節。根據不同的狀況，測試系統可能需要偵測外觀變化，或是觀測內部變數狀態來判斷待測程式的正確性，然而非侵入式的作法無法得知待測程式的內部狀態，侵入式做法又剛好相反，很難感測GUI程式的外觀變化。最後，現今的GUI程式大多採事件驅動（event driven）設計，事件之間經常有先後關係，重播時不可隨意變更順序，但也有些事件是交由多執行緒之間同時觸發，重播時可或不可忽略順序的變動。此種不確定性又提高了測試系統的設計難度。本論文將會探討各種Capture-and-Replay在實際應用上的優缺點，並融合它們各自的優點，以Sikuli為主，插入程式碼為輔，建立一個盡可能通用的測試自動化工具及函式庫──xTitan。 In general, it is difficult to apply test automation to GUI(Graphical User Interface) applications, because they are often too complicated. Many testing approachs for GUI applications has been developed in these days, and one of the most widespread approach is Capture-and-Replay. Capture-and-Replay can be categorized into two approaches: intrusive approach and non-intrusive approach. Non-instrusive approach is very suitable for widget-based GUI, where the testing tool Sikuli[1] from MIT that is improved by this thesis is belonged to this category. Sikuli can detect widget positions from screen by giving clipped images and simulate mouse or keyboard actions to the widgets captured by these images. To archieve so, most non-intrusive approachs inject hooks into operating system’s DLLs so that the mouse and keyboard events can be intercepted independently from the system under test. However, this approach impacts the performance of SUT(software under testing) significantly, particularly in while users do continuous drag-and-drop operations. The impact causes the SUT to run slower and behave differently from normal runs, and then fail the replay of test cases. This problem is now more critical on mobile applications, because multi-touch drag and move become the common input events to these applications. To avoid such a problem, the intrusive approach in [2] provides an alternative, but its drawback is that it needs to be done by inserting code in the source code. Another key factor in capture-and-replay test automation is the test oracles. An automatic test run needs test oracle to detect anomalies of visual effect or internal. How to derive good test oracles for both cases remain a difficult issue. At last, most of GUI applications are event-driven, some events may have dependency and some events could be concurrently executed at same time. A test automation system should be able to distinguish when should wait for next event and when should tolerate event interleaving. In this thesis, all these problems are studied and addressed. A prototype tool, xTitan, is implemented to demonstrate the results of this study.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	668	View/Open

社群 sharing

Loading...