在網頁資訊擷取(Web Data Extraction)的領域之中，如何自動從各式各樣不同的網頁中擷取出資料的相關議題至今已被探討了十多年，然而由於網頁的內容與架構的複雜，現有的方法均有其限制之處，再加上大量網頁擷取的需求，使得網頁資訊擷取的研究仍有相當大的挑戰。 網頁資料擷取系統主要分成記錄層級(Record Level)和頁面層級(Page Level)兩大類別，雖然頁面層級相較於記錄層級能夠得到更完整的網頁資訊，但由於問題的複雜及實作的困難兩大瓶頸，使得頁面層級的議題鮮少被關注，其擷取的效能與效率都有改進的空間。另一方面，雖然許多頁面層級的擷取系統標榜免標記的訓練，但是對於測試網頁的運作並無太多著墨。 有鑑於此，在本篇論文當中，我們提出了一套學習概念的頁面層級擷取系統，針對大量網頁擷取的情況，我們只需對一部分的網頁進行非監督式的訓練，並且利用訓練出的綱要(Schema)，透過擷取程式驗證(Wrapper Verification)的機制來測試其他剩餘的網頁，並同時擷取出網頁資料。本論文的實驗顯示，在表列網頁(List page)的處理上，本系統產生的綱要都比過去的頁面層級系統要來得準確且資料擷取的效率也較快，擷取效率相較於過去針對每一個網頁都進行分析的系統架構要快上數十倍。 ;The problem of automatically extracting data from web pages has been studied more than ten years. However, existing researches have limitations due to high structural complexity in web pages. On the other hand, the necessity of extracting data from large amount of web pages make it a challenging task for researchers. Web data extraction can be classified into two categories based on the extraction targets, record-level task and page-level task. Although the web data extracted by page-level approach is more complete than record-level approach, very few researches focus on this task because of the difficulties and complexities in the problem, and there are still much to be desired on effectiveness and efficiency. On the other hands, previous page-level systems focus on how to achieve unsupervised training and pay less concern about how to extract data from testing pages by matching with a wrapper. In this paper, we propose a learning based architecture for page-level extraction systems. Given a large amount of web pages for data extraction, the system use part of the input pages for training the schema, and then extract data from the rest of the input pages through wrapper verification. In our experiments, our system works better than other page-level extraction systems in terms of schema accuracy and extraction efficiency for multi-record pages. Overall, the extraction efficiency is dozens of times higher than state-of-the-art unsupervised approaches that extract data page by page without learning scheme (wrapper verification).
