| 摘要: | 遙測衛星影像具有周期性及面積廣闊的觀測資訊，常用於分析地球各環境現象的發展及變化。近年來，隨著深度學習技術的進步，許多人工神經網路(Artificial Neural Networks, ANNs)被應用於各種遙測資料的處理，例如影像融合、特徵提取、土地利用和土地覆蓋分類。雖然人工神經網路的設計變得越加複雜以獲得更好的表現性能，但其背後的邏輯隱藏在多層的網路計算，難以被學者或使用者理解，因此常被視為「黑盒子」。為了解決這個問題，文獻提出可解釋人工智慧(Explainable Artificial Intelligence, XAI)方法來觀察人工神經網路黑盒子內的運作原理。因此，本研究的目的是採用 XAI 方法了解遙測影像的 ANN 模型其運作邏輯，並選擇土地覆蓋分類為主題來證明此構想。然而，現有針對卷積神經網路(Convolutional Neural Networks, CNNs)模型之XAI方法著重於辨識模型所採用的重要空間特徵，而光譜特徵亦為遙測影像分類的重要資訊，因此本研究針對現有XAI方法進行調整以辨識重要的光譜特徵。首先，本研究設計了一個深度學習網路從多光譜影像中萃取空間和光譜特徵並進行土地覆蓋分類。本研究使用的遙測影像來自 Sentinel-2 衛星的 EuroSAT 數據集，實現了90%以上的分類準確率。再調整Guided Backpropagation、Gradient-weighted Class Activation Mapping (Grad-CAM)和 Guided Grad-CAM 三種 XAI 方法的運作程序，進而不僅能辨識重要的空間特徵，也能區隔光譜特徵。本研究之分析結果包含(1)基於各光譜波段的視覺化顯著圖(saliency map)進行定性分析以解讀ANN模型判斷依據；(2)以顯著圖進行統計排序並分析每個地物類別分類時的各光譜波段重要性；(3)最終基於本研究所提XAI方法辨識所得之重要光譜波段組合，為每個地物類別重新訓練二元分類ANN模型以驗證其有效性。實驗結果表明，相比於以所有波段為輸入之模型，以XAI方法萃取出的光譜波段組合之模型可達相似的高準確率；甚至在一些類別的表現可提升1.5-7%準確率，代表XAI方法有能力抓取正確光譜資訊並基於此資訊獲得在遙測影像分類上更佳的表現。整體而言，本研究所提出之XAI方法除了原有的空間特徵，亦能識別分類時重要的光譜特徵，進而可分析理解遙測影像分類類神經網路的內部運作機制。;As satellite images provide periodical observations of a large area, Remote Sensing (RS) data can help analyze the developments of the Earth and its environmental variation. In recent years, with the advancement of deep learning technology, many Artificial Neural Networks (ANNs) have been proposed to support various remote sensing applications, such as image fusion, feature extraction, and Land Use and Land Cover (LULC) classification. However, the design of ANN models become more and more complex for better performance, ANNs are considered as “black boxes”, where their logics are hidden behind the scenes. To address this issue, Explainable Artificial Intelligence (XAI) methods were proposed to provide a peek into the black boxes. Therefore, the objective of this research is to understand the reasoning process of a RS ANN model with XAI methods, where the land cover classification is chosen to prove the concept. Existing XAI methods designed for Convolutional Neural Networks (CNNs) mainly focus on identifying important spatial features used in CNN models. However, spectral features are also important in RS image classification. Therefore, this research modifies existing XAI methods to extract important spectral features. As the first step, this research designs a deep learning network to retrieve spatial and spectral features from multi-spectral images and perform the land cover classification. The EuroSAT dataset from the Sentinel-2 satellite is applied in this research, where more than 90% classification accuracy is achieved. Afterward, this research modified three existing XAI methods including Guided Backpropagation, Gradient-weighted Class Activation Mapping (Grad-CAM), and Guided Grad-CAM to retrieve not only the spatial but also the spectral features learnt by the deep learning network. The analyses of results includes: (1) a qualitative analysis based on the visual saliency maps of each spectral band to interpret the reasoning basis of the deep learning network; (2) a quantitative analysis of ranking important spectral bands of each class based on the saliency maps; (3) finally, based on the important spectral bands identified by the proposed XAI methods, binary classification ANN models are trained for each class to verify the identified bands.
In summary, experimental results indicate that the models constructed with the identified spectral bands achieved similar accuracies compared to the model using all bands as inputs, where the classification accuracies for some classes even increased by 1.5-7%. Hence, XAI methods can capture the important spectral information in RS land cover classification and could help achieve better accuracy. Overall, the research demonstrates that besides the spatial features, the proposed XAI methods can also identify important spectral features for a better understanding to the underlying mechanisms of a RS land cover classification ANN model. |
