在電子元件影像上複製可控制變異瑕疵的深度學習系統;Deep learning system for reproducing variation -controllable defects on electronic-component images

NCU Institutional Repository > 資訊電機學院 > 資訊工程研究所 > 博碩士論文 > Item 987654321/92577

jsp.display-item.identifier=請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/92577

题名:	在電子元件影像上複製可控制變異瑕疵的深度學習系統;Deep learning system for reproducing variation -controllable defects on electronic-component images
作者:	范仲瑜;Fan, Zhong-Yu
贡献者:	資訊工程學系
关键词:	瑕疵轉移;可控制變異瑕疵
日期:	2023-07-25
上传时间:	2023-10-04 16:05:33 (UTC+8)
出版者:	國立中央大學
摘要:	深度學習 (deep learning) 技術已廣泛應用在各項領域中；近年來許多行業都引入此項技術來提升作業上的效率及精密度，特別是與影像結合應用的部份。在深度學習的訓練策略中，最為理想的方式就是監督式學習，但現實生活中資料集的正負樣本的比例經常不平衡，或是資料量不充足，導致監督式學習的效果不佳。早期人們會使用影像翻轉、旋轉等資料擴增的方式來填補資料不充足的問題，但是此作法容易造就出真實情況中不會發生或不合理的資料集，反而會誤導網路模型的學習方向。為了讓資料擴增能夠更貼近現實資料集的狀況，生成模型 (generative model) 將會是一項重要的技術之一。在瑕疵檢測任務當中，經常發生瑕疵樣本的資料量不足，所以通常網路模型會使用半監督式或非監督式學習的方法來進行訓練。因此我們將針對如何轉移瑕疵樣本進行相關研究，提出了瑕疵影像轉移的系統，在轉移瑕疵影像時，可以針對影像中的瑕疵部位進行調整。我們的網路模型改自於本實驗室優化過的 pix2pix 條件式生成對抗網路，為了更好掌控轉移影像的瑕疵部位，我們重新定義輸入端的條件向量，此條件向量第一維度的數值將會影響到瑕疵部位顏色亮暗的變化。我們所提出的瑕疵轉移網路 (defect reproducing GAN, DRPGAN)，主要的改進有：i. 為訓練階段條件向量的第一維度設計瑕疵區域亮暗變化演算法，在測試階段可以透過此條件向量第一維度的數值去掌控影像中瑕疵區域的亮暗變化；ii. 為測試階段條件向量的第一維度設計預設值演算法。實驗中我們主要使用鍵盤按鍵的影像來進行訓練與測試，鍵盤按鍵的影像共有 218 組影像，每組影像擁有非瑕疵影像、瑕疵影像與瑕疵遮罩，資料集裡的非瑕疵影像均透過人工影像編輯修復瑕疵影像所得。為了維持轉移影像整體的品質，我們的資料集不區分訓練集與測試集，將所有資料一同進行訓練，在測試階段僅專注在瑕疵區域上的變化。在鍵盤按鍵的影像當中，我們可以透過調整條件向量第一維度的數值達到控制影像中瑕疵區域的亮暗變化，再搭配瑕疵遮罩來提供位置與形狀，將期望的瑕疵轉移到非瑕疵影像上。針對自行設計條件向量第一維度的數值演算法，我們額外蒐集了不同類型的影像資料集進行訓練與測試。新類型資料集擁有 301 組影像，每組影像擁有非瑕疵影像、瑕疵影像、瑕疵遮罩，其非瑕疵影像是與瑕疵影像極其相似的背景。在測試結果中，我們可以觀察到不管是新資料集還是舊資料集，條件向量第一維度數值的調整皆能夠明確的調整瑕疵區域的亮暗變化。最後，我們將這些瑕疵轉移影像提供給現有的辨識器 EfficientNet-b0 作為訓練樣本，讓辨識器在測試階段時，能夠將七至八成的真實瑕疵樣本進行成功的分類，證實我們瑕疵轉移網路所轉移的瑕疵影像，可以作為瑕疵影像資料擴增的來源之一。 ;“Deep learning” is widely used in various application fields. In recent years, many industries have introduced this technology to improve efficiency and accuracy in operations, especially in the part of combining with images. In deep learning training strategies, the most ideal approach is supervised learning. However, in real life, the ratio of positive and negative samples in datasets is often unbalanced or the amount of data is insufficient, resulting in poor performance of supervised learning. In the early days, people used image flipping and rotation to expand data to solve the problem of insufficient data. However, this method is easy to create data sets that will not occur or are unreasonable in real situations, which will mislead the learning direction of network models. In order to make data expansion more close to the situation of real data sets, generative models will be one of the important technologies. In the task of defect detection, it often happens that there is not enough data for defective samples. Therefore, network models usually use semi supervised or unsupervised learning methods for training. Therefore, we will focus on how to reproduce defective samples and propose a system for reproducing defective images. When reproducing images, we can adjust the defective parts of images. Our network model is adapted from the optimized pix2pix conditional generative adversarial network in our laboratory. In order to better control the defective parts of the transferred image, we redefine the input conditional vector. The value of the first dimension of this conditional vector will affect the change in brightness of the defective part. The main improvements of our proposed defect transfer network (defect reproducing GAN, DRPGAN) are: i. The design of the brightness change algorithm for the first dimension of the conditional vector during the training phase, which can control the brightness change of the defective area in the image through the value of the first dimension of this conditional vector during the testing phase; ii. The design of a default value algorithm for the first dimension of the conditional vector during the testing phase. In our experiment, we mainly use keyboard image to train and test. There are 218 groups of keyboard images in total. Each group has non-defective images, defective images and defective masks. Non-defective images in dataset are obtained by manual image editing and repairing defective images. In order to maintain the overall quality of reproduced images, our dataset does not distinguish between training set and test set. All data are trained together and only focus on changes in defective areas during testing phase. In keyboard images, we can adjust the numerical value of the first dimension of condition vector to control brightness change of defective areas in images and use defect masks to provide location and shape to transfer expected defects to non defective images. For self-designed numerical value algorithm for condition vector’s first dimension, we also collected different types of image datasets for training and testing. The new type dataset has 301 groups of images with non-defective images similar to defective ones as background. In test results, we can observe that whether it is a new dataset or an old one, adjusting numerical value of first dimension can clearly adjust brightness change of defective areas. Finally, we provide these reproduced defect images to existing recognizer EfficientNet-b0 as training samples so that recognizer can successfully classify 70-80% real defect samples during testing phase. This confirms that defect images reproduced by our defect transfer network can be used as one source for expanding defect image datasets.
显示于类别:	[資訊工程研究所] 博碩士論文

文件中的档案:

档案	描述	大小	格式	浏览次数
index.html		0Kb	HTML	44	检视/开启

在NCUIR中所有的数据项都受到原著作权保护.

社群 sharing

数据加载中.....