| dc.description.abstract | “Deep learning” is widely used in various application fields. In recent years, many industries have introduced this technology to improve efficiency and accuracy in operations, especially in the part of combining with images. In deep learning training strategies, the most ideal approach is supervised learning. However, in real life, the ratio of positive and negative samples in datasets is often unbalanced or the amount of data is insufficient, resulting in poor performance of supervised learning. In the early days, people used image flipping and rotation to expand data to solve the problem of insufficient data. However, this method is easy to create data sets that will not occur or are unreasonable in real situations, which will mislead the learning direction of network models. In order to make data expansion more close to the situation of real data sets, generative models will be one of the important technologies.
In the task of defect detection, it often happens that there is not enough data for defective samples. Therefore, network models usually use semisupervised or unsupervised learning methods for training. Therefore, we will focus on how to reproduce defective samples and propose a system for reproducing defective images. When reproducing images, we can adjust the defective parts of images.
Our network model is adapted from the optimized pix2pix conditional generative adversarial network in our laboratory. In order to better control the defective parts of the transferred image, we redefine the input conditional vector. The value of the first dimension of this conditional vector will affect the change in brightness of the defective part. The main improvements of our proposed defect transfer network (defect reproducing GAN, DRPGAN) are: i. The design of the brightness change algorithm for the first dimension of the conditional vector during the training phase, which can control the brightness change of the defective area in the image through the value of the first dimension of this conditional vector during the testing phase; ii. The design of a default value algorithm for the first dimension of the conditional vector during the testing phase.
In our experiment, we mainly use keyboard image to train and test. There are 218 groups of keyboard images in total. Each group has non-defective images, defective images and defective masks. Non-defective images in dataset are obtained by manual image editing and repairing defective images. In order to maintain the overall quality of reproduced images, our dataset does not distinguish between training set and test set. All data are trained together and only focus on changes in defective areas during testing phase. In keyboard images, we can adjust the numerical value of the first dimension of condition vector to control brightness change of defective areas in images and use defect masks to provide location and shape to transfer expected defects to nondefective images.
For self-designed numerical value algorithm for condition vector’s first dimension, we also collected different types of image datasets for training and testing. The new type dataset has 301 groups of images with non-defective images similar to defective ones as background. In test results, we can observe that whether it is a new dataset or an old one, adjusting numerical value of first dimension can clearly adjust brightness change of defective areas.
Finally, we provide these reproduced defect images to existing recognizer EfficientNet-b0 as training samples so that recognizer can successfully classify 70-80% real defect samples during testing phase. This confirms that defect images reproduced by our defect transfer network can be used as one source for expanding defect image datasets. | en_US |