利用權重標準分流二進位神經網路做邊緣計算之影像辨識;Weight Standardization Fractional Binary Neural Network (WSFracBNN) for Image Recognition in Edge Computing

NCUIR > College of Electrical Engineering & Computer Science > Graduate Institute of Computer Science and Information Engineering > Electronic Thesis & Dissertation > Item 987654321/95414

Please use this identifier to cite or link to this item: http://ir.lib.ncu.edu.tw/handle/987654321/95414

Title:	利用權重標準分流二進位神經網路做邊緣計算之影像辨識;Weight Standardization Fractional Binary Neural Network (WSFracBNN) for Image Recognition in Edge Computing
Authors:	梁字清;Liang, Zi-Qing
Contributors:	資訊工程學系
Keywords:	人工智慧;模型辨識;邊緣計算;深度學習;二進制神經網路;影像辨識;模型壓縮;網路量化;Artificial Intelligence;Model Recognition;Edge Computing;Deep Learning;Binary Neural Networks;Image Recognition;Model Compression;Network Quantization
Date:	2024-07-02
Issue Date:	2024-10-09 16:47:08 (UTC+8)
Publisher:	國立中央大學
Abstract:	現今的模型為了得到更好的準確率會將網路設計的更加龐大，模型的運算量也成指數增長，在這個情況下要應用於邊緣計算相當有難度。而Binary Neural Networks （BNNs）二進制神經網路是將卷積核(Filter)權重和激勵值量化至1位元（Bit）的模型，這種模型非常適合ARM、FPGA等小晶片或其他邊緣計算裝置，為了設計一個對邊緣計算裝置更友善的模型，如何降低模型浮點數運算量起著重要的作用。Batch normalization （BN）是二進制神經網路的重要工具，然而在卷積層被量化至1位元（Bit）的情況下，BN層的浮點數計算成本變得較為高昂，本論文透過移除模型的BN層來降低浮點數運算量，並加入Scaled Weight Standardization Convolution（WS-Conv）方法來避免無BN層後準確率大幅降低的問題，並透過一系列的優化方式提升模型的性能。具體來說我們的模型在沒有BN層的情況下仍使模型的計算成本及準確度保持著競爭力，再加入一系列訓練方法讓模型在Cifar-100的準確率仍高於Baseline 0.6%，而總運算量則只有Baseline的46%，其中在BOPs不變的情況下FLOPs降低至接近0，使其更適合FPGA等嵌入式平台。;In order to achieve better accuracy, modern models have become increasingly large, leading to an exponential increase in computational load, making it challenging to apply them to edge computing. Binary Neural Networks (BNNs) are models that quantize the filter weights and activations to 1-bit. These models are highly suitable for small chips like ARM, FPGA, and other edge computing devices. To design a model that is more friendly to edge computing devices, it is crucial to reduce the floating-point operations (FLOPs). Batch normalization (BN) is an essential tool for binary neural networks; however, when convolution layers are quantized to 1-bit, the floating-point computation cost of BN layers becomes significantly high. This thesis aims to reduce the floating-point operations by removing the BN layers from the model and introducing the Scaled Weight Standardization Convolution (WS-Conv) method to avoid the significant accuracy drop caused by the absence of BN layers, and to enhance the model performance through a series of optimizations. Specifically, our model maintains competitive computational cost and accuracy even without BN layers. Furthermore, by incorporating a series of training methods, the model′s accuracy on CIFAR-100 is 0.6% higher than the baseline, while the total computational load is only 46% of the baseline. With unchanged BOPs, the FLOPs are reduced to nearly zero, making it more suitable for embedded platforms like FPGA.
Appears in Collections:	[Graduate Institute of Computer Science and Information Engineering] Electronic Thesis & Dissertation

Files in This Item:

File	Description	Size	Format
index.html		0Kb	HTML	80	View/Open

社群 sharing

Loading...