Multi-Stage Image Deraining on Embedded Devices

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：24

、訪客IP：18.218.78.102

姓名

陳柏穎(Bo-Ying Chen) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

(Multi-Stage Image Deraining on Embedded Devices)

相關論文

★ Dynamic Overlay Construction for Mobile Target Detection in Wireless Sensor Networks	★ 車輛導航的簡易繞路策略
★ 使用傳送端電壓改善定位	★ 利用車輛分類建構車載網路上的虛擬骨幹
★ Why Topology-based Broadcast Algorithms Do Not Work Well in Heterogeneous Wireless Networks?	★ 針對移動性目標物的有效率無線感測網路
★ 適用於無線隨意網路中以關節點為基礎的分散式拓樸控制方法	★ A Review of Existing Web Frameworks
★ 將感測網路切割成貪婪區塊的分散式演算法	★ 無線網路上Range-free的距離測量
★ Inferring Floor Plan from Trajectories	★ An Indoor Collaborative Pedestrian Dead Reckoning System
★ Dynamic Content Adjustment In Mobile Ad Hoc Networks	★ 以影像為基礎的定位系統
★ 大範圍無線感測網路下分散式資料壓縮收集演算法	★ 車用WiFi網路中的碰撞分析

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

圖像去雨對於自動駕駛系統非常重要，因為這些系統需要清晰的圖像來計算駕駛狀況，例如先進駕駛輔助系統(ADAS)。近期，已經有許多關於圖像去雨的研究工作。但是這些研究都沒有關注影像去雨系統的推理速度。此外，其中大多數的研究都使用合成的下雨圖像來訓練模型，這會造成對真實下雨圖像的泛化性較低。為了解決模型的推理速度，我們提出了輕量級通道注意模塊，並將火模塊引入我們的模型之中，來減少模型中的參數數量。此外，基於相同目的，我們也刪除了一些卷積核過濾器。最後，儘管我們的模型生成的圖像品質較低，但我們部屬在NVIDIA AGX Xavier開發者套件上的模型比基準模型快2.74倍。為了解決模型泛化問題，我們在訓練集中混合了真實和合成的下雨圖像。當我們的模型部署在 NVIDIA 的下一代嵌入式設備 Jetson AGX Orin 開發者套件上時，我們希望我們的模型能夠即時處理去雨（即超過10 FPS）。

摘要(英)

Image deraining is essential for autonomous driving systems which may require
clear images to calculate driving situations, such as ADAS. Recently, many research
works have been proposed for image deraining. However, none of them focuses on
the inference speed. In addition, most of them use synthetic raining images to
train their models, which leads to lower generalization for real raining images. To
address the inference speed, we propose Lightweight Channel Attention Block and
introduce fire modules into our model to reduce the number of parameters in the
model. In addition, several kernel filters are removed from subnetworks for the same
purpose. At the end, although our model generates deraining images with slightly
lower quality, our model deployed on NVIDIA AGX Xavier Developer Kit is 2.74
times faster than the baseline model. To address the model generalization issue,
we mix the real and synthetic raining images in the training set. We expect that
the our model can deal with deraining in real-time (i.e., exceeding 10 FPS) when
our model is deployed on the next generation embedded device of NVIDIA, Jetson
AGX Orin Developer Kit.

關鍵字(中)

★ 圖像去雨
★ 嵌入式設備
★ 多階段

關鍵字(英)

★ image deraining
★ embedded device
★ multi-stage

論文目次

1 Introduction 1
2 RelatedWork 4
2.1 Video Deraining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Single-Image Deraining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 Preliminary 7
3.1 Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2 Encoder-Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2.1 Squeeze U-Net . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.3 Attention Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3.1 Channel Attention Block . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4 Depthwise Separable Convolution . . . . . . . . . . . . . . . . . . . . . . . 10
3.5 MPRNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.6 Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4 Design 16
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2 Lightweight Channel Attention Block . . . . . . . . . . . . . . . . . . . . . 18
4.3 Encoder and Decoder Subnetwork . . . . . . . . . . . . . . . . . . . . . . . 19
4.3.1 Fire Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.4 Reduce Channel Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.5 Synthetic and Real Raining Dataset . . . . . . . . . . . . . . . . . . . . . . 21
5 Performance Comparison 23
5.1 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.2 Experimental Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.2.1 Training Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.2.2 Testing Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.3 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.4 Experimental Results and Analysis . . . . . . . . . . . . . . . . . . . . . . 27
5.5 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.5.1 Variation of training dataset . . . . . . . . . . . . . . . . . . . . . . 30
5.5.2 Amount of parameters . . . . . . . . . . . . . . . . . . . . . . . . . 30
6 Conclusion 32
Reference 33

參考文獻

[1] Frames per second. https://en.wikipedia.org/wiki/Frame_rate. Accessed:
2022-09-22.
[2] Jetson agx orin developer kit. https://developer.nvidia.com/buy-jetson?
product=jetson_agx_orin&location=TW. Accessed: 2022-09-10.
[3] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolu-
tional encoder-decoder architecture for image segmentation. CoRR, abs/1511.00561,
2015.
[4] Nazanin Beheshti and Lennart Johnsson. Squeeze u-net: A memory and energy
eﬀicient image segmentation network. In Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition Workshops, pages 364–365, 2020.
[5] Jérémie Bossu, Nicolas Hautiere, and Jean-Philippe Tarel. Rain or snow detection in
image sequences through use of a histogram of orientation of streaks. International
journal of computer vision, 93(3):348–367, 2011.
[6] Pierre Charbonnier, Laure Blanc-Feraud, Gilles Aubert, and Michel Barlaud. Two
deterministic half-quadratic regularization algorithms for computed imaging. In Pro-
ceedings of 1st International Conference on Image Processing, volume 2, pages 168–
172. IEEE, 1994.
[7] Jie Chen, Cheen-Hau Tan, Junhui Hou, Lap-Pui Chau, and He Li. Robust video
content alignment and compensation for rain removal in a CNN framework. CoRR,
abs/1803.10433, 2018.
[8] François Chollet. Xception: Deep learning with depthwise separable convolutions.
In Proceedings of the IEEE conference on computer vision and pattern recognition,
pages 1251–1258, 2017.
[9] Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. Clear-
ing the skies: A deep network architecture for single-image rain removal. IEEE
Transactions on Image Processing, 26(6):2944–2956, 2017.
[10] Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley.
Removing rain from single images via a deep detail network. In Proceedings of the
IEEE conference on computer vision and pattern recognition, pages 3855–3863, 2017.
[11] Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley.
Removing rain from single images via a deep detail network. In 2017 IEEE Conference
on Computer Vision and Pattern Recognition (CVPR), pages 1715–1723, 2017.
[12] K. Garg and S.K. Nayar. Detection and removal of rain from videos. In Proceedings
of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition, 2004. CVPR 2004., volume 1, pages I–I, 2004.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into
rectifiers: Surpassing human-level performance on imagenet classification. CoRR,
abs/1502.01852, 2015.
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning
for image recognition. In Proceedings of the IEEE conference on computer vision and
pattern recognition, pages 770–778, 2016.
[15] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang,
Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Eﬀicient
convolutional neural networks for mobile vision applications. arXiv preprint
arXiv:1704.04861, 2017.
[16] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.
Densely connected convolutional networks. In Proceedings of the IEEE conference
on computer vision and pattern recognition, pages 4700–4708, 2017.
[17] Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J
Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer param-
eters and< 0.5 mb model size. arXiv preprint arXiv:1602.07360, 2016.
[18] Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Baojin Huang, Yimin Luo, Jiayi
Ma, and Junjun Jiang. Multi-scale progressive fusion network for single image de-
raining. In Proceedings of the IEEE/CVF conference on computer vision and pattern
recognition, pages 8346–8355, 2020.
[19] Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, Liang-Jian Deng, and Yao Wang.
A novel tensor-based video rain streaks removal approach via utilizing discrimina-
tively intrinsic priors. In 2017 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), pages 2818–2827, 2017.
[20] Behzad Kamgar-Parsi and Azriel Rosenfeld. Optimally isotropic laplacian operator.
IEEE Transactions on Image Processing, 8(10):1467–1472, 1999.
[21] Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu. Automatic single-image-based rain
streaks removal via image decomposition. IEEE Transactions on Image Processing,
21(4):1742–1755, 2012.
[22] Jin-Hwan Kim, Chul Lee, Jae-Young Sim, and Chang-Su Kim. Single-image deraining
using an adaptive nonlocal means filter. In 2013 IEEE International Conference on
Image Processing, pages 914–917, 2013.
[23] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.
arXiv preprint arXiv:1412.6980, 2014.
[24] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification
with deep convolutional neural networks. Advances in neural information processing
systems, 25, 2012.
[25] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based
learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–
2324, 1998.
[26] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee.
Enhanced deep residual networks for single image super-resolution. CoRR,
abs/1707.02921, 2017.
[27] Jiaying Liu, Wenhan Yang, Shuai Yang, and Zongming Guo. D3r-net: Dynamic
routing residue recurrent network for video rain removal. IEEE Transactions on
Image Processing, 28(2):699–712, 2019.
[28] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm
restarts. arXiv preprint arXiv:1608.03983, 2016.
[29] Yu Luo, Yong Xu, and Hui Ji. Removing rain from a single image via discriminative
sparse coding. In 2015 IEEE International Conference on Computer Vision (ICCV),
pages 3397–3405, 2015.
[30] James MacQueen et al. Some methods for classification and analysis of multivari-
ate observations. In Proceedings of the fifth Berkeley symposium on mathematical
statistics and probability, volume 1, pages 281–297. Oakland, CA, USA, 1967.
[31] Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual
attention. Advances in neural information processing systems, 27, 2014.
[32] Wan-Joo Park and Kwae-Hi Lee. Rain removal using kalman filter in video. In 2008
International Conference on Smart Manufacturing Application, pages 494–497, 2008.
[33] Soo-Chang Pei, Yu-Tai Tsai, and Chen-Yu Lee. Removing rain and snow in a single
image using saturation and visibility features. In 2014 IEEE International Conference
on Multimedia and Expo Workshops (ICMEW), pages 1–6, 2014.
[34] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional net-
works for biomedical image segmentation. In International Conference on Medical
image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
[35] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional net-
works for biomedical image segmentation. CoRR, abs/1505.04597, 2015.
[36] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-
scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[37] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir
Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going
deeper with convolutions. In Proceedings of the IEEE conference on computer vision
and pattern recognition, pages 1–9, 2015.
[38] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N
Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in
neural information processing systems, 30, 2017.
[39] Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson W. H.
Lau. Spatial attentive single-image deraining with a high quality real rain dataset.
CoRR, abs/1904.01538, 2019.
[40] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Con-
volutional block attention module. In Proceedings of the European conference on
computer vision (ECCV), pages 3–19, 2018.
[41] Jing Xu, Wei Zhao, Peng Liu, and Xianglong Tang. An improved guidance image
based method to remove rain and snow in a single image. Computer and Information
Science, 5(3):49, 2012.
[42] Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and
Shuicheng Yan. Joint rain detection and removal via iterative region dependent
multi-task learning. CoRR, abs/1609.07769, 2016.
[43] Syed Waqas Zamir, Aditya Arora, Salman H. Khan, Munawar Hayat, Fahad Shahbaz
Khan, Ming-Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration.
CoRR, abs/2102.02808, 2021.
[44] Albert Zeyer, Kazuki Irie, Ralf Schlüter, and Hermann Ney. Improved training of
end-to-end attention models for speech recognition. arXiv preprint arXiv:1805.03294,
2018.
[45] Xiaopeng Zhang, Hao Li, Yingyi Qi, Wee Kheng Leow, and Teck Khim Ng. Rain
removal in video by combining temporal and chromatic properties. In 2006 IEEE
International Conference on Multimedia and Expo, pages 461–464, 2006.
[46] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image
super-resolution using very deep residual channel attention networks. In Proceedings
of the European conference on computer vision (ECCV), pages 286–301, 2018.

指導教授

孫敏德(Min-Te Sun)

審核日期

2022-9-28

推文