相機變形校正晶片與系統的設計與實作

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：28

、訪客IP：3.133.149.165

姓名

姚敦凱(YAO, TUN-KAI) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

相機變形校正晶片與系統的設計與實作
(Design and Implementation of a Camera Distortion Correction System and Chip)

相關論文

★ 整合GRAFCET虛擬機器的智慧型控制器開發平台	★ 分散式工業電子看板網路系統設計與實作
★ 設計與實作一個基於雙攝影機視覺系統的雙點觸控螢幕	★ 智慧型機器人的嵌入式計算平台
★ 一個即時移動物偵測與追蹤的嵌入式系統	★ 一個固態硬碟的多處理器架構與分散式控制演算法
★ 基於立體視覺手勢辨識的人機互動系統	★ 整合仿生智慧行為控制的機器人系統晶片設計
★ 嵌入式無線影像感測網路的設計與實作	★ 以雙核心處理器為基礎之車牌辨識系統
★ 基於立體視覺的連續三維手勢辨識	★ 微型、超低功耗無線感測網路控制器設計與硬體實作
★ 串流影像之即時人臉偵測、追蹤與辨識─嵌入式系統設計	★ 一個快速立體視覺系統的嵌入式硬體設計
★ 即時連續影像接合系統設計與實作	★ 基於雙核心平台的嵌入式步態辨識系統

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

本論文提出一個使用輕量化的類神經相機變形模型與虛擬校正技術的高效率的
相機變形校正方法，針對低價與低品質的廣角鏡頭做精確的變形校正。有別於傳
統針對廣角變形的理想放射狀多項式模型。類神經逆映射方法使用非線性建模
的方式，同時擬合各種在低價相機上的變形，包括；多鏡片疊合、廣角變形、製
造工差等等光學變形。提出的方法使用一個輕型的單層前饋式類神經網路，並
使用倒傳遞錯誤修正的訓練方法，擬合複雜的相機變形曲面。透過最佳化過的變
形校正資訊，僅使用 4 個神經元的單隱藏層的神經網路，就能精確的映射變形影
像空間與校正影像空間的資訊。

所提出的類神經相機變形模型同時使用了兩種不同角度的低價廣角鏡頭來驗證
校正精確度，並比較傳統的數學廣角變形校正模型；而結果表示，使用提出的方
法校正 120 度廣角相機所取得的變形影像，校正誤差低於 2 像素，同時整張影像
的均方誤差僅有 0.2050。相較於傳統多項式方法能夠提升超過 429 倍的精確度。

並根據所提出的類神經方法的基礎上，提出一個硬體加速器架構；在 28nm FPGA
上，類神經加速器可以操作在超過 100Mhz 的時脈，可以校正超過 Full HD 解析
度的低品質 120 度廣角相機影像。而使用台積電 90 奈米製程合成所提出的硬體
架構，所提出的硬體架構可以工作在超過 300Mhz 的時脈。最後，使用整數運
算的晶片電路與理想使用浮點數的軟體程式的輸出誤差低於 10 的-6 次方；工作
在 240MHz 時，可以即時校正每秒 30 張 4K2K 解析度的影像。

摘要(英)

This study proposes a rapid neural network-based camera distortion correction
(NCDC), based on a lightweight neural network, to accurately correct the distortion of
low-cost cameras. The NCDC is different from general camera distortion models
because it uses a neural network to simultaneously model numerous camera
distortions, including multi-lens and wide-angle distortion, as well as various
manufacturing flaws, in a low-cost camera. The proposed NCDC uses a neural
network with an error backpropagation training algorithm to map the complex
distortion surface. The optimal number of neurons was assigned as 4 to associate the
mapping model between the distortion image space and the correction image space. In
offline calibration processing, the NCDC calculates the distortion vector from a single
captured calibration image without requiring an estimation of the optical center. Two
different wide-angle lenses use the proposed neuron-base method to correct the
distortion. Results show that the maximal corrected error in a whole image is less than
2 pixels with 120° wide-angle lens, and that the mean square error (MSE) approaches
0.2050 between the corrected and ideal results. The NCDC is 429x more accurate
than the traditional polynomial method.

Simultaneously, this study proposes a VLSI architecture bases on the NCDC, and
built a verification system of FPGA that consists of microprocessor, bus, memory,
GbE and wide-angle camera to really capture a distortion image, and correct the
camera distortions. The neural network-based corrector can correct an over
1920x1080 resolution image from the low-price 120° wide-angle camera. To obtain a
more accurate correction, the neural network-based correction method was
implemented using a 24-bit fixed point, after which the difference in error between
the floating point in the software and the fixed point in the hardware was under 10
-6
.
The NCDC chip is sized 1.51  1.51 mm
2
, and contains 126K gates built using TSMC
90 nm CMOS technology. Working at 240 Mhz, this architecture can correct 30
frames of 4K2K-resolution video per second.

關鍵字(中)

★ 廣角鏡頭
★ 變形校正
★ 相機

關鍵字(英)

★ wide-angle lens
★ distortion correction
★ camera

論文目次

III

Table of Contents
摘要................................................................................................................................. I
Abstract .......................................................................................................................... II
Table of Contents .......................................................................................................... III
List of Figure..................................................................................................................V
List of Table ................................................................................................................... X
Chapter 1. Introduction ............................................................................................ 1
1.1. Problematic ................................................................................................ 2
1.2. Acceleration for Real-Time ....................................................................... 3
1.3. System Bottleneck ..................................................................................... 5
1.4. Design Methodology .................................................................................. 7
1.5. Organization ............................................................................................... 9
Chapter 2. Traditional Camera Model ................................................................... 10
2.1. Camera Distortion Model ........................................................................ 10
2.2. Lens Distortion Model ............................................................................. 12
2.3. Virtual Calibration ................................................................................... 15
2.4. Interpolation ............................................................................................. 19
Chapter 3. Neural Network-Based Camera Distortion Correction ........................ 21
3.1. Neural Network ........................................................................................ 21
3.2. Neural Netwok-Based Camera Distortion Model .................................... 24
3.3. Neuron-Based Geometric Transformation ............................................... 35
3.4. Neural Network Acceleration .................................................................. 42
3.5. Tan-sigmoid Acceleration ........................................................................ 44
3.5.1. Exponential using CORDIC ........................................................ 45
3.5.2. Exponential using Modified CORDIC......................................... 48
3.5.3. Tan-sigmoid Hardware Arithmetic .............................................. 60
Chapter 4. Correction System ................................................................................ 62
4.1. Fast Integration Interpolation ................................................................... 62
4.2. NCDC System in a DE2-115 ................................................................... 70
4.3. NCDC System in a FAKA2-FPGA .......................................................... 81
Chapter 5. Experiment ........................................................................................... 90
5.1. Neural Network Training ......................................................................... 90
5.2. Correction Analytic .................................................................................. 91
5.3. Front-End Chip Measurement................................................................ 105
Chapter 6. Conclusion ......................................................................................... 119 Reference ................................................................................................................... 121

參考文獻

121

Reference
[1] R. Melo, J. P. Barreto, and G. Falcao, "A New Solution for Camera Calibration
and Real-Time Image Distortion Correction in Medical Endoscopy–Initial
Technical Evaluation," IEEE Transactions on Biomedical Engineering, vol. 59,
pp. 634-644, Mar. 2012.
[2] R. Y. Tsai, "An Efficient and Accurate Camera Calibration Technique for 3D
Machine Vision," in Proceedings of IEEE Conference on Computer Vision and
Pattern Recognition, Miami Beach, FL, 1986, pp. 364-374.
[3] A. Wang, T. Qiu, and L. Shao, "A Simple Method of Radial Distortion
Correction with Centre of Distortion Estimation," Journal of Mathematical
Imaging and Vision, vol. 35, pp. 165-172, Jul. 2009.
[4] J. Park, S. C. Byun, and B. U. Lee, "Lens Distortion Correction Using Ideal
Image Coordinates," IEEE Transactions on Consumer Electronics, vol. 55, pp.
987-991, Aug. 2009.
[5] M. Ahmed and A. Farag, "Nonmetric calibration of camera lens distortion:
Differential methods and robust estimation," IEEE Transactions on Image
Processing, vol. 14, pp. 1215-1230, Aug. 2005.
[6] Y. Wonpil, "An embedded camera lens distortion correction method for mobile
computing applications," IEEE Transactions on Consumer Electronics, vol. 49,
pp. 894-901, Nov. 2003.
[7] H. Hideaki, Y. Yutaka, and M. Yoichi, "A new method for distortion correction
of electronic endoscope images," IEEE Transactions on Medical Imaging, vol.
14, pp. 548-555, Sep. 1995.
[8] K. V. Asari, S. Kumar, and D. Radhakrishnan, "A new approach for nonlinear
distortion correction in endoscopic images based on least squares estimation,"
IEEE Transactions on Medical Imaging, vol. 18, pp. 345-354, Apr. 1999.
[9] S.-L. Chen, H.-Y. Huang, and C.-H. Luo, "Time Multiplexed VLSI
Architecture for Real-Time Barrel Distortion Correction in Video-Endoscopic
Images," IEEE Transactions on Circuits and Systems for Video Technology,
vol. 21, pp. 1612-1621, Nov. 2011.
[10] P. Y. Chen, C. C. Huang, Y. H. Shiau, and Y. T. Chen, "A VLSI
Implementation of Barrel Distortion Correction for Wide-Angle Camera
Images," IEEE Transactions on Circuits and Systems II-Express Briefs, vol. 56,
pp. 51-55, Jan. 2009.
[11] L. Qiang and N. M. Allinson, "FPGA Implementation of Pipelined 122

Architecture for Optical Imaging Distortion Correction," in IEEE Workshop on
Signal Processing Systems Design and Implementation, SIPS ’06. , Banff,
Canada, 2006, pp. 182-187.
[12] H. T. Ngo and V. K. Asari, "A pipelined architecture for real-time correction of
barrel distortion in wide-angle camera images," IEEE Transactions on Circuits
and Systems for Video Technology, vol. 15, pp. 436-444, Mar. 2005.
[13] K. V. Asari, "Design of an efficient VLSI architecture for non-linear spatial
warping of wide-angle camera images," Journal of Systems Architecture, vol.
50, pp. 743-755, Aug. 2004.
[14] D. C. Brown, "Close-range camera calibration," Photogramm. Eng. Remote
Sens., vol. 37, pp. 855-866, Jan. 1971.
[15] J. Weng, P. Cohen, and M. Herniou, "Camera calibration with distortion
models and accuracy evaluation," IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol. 14, pp. 965-980, Oct. 1992.
[16] H. Wang, G. Cao, H. Xu, and P. Wang, "Application of neural network on
distortion correction based of standard grid," in Mechatronics and Automation,
2009. ICMA 2009. International Conference on, 2009, pp. 2717-2722.
[17] J. Choi, S. H. Bang, and B. J. Sheu, "A programmable analog VLSI neural
network processor for communication receivers," IEEE Transactions on
Neural Networks, vol. 4, pp. 484-495, May 1993.
[18] S. Satyanarayana, Y. P. Tsividis, and H. P. Graf, "A reconfigurable VLSI neural
network," IEEE Journal of Solid-State Circuits, vol. 27, pp. 67-81, Jan. 1992.
[19] T. Orlowska-Kowalska and M. Kaminski, "FPGA Implementation of the
Multilayer Neural Network for the Speed Estimation of the Two-Mass Drive
System," IEEE Transactions on Industrial Informatics, vol. 7, pp. 436-445,
Aug. 2011.
[20] A. W. Savich, M. Moussa, and S. Areibi, "The Impact of Arithmetic
Representation on Implementing MLP-BP on FPGAs: A Study," IEEE
Transactions on Neural Networks, vol. 18, pp. 240-252, Jan. 2007.
[21] S. Hariprasath and T. N. Prabakar, "FPGA implementation of multilayer feed
forward neural network architecture using VHDL," in International
Conference on Computing, Communication and Applications, ICCCA ’12,
Dindigul, Tamilnadu, India, 2012, pp. 1-6.
[22] F. Zhou, J. Liu, Y. Yu, X. Tian, H. Liu, Y. Hao, et al., "Field-programmable
gate array implementation of a probabilistic neural network for motor cortical
decoding in rats," Journal of Neuroscience Methods, vol. 185, pp. 299-306,
Oct. 2010.
[23] X. P. Zhu and Y. W. Chen, "Improved FPGA implementation of Probabilistic 123

Neural Network for neural decoding," in International Conference on
Apperceiving Computing and Intelligence Analysis, ICACIA ’10, Chengdu,
China, 2010, pp. 198-202.
[24] C.-H. Chen, T.-K. Yao, and C.-M. Kuo, "Wide-Angle Camera Distortion
Correction Using Neural Back Mapping " in IEEE International Symposium
on Consumer Electronics, ISCE ’13, Hsinchu, Taiwan, 2013, pp. 171-172.
[25] C.-H. Chen, C.-M. Kuo, C.-Y. Chen, and J.-H. Dai, "The design and synthesis
using hierarchical robotic discrete-event modeling," Journal of Vibration and
Control, vol. 19, pp. 1603-1613, Jun. 2012.
[26] C.-H. Chen, T.-K. Yao, C.-M. Kuo, and C.-Y. Chen, "Evolutionary design of
constructive multilayer feedforward neural network," Journal of Vibration and
Control, Sept. 2012.
[27] C.-H. Chen, T.-K. Yao, J.-H. Dai, and C.-Y. Chen, "A pipelined multiprocessor
system-on-a-chip (SoC) design methodology for streaming signal processing,"
Journal of Vibration and Control, Oct. 2012.
[28] C.-H. Chen, C.-M. Kuo, S.-H. Hsieh, and C.-Y. Chen, "High efficient
very-large-scale integration (VLSI) implementation of probabilistic neural
network image interpolator," Journal of Vibration and Control, 2012.
[29] G. J. Colquhoun and R. W. Baines, "A generic IDEF0 model of process
planning," Int. J. Production Research, vol. 29, pp. 2239-2257, 1991.
[30] R. David, "Grafcet :A powerful tool for specification of logic controllers,"
IEEE Transactions on control systems technology, vol. 3, pp. 253-268, 1995.
[31] C. Synopsys. (2013). DesignWare Library. Available:
http://www.synopsys.com/ip/socinfrastructureip/designware/Pages/default.asp
x
[32] C. Altera. (2013). MegaCore IP Library. Available:
http://www.altera.com/download/megacore-ip/mip-index.jsp
[33] C. Digia. (2011). QT Project Wiki. Available: http://qt-project.org/wiki/
[34] R. Y. Tsai, "A versatile camera calibration technique for high-accuracy 3D
machine vision metrology using off-the-shelf TV cameras and lenses," IEEE
Journal of Robotics and Automation, vol. 3, pp. 323-344, Aug. 1987.
[35] H. T. Ngo and V. K. Asari, "Design of a High Performance Digital
Architecture for Real-Time Correction of Radial Lens Distortion," in Circuits
and Systems, 2006. MWSCAS ’06. 49th IEEE International Midwest
Symposium on, 2006, pp. 526-530.
[36] T. Rahman and N. Krouglicof, "An Efficient Camera Calibration Technique
Offering Robustness and Accuracy Over a Wide Range of Lens Distortion,"
Image Processing, IEEE Transactions on, vol. 21, pp. 626-637, 2012. [37] H. Kim, Y. Cha, and S. Kim, "Curvature Interpolation Method for Image
Zooming," Image Processing, IEEE Transactions on, vol. 20, pp. 1895-1903,
2011.
[38] X. Hu, R. G. Harber, and S. C. Bass, "Expanding the range of convergence of
the CORDIC algorithm," IEEE Transactions on Computers, vol. 40, pp. 13-21,
Jan. 1991.
[39] B. Gisutham, T. Srikanthan, and K. V. Asari, "A high speed flat CORDIC
based neuron with multi-level activation function for robust pattern
recognition," in IEEE International Workshop on Computer Architectures for
Machine Perception, Padova, Italy 2000, pp. 87-94.
[40] C. Altera. (2012, Feb.). Cyclone IV FPGA Device Family Overview (12.1 ed.).
Available: http://www.altera.com/literature/hb/cyclone-iv/cyiv-51001.pdf
[41] C. Altera. (2012, May ). Avalon Interface Specifications (11.0 ed.). Available:
http://www.altera.com/literature/manual/mnl_avalon_spec.pdf
[42] H. ARM. (2008). AMBA Open Specifications (2.0 ed.). Available:
http://www.arm.com/zh/products/system-ip/amba/amba-open-specifications.p
hp
[43] C. Aptina. (2012). MT9D111D00STC Data Sheet. Available:
http://www.aptina.com/products/soc/mt9d111d00stc/
[44] C. Aptina. (2012). MT9P014D00STC Data Sheet. Available:
http://www.aptina.com/products/image_sensors/mt9p014d00stc/
[45] C. Altera. (2013, Jan.). Triple-Speed Ethernet MegaCore Function User Guide
(12.1 ed.). Available: http://www.altera.com/literature/ug/ug_ethernet.pdf
[46] C. N. Instruments. (2012). LabVIEW. Available: http://www.ni.com/labview/
[47] R. O. Duda and P. E. Hart, "Use of the Hough transformation to detect lines
and curves in pictures," Communications of the ACM, vol. 15, pp. 11-15, 1972.
[48] J. Canny, "A Computational Approach to Edge Detection," Pattern Analysis
and Machine Intelligence, IEEE Transactions on, vol. PAMI-8, pp. 679-698,
1986.
[49] C. Altera. (2013, Feb.). SCFIFO and DCFIFO Megafunctions (12.1 ed.).
Available: http://www.altera.com/literature/ug/ug_fifo.pdf
[50] C. Altera. (2012, Feb.). Analyzing and Debugging Designs with the System
Console (12.1 ed.). Available:
http://www.altera.com/literature/hb/qts/qts_qii53028.pdf

指導教授

陳慶瀚(Chen, Ching-Han)

審核日期

2013-10-7

推文