結合點雲密度熵計算方法和運動回復結構在虛幻引擎中進行影像三維點雲模型及渲染重建

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：54

、訪客IP：13.59.181.246

姓名

鄧可梵(Ko-Fan Teng) 查詢紙本館藏

畢業系所

人工智慧國際碩士學位學程

論文名稱

結合點雲密度熵計算方法和運動回復結構在虛幻引擎中進行影像三維點雲模型及渲染重建
(Combining Point Cloud Density Entropy Calculation Methods with Structure from Motion for Image 3D Point Cloud Modeling and Rendering Reconstruction in Unreal Engine)

相關論文

★ 基於適應性徑向基神經網路與非奇異快速終端滑模控制結合線上延遲估測器應用於二軸機械臂運動軌跡精確控制	★ 新型三維光學影像量測系統之設計與控制
★ 新型雙紐線軌跡設計與進階控制實現壓電平台快速與精確定位	★ 基於深度座標卷積與自動編碼器給予行人實時路徑及終點位置精確預測
★ 修正式雙紐線軌跡結合自適應積分終端滑動模態控制與逆模型遲滯補償實現壓電平台精確追蹤	★ 以粒子群最佳化-倒傳遞類神經網路-比例積分微分控制器和影像金字塔轉換融合方法實現三維光學顯微影像系統
★ 以局部熵亂度分布與模板匹配方法結合自適應ORB特徵提取達成多影像精確拼接	★ 低扭矩機械手臂機構開發與脈寬調變進階控制器設計
★ 使用時域門控與梅森增益公式構建四埠夾具的散射參數表徵	★ 通過強化學習與積分滑模動量觀測器實現機器手臂的強健近佳PD控制策略
★ 基於類代理注意力特徵融合模型的聯合實體關係抽取方法	★ 新型修正式柵欄軌跡結合擴增狀態估測滑模回授與多自由度Bouc-Wen遲滯前饋補償控制器給予壓電平台快速精確追蹤

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2029-7-1以後開放)

摘要(中)

在當今的數位時代，三維掃描和重建技術在自動駕駛、精密工程和文化遺產保存等多個領域中變得至關重要，允許我們準確地創建物理對象的數位模型以進行分析和虛擬互動。然而，這些先進技術在實施過程中經常會受到各種雜訊的干擾，這些雜訊可能來自於影像特徵識別錯誤、環境條件或物體表面的複雜性，從而在3D重建過程中造成物體特徵的缺失，對物體特徵的重建和描述構成重大挑戰。
本研究提出一種基於運動恢復結構與熵計算相結合的創新三維點雲重建方法。通過在圖像處理階段引入熵計算，此方法有效地提升對極幾何的豐富度和點雲數量。此外，改進的密度熵束調整使點雲能夠更佳的豐富，展現更細緻紋理的三維點雲模型。最終，將所得的點雲模型整合到虛幻引擎中，使其具備即時更新和高度逼真的視覺效果，展示本研究在3D重建領域的潛在應用和創新價值。
實驗結果表明，此方法在點雲密度和細節豐富性方面顯著優於現有的SfM方法。儘管計算時間較長，但其生成的點雲模型在密度和細節上具有明顯的優勢，這對於需要詳細三維重建的應用尤其有利。通過將此方法應用於工業相機捕捉的圖像中，成功創建數位孿生模型，並將其導入虛幻引擎中，實現即時監控。不僅展示所提出方法的實際應用潛力，還強調其在數位孿生和虛擬現實領域的創新價值。

摘要(英)

In today’s digital era, 3D scanning and reconstruction technologies have become pivotal in various fields such as autonomous driving, precision engineering, and cultural heritage preservation, allowing us to accurately create digital model of physical objects for analysis and virtual interactions. However, these advanced technologies often encounter obstruction from various types of noise during implementation, which from misidentification of image features, environmental conditions, or the complexity of object surfaces, thereby causing loss of object features in the 3D reconstruction process and posing significant challenges to the reconstruction and description of object features. Therefore, developing efficient methods for noise identification and reduction becomes particularly important to enhance the accuracy and reliability of 3D scanning and reconstruction technologies.
This study proposes an innovative 3D point cloud reconstruction method based on Structure from Motion (SfM) and entropy calculation. By introducing entropy calculation in the image processing phase, this method effectively enhances the richness of epipolar geometry and point clouds numbers. Additionally, the improved density entropy Bundle Adjustment allows point clouds to be more detailed, presenting a more refined texture in the 3D point cloud model. Finally, the resulting point cloud model is integrated into the Unreal Engine, enabling real-time updates and highly realistic visual effects, displaying the potential applications and innovative value of this research in the field of 3D reconstruction.
Experimental results show that this method significantly outperforms existing SfM methods in terms of point cloud density and detail richness. Although the computation time is longer, the generated point cloud model has clear advantages in density and detail, which is especially beneficial for applications requiring detailed 3D reconstruction. By applying this method to images captured by industrial cameras, it successfully creates a digital twin model and integrates it into the Unreal Engine for real-time monitoring. This not only demonstrates the practical application potential of the proposed method but also highlights its innovative value in the fields of digital twins and virtual reality.

關鍵字(中)

★ 熵
★ 對極幾何
★ 點雲
★ 運動恢復結構
★ 虛幻引擎
★ 數位孿生

關鍵字(英)

論文目次

摘要 i
ABSTRACT ii
誌謝 iv
Table of content v
List of Figures vii
List of Tables ix
Explanation of Symbols x
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Literature Survey 3
1.2.1 Feature Extraction and Matching 3
1.2.2 Structure from Motion 7
1.2.3 Game Engine Integration 10
1.3 Contribution 11
1.4 Thesis Organization 12
Chapter 2 Preliminaries 14
2.1 Image Correspondence Computation 14
2.2 Concept of Point Cloud Projection 21
2.3 Unreal Engine 25
Chapter 3 Entropy based SfM Design Strategy 28
3.1 2D Image Processing 29
3.2 Epipolar Geometry 33
3.2.1 Fundamental Matrix 33
3.2.2 Essential Matrix 38
3.3 Triangulation 42
3.4 Entropy Bundle Adjustment 47
Chapter 4 Digital Twin Model in Unreal Engine 50
4.1 The Architecture of Digital Twin System 50
4.2 3D Model Combined with Unreal Engine 52
Chapter 5 Experimental Result 61
5.1 Comparison of Epipolar Geometry 61
5.2 Comparison of Density Entropy Optimization 63
5.3 Industrial Camera 68
Chapter 6 Conclusions 71
Reference 73

參考文獻

[1] Y. Yuan, Y. Ding, L. Zhao, and L. Lv, “An improved method of 3d scene reconstruction based on sfm,” International Conference on Robotics and Automation Engineering, Guangzhou, China, pp. 228-232, 2018.
[2] Y. Li and G. Baciu, “Multiscale point cloud optimization for sfm reconstruction,” IEEE International Conference on Cognitive Informatics and Cognitive Computing, Milan, Italy, pp. 439-445, 2019.
[3] J. Scho ̈nberger and J. Frahm, “Structure-from-motion revisited.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104-4113, 2016.
[4] E. Rosten and T. Drummond, “Machine learning for high-speed corner detection” European Conference on Computer Vision, pp. 430-443, 2006.
[5] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[6] P. F. Alcantarilla, A. Bartoli, and A. J. Davison, “Kaze features,” European Conference on Computer Vision, pp. 214-227, 2012.
[7] M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” International Conference on Computer Vision Theory and Applications, Lisboa, Portugal, February 5-8, 2009.
[8] P. Indyk and R. Motwani, “Approximate nearest neighbors: towards removing the curse of dimensionality,” Annual ACM Symposium on Theory of Computing, pp. 1-13,1998.
[9] C. Wu, “Towards linear-time incremental structure from motion,” International Conference on 3D Vision, pp. 127-134, 2013.
[10] P. Moulon, P. Monasse, R. Perrot, and R. Marlet. “Openmvg: open multiple view geometry.” International Workshop on Reproducible Research in Pattern Recognition, pp. 60-74, 2016.
[11] N. Snavely, S. M. Seitz, and R. Szeliski. “Photo tourism: exploring photo collections in 3D.” ACM transactions on graphics, vol. 25, pp. 835-846, 2006.
[12] Unity Technologies. (n.d.). Unity. Retrieved April 20, 2024, from https://unity.com/cn
[13] Epic Games. (n.d.). Unreal Engine. Retrieved April 20, 2024, from https://www.unrealengine.com/en
[14] C. E. Shannon, “A mathematical theory of communication,” Bell System Technical Journal, vol. 27, pp. 379-423, 1948.
[15] J. W. Wu, Y. C. Lin, K. F. Teng, C. A. Chou, Y. C. Yan, and H. K. Ye, “A new image stitching method with entropy and template matching,” IEEE International Conference on Consumer Electronics, pp. 1-6, 2024.
[16] B. Zhong and Y. Li, “Image feature point matching based on improved sift algorithm,” IEEE International Conference on Image, Vision and Computing, Xiamen, pp. 489-493, 2019.
[17] W. Peng, D. Yichen, D. Mianmian, X. Miaomiao, and L. Xue, “Vehicle image mosaics technology based on improved sift algorithm,” IEEE International Conference on Electronic Measurement and Instruments, pp. 516-520, 2017.
[18] A. Jakubović and J. Velagić, “Image feature matching and object detection using brute-force matchers,” International Symposium ELMAR, pp. 83-86, 2018.
[19] R. Renjith, R. Reshma, and K. V. Arun, “Design and implementation of traffic sign and obstacle detection in a self-driving car using surf detector and brute force matcher,” IEEE International Conference on Power, Control, Signals and Instrumentation Engineering, pp. 1985-1989, 2017.
[20] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, 381-395, 1981.
[21] P. Trivedi, T. Agarwal, and K. Muthunagai, “Mc-ransac: a pre-processing model for ransac using monte carlo method implemented on a gpu,” International Conference on Advances in Computing, Communications and Informatics, pp. 1380-1383, 2013.
[22] R. Shah, A. Deshpande, and P. J. Narayanan, “Multistage SFM: Revisiting Incremental Structure from Motion,” International Conference on 3D Vision, pp. 417-424, 2014.
[23] R. Hartley and A. Zisserman, “Multiple view geometry in computer vision,” 2nd ed. Cambridge, 2003.
[24] H. Pan, Y. Li, C. Gao, and Y. Lei, “A method for multi-cameras human tracking based on reverse epipolar line geometry,” IEEE Conference Anthology, pp. 1-4, 2013.
[25] L. Guodong, L. Sha, and L. Jianxun, “A stereo vision tracking algorithm based on 3-dimensional delaunay triangulation,” Chinese Control and Decision Conference, pp. 3779-3784, 2016.
[26] S. El Hazzat, A. Saaidi, and K. Satori, “Structure from motion for 3d object reconstruction based on local and global bundle adjustment,” Third World Conference on Complex Systems, pp. 1-6, 2015.
[27] Q. Demoulin, F. Lefebvre-Albaret, A. Basarab, D. Kouamé, and J. -Y. Tourneret, “Wing 3d reconstruction by constraining the bundle adjustment with mechanical limitations,” European Signal Processing Conference, pp. 570-574, 2021.
[28] A. Alouache, “Experimental validation of nonlinear optimization frameworks for solving bundle adjustment in structure from motion,” International Conference on Electrical Engineering, Computing Science and Automatic Control, pp. 1-5, 2022.
[29] F. Sang, H. Wu, Z. Liu, and S. Fang, “Digital twin platform design for zhejiang rural cultural tourism based on unreal engine,” International Conference on Culture-Oriented Science and Technology, pp. 274-278, 2022.
[30] M. Wang, F. Liu, X. Bi, J. Zhu, Z. Zhang, and S. Zhou, “Design and implementation of campus digital twin system based on game engine,” IEEE International Conference on Sensors, Electronics and Computer Engineering, pp. 1343-1349, 2023.
[31] L. Wei, “Research on digital twin city platform based on unreal engine,” International Conference on Information Processing and Network Provisioning, pp. 142-146, 2022.
[32] Unreal Engine. (n.d.). Getting started with digital twins. Unreal Engine. Retrieved April 28, 2024, from https://www.unrealengine.com/en-US/blog/getting-started-with-digital-twins.
[33] T. Zhang, S. Guo, L. Ma and W. Meng, “Hierarchical image retrieval method based on bag-of-visual-word and eight-point algorithm with feature clouds for visual indoor positioning,” Asia Pacific Conference on Communications, pp. 455-459, 2022.
[34] I. Enesi, A. Kuqi and A. Korra, “The background role in 3d object reconstruction,” Mediterranean Conference on Embedded Computing, pp. 1-7, 2023.
[35] M. Bailey and C. Law, “A summer blender camp: modeling, rendering, and animation for high school students,” IEEE Computer Graphics and Applications, pp. 65-67, 2014.
[36] Cubic. (n.d.). Cubic products. Retrieved June 17, 2024, from https://www.cubic-vs.com/products_lists/1_88/1.htm

指導教授

吳俊緯

審核日期

2024-7-23

推文