基於卷積神經網路之光場顯示眼動追蹤模型

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：59

、訪客IP：18.188.107.47

姓名

朱冠宇(Guan-Yu Zhu) 查詢紙本館藏

畢業系所

光電科學與工程學系

論文名稱

基於卷積神經網路之光場顯示眼動追蹤模型
(CNN-Based Gaze Estimation for Light Field Display)

相關論文

★ 氮化鋁鎵深紫外光發光二極體高光效之封裝研究	★ 歐規之高對比度近遠燈設計與雜散光分析
★ 精準色彩取像與顯示系統之設計與製作	★ 符合多種道路路面需求之通用型路燈設計
★ 利用編碼孔徑之高亮度高光譜成像系統	★ 應用DMD提高幀率之數位光學相位共軛投影系統之研究
★ 應用四步相移解碼多階相位之消除碟片位移雜訊之研究	★ 費奈爾透鏡之光學效率與雜散光分析
★ 用於牙齒頻譜的多點量測之高光譜系統	★ 結合全像光學元件的微型化數位全像顯微鏡
★ 隨讀取位置改變之多頁繞射疊加訊號之相位誤差容忍度分析	★ 多波繞射疊加訊號法之參考光位置誤差分析
★ 使用方解石於數位全像顯微系統的深度測量系統	★ 陣列式燈具光學特性快速量測之研究
★ 使用透鏡陣列做為屏幕之數位光學相位共軛投影系統與適應性光學優化之研究	★ 使用體積全像光學波導之可變焦無透鏡數位全像顯微鏡

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-8-1以後開放)

摘要(中)

本論文旨在開發應用於頭戴式光場顯示的眼動追蹤模型。本模型以機器學習為基礎，以可見光的攝影機進行拍攝，利用捕捉到人眼的可見光影像作為輸入，經過神經網路得到對應的人眼凝視點作為輸出。
本模型是由兩種網路架構串連而成的，分別為特徵定位模型以及映射模型，其中特徵定位模型利用卷積神經網路(convolution neural network，簡稱CNN)提取RGB影像的特徵圖，再使用特徵圖計算人眼在影像中的對應座標X_e、Y_e，目前並沒有對應的資料庫能夠符合光場顯示的應用場域，因此我們設計了一套拍攝架構用於產生眼睛影像的資料庫；映射模型為全連接網路(fully connected network，簡稱FCN)架構，在每次眼動追蹤前紀錄一組校正影像，接著使用校正影像訓練映射模型的參數，訓練完成的映射模型能將眼睛(影像)座標X_e、Y_e轉換成凝視點(螢幕)座標X_g、Y_g，達到眼動追蹤的目的。
本研究的主要貢獻為(1)建立光場顯示的眼動追蹤資料庫、(2)開發應用於光場顯示的眼動追蹤模型、(3)利用RGB影像進行追蹤，不需要額外的光源。

摘要(英)

This study aims to develop an eye-tracking model for use in head-mounted light field displays. The model is based on machine learning and utilizes a visible light camera to capture images. It takes the captured visible light images of the human eye as input and employs a neural network to output the corresponding gaze point.
The model consists of two interconnected network architectures: the feature localization model and the mapping model. The feature localization model utilizes a Convolutional Neural Network (CNN) to extract feature maps from RGB images. These feature maps are then used to compute the corresponding coordinates, X_e and Y_e, of the human eye in the image. Since there is currently no existing database that matches the application domain of light field displays, we designed a capture setup to generate a database of eye images.
The mapping model employs a Fully Connected Network (FCN) architecture. Before each eye-tracking session, a set of calibration images is recorded. The parameters of the mapping model are then trained using these calibration images. The trained mapping model can convert the eye (image) coordinates X_e and Y_e to gaze point (screen) coordinates X_g and Y_g, thereby achieving eye-tracking.
The main contributions of this research are as follows: (1) Establishing an eye-tracking database for light field displays, (2) Developing an eye-tracking model specifically designed for light field displays, and (3) Utilizing RGB images for tracking without the need for additional light sources.

關鍵字(中)

★ 眼動追蹤
★ 卷積神經網路
★ 光場

關鍵字(英)

論文目次

摘要 I
Abstract II
致謝 III
目錄 IV
圖目錄 VI
表目錄 IX
第一章緒論 1
1.1 引言 1
1.2 研究動機 2
第二章眼動追蹤之基本原理 4
2.1 眼動追蹤簡介 4
2.2 眼動追蹤方法 6
2.2.1 2D回歸法 6
2.2.2 3D模型法 8
2.2.3 交叉比率法 10
2.2.4 基於容貌的方法 11
2.2.5 基於形狀的方法 13
2.3 MR眼鏡相關技術 15
2.3.1 擴瞳技術 16
2.3.2 光場顯示技術 18
2.3.3 MR眼鏡之眼動追蹤 21
2.4 人工智慧簡介 23
2.5 機器學習 23
2.5.1 類神經網路 24
2.5.2 反向傳遞 26
2.6 卷積神經網絡 29
第三章實驗設計與流程 32
3.1 實驗設計 32
3.2 實驗流程 35
3.3 資料庫的建立 39
3.4 模型參數 42
第四章實驗結果 45
4.1 特徵定位模型 45
4.2 映射模型 49
第五章結論 68
參考文獻 69

參考文獻

1. S. Melumad, and M. T. Pham, “The Smartphone as a Pacifying Technology,” Journal of Consumer Research 47, 237-235 (2020).
2. IDC, “Global Shipments of AR/VR Headsets Decline in 2022 While Rebounding in 2023,” https://www.idc.com/promo/arvr.
3. Proven Reality, “applications of augmented reality : 8 key industries to consider,” https://provenreality.com/augmented-reality/applications-of-augmented-reality-8-key-industries-to-consider/.
4. T. Armstrong and B. O. Olatunji, “Eye tracking of attention in the affective disorders: A meta-analytic review and synthesis,” Clinical Psychology Review 32(8), 704-723 (2012).
5. J. Z. Lim, J. Mountstephens, and J. Teo, “Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges,” Sensors 20(8), 2384 (2020).
6. G. Roper-Hall, “Louis Émile Javal (1839–1907): The Father of Orthoptics,” American Orthoptic Journal 57(1), 131-136 (2017).
7. M. PŁUŻYCZKA, “The First Hundred Years: a History of Eye Tracking as a Research Method,” Applied Linguistics Papers 25(4), 101-106 (2018).
8. A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn, “Towards foveated rendering for gaze-tracked virtual reality,” ACM Trans. Graph. 35(6), 1-12 (2016).
9. A. Kar and P. Corcoran, “A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms,” IEEE Access 5, 16495-16519 (2017).
10. P. Blignaut, ‘‘Mapping the pupil-glint vector to gaze coordinates in a simple video-based eye tracker,’’ J. Eye Movement Res. 7(1), 1–11 (2014).
11. K. Harezlak, P. Kasprowski, and M. Stasch, “Towards accurate eye tracker calibration – methods and procedures,” Procedia Computer Science 35, 1073 – 1081 (2014).
12. Z. R. Cherif, A. Nait-Ali, J. F. Motsch, and M. O. Krebs, ‘‘An adaptive calibration of an infrared light device used for gaze tracking,’’ Proc. 19th IEEE Instrum. Meas. Technol. Conf. 2, 1029-1033 (2002).
13. J. Wang, G. Zhang, and J. Shi, ‘‘2D gaze estimation based on pupil-glint vector using an artificial neural network,’’ Appl. Sci. 6(6), 174 (2016).
14. Z. Zhu and Q. Ji, ‘‘Eye and gaze tracking for interactive graphic display,’’ Mach. Vis. Appl. 15(3), 139-148 (2004).
15. C. Jian-Nan, Z. Chuang, Y. Yan-Tao, L. Yang, and Z. Han, ‘‘Eye gaze calculation based on nonlinear polynomial and generalized regression neural network,’’ Int. Conf. Natural Comput. 5, 617-623 (2009).
16. T. Nagamatsu, Y. Iwamoto, J. Kamahara, N. Tanaka, and M. Yamamoto, “Gaze estimation method based on an aspherical model of the cornea: surface of revolution about the optical axis of the eye,” in Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications (2010), 255-258.
17. F. L. Coutinho and C. H. Morimoto, “Free head motion eye gaze tracking using a single camera and multiple light sources,” Brazilian Symposium on Computer Graphics and Image Processing 19, 171-178 (2006).
18. I. Bacivarov, M. Ionita, and P. Corcoran, “Statistical models of appearance for eye tracking and eye-blink detection and measurement,” IEEE Transactions on Consumer Electronics 54(3), 1312-1320 (2008).
19. P. Koutras, and P. Maragos, “Estimation of eye gaze direction angles based on active appearance models,” in 2015 IEEE International Conference on Image Processing (ICIP)(IEEE2015), 2424-2428.
20. W. Wang, Y. Huang, and R. Zhang, “Driver gaze tracker using deformable template matching,” Proceedings of 2011 IEEE International Conference on Vehicular Electronics and Safety, 244-247 (2011).
21. M. Reinders, “Eye tracking by template matching using an automatic codebook generation scheme,” in Third Annual Conference of ASCI, ASCI, Delft (1997), 85-91.
22. S. Ramadan, W. Abd-Almageed, and C. E. Smith, “Eye Tracking Using Active Deformable Models,” in ICVGIP (2002).
23. I. F. Ince, and J. W. Kim, “A 2D eye gaze estimation system with low-resolution webcam images,” EURASIP Journal on Advances in Signal Processing 2011, 1-11 (2011).
24. L. Gabriel, “Épreuves réversibles donnant la sensation du relief,” J. Phys. Theor. Appl. 7(1), 821-825 (1908).
25. D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Trans. Graph. 32(6), 1-10 (2013).
26. K. Hong, J. Yeom, C. Jang, J. Hong, and B. Lee, “Full-color lens-array holographic optical element for three-dimensional optical see-through augmented reality,” Opt. Lett. 39, 127-130 (2014).
27. C. C. Sun, “Simplified model for diffraction analysis of volume holograms,” Opt. Eng. 42, 1184-1185 (2003).
28. O. I. Abiodun and et al., “State-of-the-art in artificial neural network applications: A survey,” Heliyon 4(11), e00938 (2018).
29. H. Simon, Neural networks and learning machines (Pearson Education India, 2009).
30. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by backpropagating errors,” Nature 323(6088), 533–536 (1986).
31. S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv preprint arXiv:1609.04747 (2016).
32. M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. De Freitas, “Learning to learn by gradient descent by gradient descent,” Advances in neural information processing systems 29 (2016).
33. C. Francois. Deep learning with Python (Simon and Schuster, 2021).
34. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” https://arxiv.org/abs/1409.1556.
35. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition (2015), 1-9.
36. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition (2014), 580-587.
37. K. He and et al., “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE transactions on pattern analysis and machine intelligence, 37(9), 1904-1916 (2015).
38. P. Rolet, M. Sebag, and O. Teytaud, “Integrated recognition, localization and detection using convolutional networks,” in Proceedings of the ECML Conference (2012), 1255-1263.
39. W. Wang, and J. Gang, “Application of convolutional neural network in natural language processing,” in 2018 international conference on information Systems and computer aided education (ICISCAE)(IEEE2018), 64-70.
40. T. Ishikawa, S. Baker, I. Matthews, and T. Kanade, “Passive driver gaze tracking with active appearance models,” in Proc. 11th World Congress on Intelligent Transportation Systems (2004).
41. K. Yamashiro, D. Deguchi, T. Takahashi, I. Ide, H. Murase, K. Higuchi, and T. Naito, “Automatic calibration of an in-vehicle gaze tracking system using driver′s typical gaze behavior,” in 2009 IEEE Intelligent Vehicles Symposium (IEEE2009), 998-1003.
42. H. C. Lee, D. T. Luong, C. W. Cho, E. C. Lee, and K. R. Park, “Gaze tracking system at a distance for controlling IPTV,” IEEE Trans. Consum. Electron. 56(4), 2577-2583 (2010).
43. J. W. Lee, C. W. Cho, K. Y. Shin, E. C. Lee, and K. R. Park, “3D gaze tracking method using Purkinje images on eye optical model and pupil,” Opt. Lasers Eng. 50(5), 736-751 (2012).
44. K. Takemura, K. Takahashi, J. Takamatsu, and T. Ogasawara, “Estimating 3-D point-of-regard in a real environment using a head-mounted eye-tracking system,” IEEE Trans. Human-Mach. Syst. 44(4), 531-536 (2014).
45. W. J. Ryan, A. T. Duchowski, and S. T. Birchfield, “Limbus/pupil switching for wearable eye tracking under variable lighting conditions,” in Proceedings of the 2008 symposium on Eye tracking research & applications (2008), 61-64.
46. M. Stengel, S. Grogorick, M. Eisemann, E. Eisemann, and M. A. Magnor, “An affordable solution for binocular eye tracking and calibration in head-mounted displays,” in Proceedings of the 23rd ACM international conference on Multimedia (2015), 15-24.

指導教授

余業緯

審核日期

2023-8-11

推文