應用生成對抗網路於嬰兒骨架偵測與早產兒整體動作指標分析

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：56

、訪客IP：3.15.180.175

姓名

李妮蓁(Ni-Chen Lee) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

應用生成對抗網路於嬰兒骨架偵測與早產兒整體動作指標分析
(The Application of the Generative Adversarial Network in Detecting Body Poses of Infants and Assessing General Movements in Preterm Infants)

相關論文

★ 以Q-學習法為基礎之群體智慧演算法及其應用	★ 發展遲緩兒童之復健系統研製
★ 從認知風格角度比較教師評量與同儕互評之差異：從英語寫作到遊戲製作	★ 基於檢驗數值的糖尿病腎病變預測模型
★ 模糊類神經網路為架構之遙測影像分類器設計	★ 複合式群聚演算法
★ 身心障礙者輔具之研製	★ 指紋分類器之研究
★ 背光影像補償及色彩減量之研究	★ 類神經網路於營利事業所得稅選案之應用
★ 一個新的線上學習系統及其於稅務選案上之應用	★ 人眼追蹤系統及其於人機介面之應用
★ 結合群體智慧與自我組織映射圖的資料視覺化研究	★ 追瞳系統之研發於身障者之人機介面應用
★ 以類免疫系統為基礎之線上學習類神經模糊系統及其應用	★ 基因演算法於語音聲紋解攪拌之應用

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

早產兒因發育尚未成熟，所以相較於一般足月嬰兒有更高的風險在運動神經發展上有所受損或罹患腦性麻痺。臨床上，要精準判斷一個嬰兒是否在運動神經發展上有所受損，不是依賴一個有經驗的醫生在一段錄影好的影片上利用視覺來評估，就是得要仰賴昂貴的儀器。可想而知，這是個耗時且昂貴的診療程序。所以，一個能夠自動化評估的輔助系統就變得十分迫切了。最近幾年來，OpenPose 是一個有效且被普遍採用的姿態估測的演算法，但它的缺點是它是針對成人姿態所訓練而成的，導致若是應用於嬰兒姿態偵測時，整體效果會下降。所以，此論文會開發一個嬰兒姿態偵測演算法，然後，據此演算法開發出一個嬰兒整體動作指標分析輔助系統。首先透過單眼相機記錄嬰兒影像，再透過生成對抗網路 (Generative Adversarial Network)生成嬰兒骨架，最後使用一些後處理技術 (如：平滑化、去雜訊濾波器等)萃取出骨架點資訊。基於以上骨架點資訊，嬰兒運動狀態可以透過位移與速度的變異數與運動頻率來呈現，之後再自動化輸出相對應的嬰兒評估指標。最後本論文以實驗驗證提出生成對抗網路的關節預測準確性與整體嬰兒評估指標之有效性。

摘要(英)

Premature babies are not yet mature, so they have a higher risk of developing motor nerve damage or cerebral palsy than full-term infants. Clinical assessments for infant’s risk of developing neuromotor impairment either are assessed through visual examination by specialized clinicians via recorded videos or involve expensive equipment, which are usually time-consuming, expensive, and only available in highly-resourced environments. This makes assessment inaccessible for families of limited means and in low resource countries. Therefore, it is desirable to automate the process of evaluating the quality of infant movements; otherwise, the early identification is not possible. In recent years, the OpenPose is a very popular human pose estimation algorithm; however, it focuses on adults, leading a degradation of accuracy if applied to infants. Therefore, this paper will develop an algorithm for infant posture detection, and then, based on this algorithm, an auxiliary system for the analysis of infant overall motion indicators will be developed. First record the baby image through the monocular camera, then generate the baby skeleton through the Generative Adversarial Network, and finally use some post-processing techniques (such as smoothing, noise reduction filter, etc.) to extract the skeleton point information. Based on the above skeleton point information, the baby′s movement status can be presented through the variation of speed and acceleration and the movement frequency, and then the corresponding infant evaluation index is automatically output. Finally, some experiments will be designed to validate the effectiveness of proposed method.

關鍵字(中)

★ 生成對抗網路
★ 嬰兒骨架偵測
★ 嬰兒姿態分析
★ 嬰兒整體動作指標
★ 腦性麻痺

關鍵字(英)

★ GAN (generative adversarial network)
★ Infant skeleton detection
★ Infant motion analysis
★ cerebral palsy
★ GMA (General Movement Assessment)

論文目次

摘要 i
ABSTRACT i
致謝 iii
目錄 iv
圖目錄 vi
表目錄 vii
第一章、緒論 1
1-1 研究動機 1
1-2 研究目的 3
1-3 論文架構 4
第二章、相關研究 5
2-1 Mask R-CNN 5
2-2 嬰兒姿態辨識 7
2-2-1 穿戴式感測器姿態辨識 8
2-2-2 影像姿態辨識 9
2-3 生成對抗網路 13
2-3-1 Pix2Pix GAN 14
2-4 嬰兒整體動作評估 15
第三章、研究方法 18
3-1 嬰兒資料分析 18
3-1-1 早產兒資料集 18
3-1-2 嬰兒整體運動指標分析 20
3-2 系統流程 21
3-3 嬰兒骨架偵測 23
3-3-1 擷取嬰兒影像 23
3-3-2 骨架填補 25
3-3-3 後處理骨架資料 27
3-3-4 座標校正 32
3-3-5 後處理時序關節點資料 33
3-3-6 正規化 35
第四章、實驗設計與結果 36
4-1 對抗生成網路填補骨架顏色實驗 36
4-1-1 實驗設計 36
4-1-2 實驗評估方式 38
4-1-3 實驗結果與分析 39
4-2 尋找嬰兒關節點後處理實驗 41
4-2-1 實驗設計 41
4-2-2 實驗評估方式 43
4-2-3 實驗結果與分析 44
4-3 嬰兒骨架點偵測實驗 45
4-3-1 實驗設計 45
4-3-2 實驗評估方式 46
4-3-3 實驗結果與分析 47
4-4 不同嬰兒間推廣性實驗 51
4-4-1 實驗設計 51
4-4-2 實驗評估方式 52
4-4-3 實驗結果分析 52
4-5 成人與嬰兒骨架點偵測差異性實驗 53
4-5-1 實驗設計 53
4-5-2 實驗評估方式 53
4-5-3 實驗結果分析 53
4-6 嬰兒動作指標去雜訊化實驗 55
4-6-1 實驗設計 55
4-6-2 實驗評估方式 55
4-6-3 實驗結果與分析 57
第五章、結論與未來展望 58
5-1 結論 58
5-2 未來展望 59
參考文獻 61

參考文獻

[1] 廖華芳, “台北市兩醫學機構腦性麻痺兒童復健相關資料之調查,” 台灣醫學, 第一冊, 第三期, pp. 274-288, 五月1997.
[2] T. Hielkema, E. G. Hamer, and H. A. Reinders-Messelink, “LEARN 2 MOVE 0-2 years: effects of a new intervention program in infants at very high risk for cerebral palsy; a randomized controlled trial,” BMC pediatrics, vol. 10, no. 1, pp. 1-8, 2010.
[3] D. A. Nzeh, S. A. Erinle and S. A. Saidu, “Transfontanelle Ultra-Sonography: An Invaluable Tool in the Assessment of the Infant Brain,” Tropical doctor, vol. 34, no. 4, pp. 226-227, 2004.
[4] A. Pfefferbau, D. H. Mathalon and E. V. Sullivan, “A quantitative magnetic resonance imaging study of changes in brain morphology from infancy to late adulthood,” Archives of neurology, vol. 51, no. 9, pp. 874-887, 1994.
[5] K. Watanabe, F. Hayakawa and A. Okumura, “Neonatal EEG: a powerful tool in the assessment of brain damage in preterm infants,” Brain and Development, vol. 21, no. 6, pp. 361-372, 1999.
[6] M. A. Franceschini, S. Thaker, G. Themelis and K. K. Krishnamoorthy, “Assessment of infant brain development with frequency-domain near-infrared spectroscopy,” Pediatric research, vol. 61, no. 5, pp. 546-551, 2007.
[7] H. F. R. Prechtl, “General movement assessment as a method of developmental neurology: new paradigms and their consequences,” Developmental Medicine and Child Neurology, vol. 43, no. 12, pp. 836-842, 2001.
[8] C. Einspieler, H. F. R. Prechtl, A. F. Bos, F. Ferrari and G. Cioni, Prechtl′s method on the qualitative assessment of general movements in preterm, term and young infants. London: UK: Mac Keith Press, 2004.
[9] I. J. Goofellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville and Y. Bengio, “Generative adversarial nets,” In Advances in neural information processing systems, 2014, pp. 2672-2680.
[10] K. He, G. Gkioxari, P. Dollar and R. Girshick, “Mask r-cnn,” in Proc. of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
[11] S. Ren, K. He, R. Girshick and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE Trans Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2015.
[12] A. Rifar, M. Mihelj, J. Pasic, H. Kolar and M. Munih, “Infant trunk posture and arm movement assessment using pressure mattress, inertial and magnetic measurement units (IMUs),” Journal of Neuroengineering and Rehabilitation, vol. 11, no. 1, pp. 1-14, 2014.
[13] B. A. Smith, I. A. Trujillo-Priego, C. J. Lane, J. M. Finley and F. B. Horak, “Daily quantity of infant leg movement: wearable sensor algorithm and relationship to walking onset,” Sensors, vol. 15, no. 8, pp. 19006-19020, 2015.
[14] A. Rihar, G. Sgandurra, E. Beani, F. Cecchi, J. Pasic, G. Cioni, P. Dario, M. Mihelj and M. Munih, “CareToy: Stimulation and assessment of preterm infant’s activity using a novel sensorized system,” Annals of biomedical engineering, vol. 44, no. 12, pp. 3593-3605, 2016.

[15] R. F. Morgan, Baby by the Numbers: A Parent′s Quick Reference for Essential Baby Facts and Figures. Richmond:CA, Chronicle Books, 2008.
[16] M. Singh and D. J. Patterson, “Involuntary gesture recognition for predicting cerebral palsy in high-risk infants,” in Proc. International Symposium on Wearable Computers, 2010, pp. 1-8.
[17] E. Saadatian, S. P. Iyer, C. Lihui, O. N. N. Fernando, N. Hdeaki, A. D. Cheok, A. P. Madurapperuma, G. Ponnampalam and Z. Amin, “Low cost infant monitoring and communication system,” in Proc. 2011 IEEE Colloquium on Humanities, Science and Engineering, 2011, pp. 503-508.
[18] F. Heinze, K. Hesels, N. Breitbach-Faller, T. Schmitz-Rode and C. Disselhorst-Klug, “Movement analysis by accelerometry of newborns and infants for the early detection of movement disorders due to infantile cerebral palsy,” Medical & biological engineering & computing, vol. 48, no. 8, pp. 765-772, 2010.
[19] S. Boughorbel, F. Bruekers and J. Breebaart, “Baby-posture classification from pressure-sensor data,” in Proc. 2010 20th International Conference on Pattern Recognition, 2010, pp. 556-559.
[20] M. Donati, F. Cecchi, F. Bonaccorso, M. Branciforte, P. Dario and N. Vitiello,
“A modular sensorized mat for monitoring infant posture,” Sensors, vol.14, no. 1, pp. 510-531, 2014.
[21] M. Farooq, P. C. Chandler-Laney, M. Hernandez-Reif and E. Sazonov,
“Monitoring of infant feeding behavior using a jaw motion sensor,” Journal of healthcare engineering, vol. 6, no. 1, pp. 23-40, 2015.
[22] E. Koch and A. Dierzel, “Skin attachable flexible sensor array for respiratory monitoring,” Sensors and Actuators A: Physical, vol. 250, pp. 138-144, 2014.
[23] E. Rogers, P. Polygerinos, C. Waish and E. Goldfield, “Smart and connected actuated mobile and sensing suit to encourage motion in developmentally delayed infants,” Journal of Medical Devices, vol. 9, no. 3, 2015.
[24] L. Meinecke, N. Breitbach-Faller, C. Bartz and R. Damen, G.Disselhorst-Klug, “Movement analysis in the early detection of newborns at risk for developing spasticity due to infantile cerebral palsy,” Human movement science, vol. 25, no. 2, pp. 125-144, 2006.
[25] J. H. Holland, “Genetic algorithms,” Scientific american, vol. 267, no. 1, pp. 66-73, 1992.
[26] D. Karch, K. S. Kang, K. Wochner, H. Philippi, M. Hadders-Algra, J. Pietz and H. Dickhaus, “Kinematic assessment of stereotypy in spontaneous movements in infants,” Gait & posture, vol. 36, no.2, pp. 307-311, 2012.
[27] H. Rahmati, O. M. Aamo, O. Stavdahl, R. Dragon and L. Adde, “Video-based early cerebral palsy prediction using motion segmentation,” in Proc. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014, pp. 3779-3783.
[28] A. Stahl, C. Schellewald, O. Stavdahl, O. M. Aamo, L. Adde and H. Kirkerod,
“An optical flow-based method to predict infantile cerebral palsy,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 20, no. 4, pp. 605-314, 2012.
[29] Z. Cao, T. Simon, S. E. Wei and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proc. IEEE conference on computer vision and pattern recognition, 2017, pp. 7291-7299.
[30] A. Haque, B. Peng, Z. Luo, A. Alahi, S. Yeung and F. F. Li, “Towards viewpoint invariant 3d human pose estimation,” in Proc. European Conference on Computer Vision, 2016, pp. 160-177.
[31] C. Zimmermann, T. Welschehold, C. Dornhege, W. Burgard and T. Brox, “3D human pose estimation in rgbd images for robotic task learning,” in Proc. 2018 IEEE International Conference on Robotics and Automation, 2018, pp. 1986-1992.
[32] Kinect, Wikipedia. [Online]. Available: https://en.wikipedia.org/wiki/Kinect [Accessed: 20-Jun-2019]
[33] M. D. Oslen, A. Herskind, J. B. Nielsen and R. R. Paulsen, “Body part tracking of infants,” in Proc. International Conference on Pattern Recognition, 2014, pp. 2167-2172.
[34] N. Hesse, G. Stachowiak, T. Breuer and M. Arens, “Estimating body pose of infants in depth images using random ferns,” in Proc. IEEE International Conference on Computer Vision Workshops, 2015, pp. 35-43.
[35] C. Chambers, N. Seethapathi, R. Saluja H. Loeb, S. Pierce, D. Bogen, L. Prosser, M. J. Johnson and K. P. Kording, “Computer vision to automatically assess infant neuromotor risk,” 2019, BioRxiv:756262. [Online]. Available:
https://www.biorxiv.org/content/10.1101/756262v1

[36] V. Marchi, A. Hakala, A. Knight, F. D’Acunto, M. L. Scattoni, A. Guzzetta and S. Vanhatalo, “Automated pose estimation captures key aspects of General Movements at eight to 17 weeks from conventional videos,” Acta Paediatrica, vol. 108, no. 10, pp. 1817-1824, 2019.
[37] A. Radford, L. Metz and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” 2015, arXiv:1511.06434. [Online]. Available: https://arxiv.org/abs/1511.06434
[38] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” 2014, arXiv:1411.1784. [Online]. Available: https://arxiv.org/abs/1411.1784
[39] P. Isola, J. Y. Zhu, T. Zhou and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proc. IEEE conference on computer vision and pattern recognition, 2017, pp. 1125-1134.
[40] H. F. R. Prechtl, “Qualitative changes of spontaneous movements in fetus and preterm infant are a marker of neurological dysfunction,” Early human development, vol. 23, no. 3, pp. 151-158, 1990.
[41] C. Einspieler, P. B. Marschik, J. Pansy, A. Scheuchenegger, M. Krieber, H. Yang, M. K. Kornacka, E. Rowinska, M. Soloveichick and A. F. Bos, “The general movement optimality score: a detailed assessment of general movements during preterm and term age,” Developmental Medicine & Child Neurology, vol.58, no. 4, pp. 361-368, 2016.
[42] N. Kanemaru, H. Watanabe, H. Kihara, H. Nakano, R. Takaya, T. Nakamura, J. Nakano, G. Taga and Y. Konishi, “Specific characteristics of spontaneous movements in preterm infants at term age are associated with developmental delays at age 3 years,” Developmental Medicine & Child Neurology, vol. 55, no. 8, pp. 713-721, 2013.
[43] X. Chu, W. Yang, W. Ouyang, C. Ma, A. L. Yuille and X. Wang, “Multi-context attention for human pose estimation,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1831-1840.
[44] A. Mykhaylo, L. Pishchulin, P. Gehler and S. Bernt, “2D Human Pose Estimation: New Benchmark and State of the Art Analysis,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3686-3693.
[45] S. Wold, K. Esbensen and P. Geladi, “Principal component analysis,” Chemometrics and intelligent laboratory systems, vol. 2, no.1-3, pp. 37-52, 1987.
[46] G. Bradski and A. Kaehler, OpenCV. Dr. Dobb’s journal of software tools, 2000.
[47] K. Gong, X. Liang, Y. Li, C. Ma, A. Y. Chen, M. Yang and L. Lin, “Instance-level Human Parsing via Part Grouping Network,” in Proc. the European Conference on Computer Vision (ECCV), 2018, pp. 770-785.

指導教授

蘇木春(Mu-Chun Su)

審核日期

2020-8-18

推文