探討Media-Pipe和Leap Motion藉由用於發展遲緩的VR系統在分層共現網路的對指手勢預測精準度比較

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：16

、訪客IP：3.238.227.73

姓名

游盛棋(Sheng-Chi Yu) 查詢紙本館藏

畢業系所

軟體工程研究所

論文名稱

探討Media-Pipe和Leap Motion藉由用於發展遲緩的VR系統在分層共現網路的對指手勢預測精準度比較
(Using Hierarchical Co-occurrence Network to Compare the Prediction Accuracy of Fine Motor Gesture between Media-Pipe and Leap Motion via a VR System for Developmental Delays)

相關論文

★ 基於眼動的閱讀障礙分析與診斷	★ 在有干擾的虛擬教室環境下大人小孩的行為表現與腦神經反應的異同
★ 使用映射模型和跨資料集遷移式學習的輕量化居家衰弱症訓練系統	★ 心率生理回饋放鬆訓練對於海洛因使用疾患(HUD)生理資訊之影響分析
★ 基於深度學習模型的3D心理旋轉對認知障礙的診斷與評估	★ 評估注意力偵測之穿戴式腦電電極放置有效性

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 ( 永不開放)

摘要(中)

發展遲緩兒童的診斷以及治療需要有專業人員的協助才能進行診斷並規劃治療流程，並且需要隨時篩查兒童的狀況以擬訂不同的治療方針，但在偏鄉缺乏人力及資源的情況下，孩童的障礙特徵容易被忽視，因而錯失早期介入的黃金時期。我們開發了一個系統，可以藉由遠端的方式，由診療師給定任務，孩童完成任務後，遊戲會回傳遊玩結果給雲端資料庫，該雲端資料庫將數據處理為可視化的型式傳給診療師，以利後續追蹤及任務難度修改。在對指的手勢識別上，我們使用hierarchical co-occurrence network (HCN)分類手勢，該架構結合空間資訊和時間資訊的輸入，以達到全域共現的效果。在過去，我們有收集LMC在抓木塊遊戲的數據，因此嘗試利用mapping model，將LMC的數據映射為media pipe的數據，協助我們的遊戲在AIOT的應用。

摘要(英)

The diagnosis and treatment of children with developmental delays requires professional assistance to diagnose and plan the treatment process. Child’s condition needs to be screened at all times to develop a different treatment approach. However, in rural areas, where there is a lack of resources, the characteristics of the child′s impairment can easily be overlooked, thus missing a golden opportunity for early intervention. We have developed a system where tasks are given by the therapist via a remote location and when the child completes the task, the game sends back the results to a cloud-based database which processes the data into a visual format for follow-up and task modification. For finger gesture recognition and developmental recognition task, we use a hierarchical co-occurrence network to classify gestures, which combines the input of spatial and temporal information to achieve global co-presence. In the past, we have collected data from LMC in grab wood block game, so we try to use the mapping model to map the data from LMC to media pipe to help our game in artificial intelligence of things (AIOT) application.

關鍵字(中)

★ 早期療育
★ 精細動作
★ 分層共現網路
★ 轉移學習
★ 監督式學習
★ media pipe
★ AIOT
★ leap motion

關鍵字(英)

★ Early intervention
★ fine motor assessment
★ hierarchical co-occurrence network
★ transfer learning
★ supervised learning
★ media pipe
★ AIOT
★ leap motion

論文目次

摘要 I
Abstract II
致謝 III
List of Figures V
List of Tables VII
Introduction 1
Related Works 5
Methodology 10
Experimental Result 25
Conclusion and Future Work 32
Reference 33

參考文獻

[1] Huang, Pi-Hsia. "The development and current situation of the early intervention for children with developmental delay in Taiwan." International Journal of Child Care and Education Policy 1.1 (2007): 45-58.
[2] Aguilera-Rubio, Ángela, et al. "Use of the Leap Motion Controller® System in the Rehabilitation of the Upper Limb in Stroke. A Systematic Review." Journal of Stroke and Cerebrovascular Diseases 31.1 (2022): 106174.
[3] Tarakci, Ela, et al. "Leap Motion Controller–based training for upper extremity rehabilitation in children and adolescents with physical disabilities: A randomized controlled trial." Journal of Hand Therapy 33.2 (2020): 220-228.
[4] Pereira, Catarina Ramos, et al. "Acute effects of acupuncture in balance and gait of Parkinson disease patients–A preliminary study." Complementary therapies in clinical practice 45 (2021): 101479.
[5] Hill, Cherice N., et al. "Effect of osteopathic manipulation on gait asymmetry." Journal of Osteopathic Medicine 122.2 (2022): 85-94.
[6] Dranca, Lacramioara, et al. "Using Kinect to classify Parkinson’s disease stages related to severity of gait impairment." BMC bioinformatics 19.1 (2018): 1-15.
[7] Li, Qiannan, et al. "Classification of gait anomalies from kinect." The Visual Computer 34.2 (2018): 229-241.
[8] Koh, Wai Kin, et al. "End-to-End Hand Rehabilitation System with Single-Shot Gesture Classification for Stroke Patients." Soft Computing: Biomedical and Related Applications. Springer, Cham, 2021. 59-67.
[9] Trotta Lara Barbosa, Tiago, et al. "ReBase: data acquisition and management system for neuromotor rehabilitation supported by virtual and augmented reality." Symposium on Virtual and Augmented Reality. 2021.
[10] Haseeb, Mohamed Abudulaziz Ali, and Ramviyas Parasuraman. "Wisture: RNN-based learning of wireless signals for gesture recognition in unmodified smartphones." arXiv preprint arXiv:1707.08569 (2017).
[11] Do, Nhu-Tai, et al. "Robust hand shape features for dynamic hand gesture recognition using multi-level feature LSTM." Applied Sciences 10.18 (2020): 6293.
[12] Hakim, Noorkholis Luthfil, et al. "Dynamic hand gesture recognition using 3DCNN and LSTM with FSM context-aware model." Sensors 19.24 (2019): 5429.
[13] Chakraborty, Shayak, et al. "Study of Dependency on number of LSTM units for Character based Text Generation models." 2020 International Conference on Computer Science, Engineering and Applications (ICCSEA). IEEE, 2020.
[14] Sun, Yanan, et al. "Automatically designing CNN architectures using the genetic algorithm for image classification." IEEE transactions on cybernetics 50.9 (2020): 3840-3854.
[15] Gour, Mahesh, Sweta Jain, and T. Sunil Kumar. "Residual learning based CNN for breast cancer histopathological image classification." International Journal of Imaging Systems and Technology 30.3 (2020): 621-635.
[16] Jiang, Du, et al. "Gesture recognition based on skeletonization algorithm and CNN with ASL database." Multimedia Tools and Applications 78.21 (2019): 29953-29970.
[17] Chung, Yao-Liang, Hung-Yuan Chung, and Wei-Feng Tsai. "Hand gesture recognition via image processing techniques and deep CNN." Journal of Intelligent & Fuzzy Systems 39.3 (2020): 4405-4418.
[18] M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks", ECCV, 2014.
[19] Li, Chao, et al. "Co-occurrence feature learning from skeleton data for action recognition and detection with hierarchical aggregation." arXiv preprint arXiv:1804.06055 (2018).
[20] Bronfenbrenner, U., “Is early intervention effective?” Day Care and Early Education 2.2, pp. 14–18, 1974.
[21] Shuai, Lan, et al. "Executive function training for preschool children with ADHD: a randomized controlled trial." Journal of Attention Disorders 25.14 (2021): 2037-2047.
[22] Morgan, Catherine, et al. "Early intervention for children aged 0 to 2 years with or at high risk of cerebral palsy: international clinical practice guideline based on systematic reviews." JAMA pediatrics 175.8 (2021): 846-858.
[23] Wu, Jiaojiao, et al. "Early intervention for children with intellectual and developmental disability using drama therapy techniques." Children and Youth Services Review 109 (2020): 104689.
[24] Strooband, Karel FB, et al. "Systematic review and meta-analyses: Motor skill interventions to improve fine motor development in children aged birth to 6 years." Journal of Developmental & Behavioral Pediatrics 41.4 (2020): 319-331.
[25] Cameron, Claire E., et al. "Fine motor skills and executive function both contribute to kindergarten achievement." Child development 83.4 (2012): 1229-1244.
[26] Roth, Mary Ann, Edward McCaul, and Karoldene Barnes. "Who becomes an “at-risk” student? The predictive value of a kindergarten screening battery." Exceptional Children 59.4 (1993): 348-358.
[27] Williams, Justin HG, Andrew Whiten, and Tulika Singh. "A systematic review of action imitation in autistic spectrum disorder." Journal of autism and developmental disorders 34.3 (2004): 285-299.
[28] Ghosh, Dipak Kumar, and Samit Ari. "Static hand gesture recognition using mixture of features and SVM classifier." 2015 Fifth International Conference on Communication Systems and Network Technologies. IEEE, 2015.
[29] Chen, Yen-Ting, and Kuo-Tsung Tseng. "Multiple-angle hand gesture recognition by fusing SVM classifiers." 2007 IEEE International Conference on Automation Science and Engineering. IEEE, 2007.
[30] Mufarroha, Fifin Ayu, and Fitri Utaminingrum. "Hand gesture recognition using adaptive network based fuzzy inference system and K-nearest neighbor." International Journal of Technology 8.3 (2017): 559-567.
[31] Liao, Shangchun, et al. "Multi-object intergroup gesture recognition combined with fusion feature and KNN algorithm." Journal of Intelligent & Fuzzy Systems 38.3 (2020): 2725-2735.
[32] Chen, Qing, Nicolas D. Georganas, and Emil M. Petriu. "Real-time vision-based hand gesture recognition using haar-like features." 2007 IEEE instrumentation & measurement technology conference IMTC 2007. IEEE, 2007.
[33] Wang, Chieh-Chih, and Ko-Chih Wang. "Hand posture recognition using adaboost with sift for human robot interaction." Recent progress in robotics: viable robotic service to human. Springer, Berlin, Heidelberg, 2007. 317-329.
[34] Zhang, Fan, et al. "Mediapipe hands: On-device real-time hand tracking." arXiv preprint arXiv:2006.10214 (2020).
[35] Chen, Xinghao, et al. "Mfa-net: Motion feature augmented network for dynamic hand gesture recognition from skeletal data." Sensors 19.2 (2019): 239.
[36] Sabater, Alberto, et al. "Domain and view-point Agnostic hand action recognition." IEEE Robotics and Automation Letters 6.4 (2021): 7823-7830.
[37] Hou, Jingxuan, et al. "Spatial-temporal attention res-TCN for skeleton-based dynamic hand gesture recognition." Proceedings of the European conference on computer vision (ECCV) workshops. 2018.
[38] Yang, Fan, et al. "Make skeleton-based action recognition model smaller, faster and better." Proceedings of the ACM multimedia asia. 2019. 1-6.
[39] Maghoumi, M. and LaViola, J.J. 2019. DeepGRU: Deep Gesture Recognition Utility. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2019).
[40] Shin, S. and Kim, W.Y. 2020. Skeleton-Based Dynamic Hand Gesture Recognition Using a Part-Based GRU-RNN for Gesture-Based Interface. IEEE Access. 8, (2020), 50236–50243. DOI:https://doi.org/10.1109/ACCESS.2020.2980128.
[41] AQ Mohammed, Adam, et al. "Automatic 3D Skeleton-based Dynamic Hand Gesture Recognition Using Multi-Layer Convolutional LSTM." 2021 7th International Conference on Robotics and Artificial Intelligence. 2021.
[42] Chen, Y., Zhao, L., Peng, X., Yuan, J. and Metaxas, D.N. 2020. Construct dynamic graphs for hand gesture recognition via spatial-temporal attention. 30th British Machine Vision Conference 2019, BMVC 2019 (2020).

指導教授

葉士青吳曉光(Shih-Ching Yeh Hsiao-Kuang Wu)

審核日期

2022-8-30

推文