博碩士論文 110522130 詳細資訊




以作者查詢圖書館館藏 以作者查詢臺灣博碩士 以作者查詢全國書目 勘誤回報 、線上人數:128 、訪客IP:3.129.26.254
姓名 李信鋌(Shun-Ting Li)  查詢紙本館藏   畢業系所 資訊工程學系
論文名稱 可解釋人工智慧在教育之應用
(Explainable Artificial Intelligence in Education)
相關論文
★ 應用智慧分類法提升文章發佈效率於一企業之知識分享平台★ 家庭智能管控之研究與實作
★ 開放式監控影像管理系統之搜尋機制設計及驗證★ 資料探勘應用於呆滯料預警機制之建立
★ 探討問題解決模式下的學習行為分析★ 資訊系統與電子簽核流程之總管理資訊系統
★ 製造執行系統應用於半導體機台停機通知分析處理★ Apple Pay支付於iOS平台上之研究與實作
★ 應用集群分析探究學習模式對學習成效之影響★ 應用序列探勘分析影片瀏覽模式對學習成效的影響
★ 一個以服務品質為基礎的網際服務選擇最佳化方法★ 維基百科知識推薦系統對於使用e-Portfolio的學習者滿意度調查
★ 學生的學習動機、網路自我效能與系統滿意度之探討-以e-Portfolio為例★ 藉由在第二人生內使用自動對話代理人來改善英文學習成效
★ 合作式資訊搜尋對於學生個人網路搜尋能力與策略之影響★ 數位註記對學習者在線上學習環境中反思等級之影響
檔案 [Endnote RIS 格式]    [Bibtex 格式]    [相關文章]   [文章引用]   [完整記錄]   [館藏目錄]   至系統瀏覽論文 (2028-8-1以後開放)
摘要(中) 在過去的研究中,發現透過機器學習可以更好的預測學生的學習成效,並可
以透過可解釋人工智慧方式對模型的預測進行解釋。但是不同解釋方式可能對同一筆資料帶來不同的解釋,導致教學團隊無所適從。
本研究根據LBLS-467 資料集,一個記錄大學生在線上學習環境的課程中的
行為日誌與問卷資料,透過機器學習的方法識別高風險與低風險學生。另外,我們進一步使用可解釋人工智慧方法(LIME、SHAP)提取模型預測之解釋,並利用六種評估解釋的評估標準評估解釋的穩定程度(Stability) 與真實程度(Faithfulness),幫助教師選出更好的解釋,更加瞭解學生目前的學習狀況。
研究結果顯示分類器識別了大多數有風險的學生,也就是說根據LBLS-467
資料集可以找出有風險的學生。透過可解釋人工智慧方法可以得到模型預測的解釋,進一步幫助教師可以了解學生的學習情況。而透過六種評估解釋之評估標準可找出品質較好之解釋,使教學團隊可選出更好的解釋。
摘要(英) In past research, it has been found that machine learning can better predict students′ learning outcomes, and it can provide explanations for the model′s predictions through explainable artificial intelligence methods. However, different explanation methods may yield different interpretations for the same data, leaving teaching teams uncertain.
This study is based on the LBLS-467 dataset, which records behavioral logs and questionnaire data of university students in an online learning environment. Using machine learning techniques, high-risk and low-risk students are identified.
Additionally, we further utilize explainable artificial intelligence methods (LIME, SHAP) to extract explanations for the model′s predictions. We evaluate the stability and
faithfulness of the explanations using six evaluation criteria, helping teachers select better explanations and gain a deeper understanding of students′ current learning status.
The research results show that the classifiers identified the majority of at-risk students, meaning that with the LBLS-467 dataset, it is possible to identify students at
risk. Through the use of explainable artificial intelligence methods, explanations for the model′s predictions can be obtained, further assisting teachers in understanding
students′ learning situations. By utilizing the six evaluation criteria for explaining, high quality explanations can be identified, enabling teaching teams to select better explanations.
關鍵字(中) ★ 可解釋人工智慧
★ 機器學習
★ 學習分析
關鍵字(英) ★ Explainable Artificial Intelligence
★ Machine Learning
★ Learning Analytics
論文目次 摘要 i
Abstract ii
致謝 iii
目錄 iv
圖目錄 v
表目錄 vi
1. 緒論 1
2. 文獻探討 2
2.1. 人工智慧在教育之應用 2
2.2. 可解釋人工智慧 3
2.2.1. 可解釋人工智慧方法分類 4
2.2.2. 常見可解釋人工智慧方法介紹 5
2.3. 解釋分歧問題 6
2.4. 解釋評量方法 6
2.4.1. 基於評估指標進行解釋評量 6
2.4.2. 基於社會學和認知特徵進行解釋評量 9
3. 研究方法 10
3.1. 實驗設計 10
3.2. 資料集 11
3.3. 資料預處理 12
3.4. 特徵整理 13
3.5. 資料正規化 15
3.6. 模型訓練 16
3.7. 模型效能評估 18
3.8. 模型預測解釋生成 19
3.9. 解釋品質評估 21
4. 結果與討論 24
4.1. 哪些評估指標可用來衡量可解釋人工智慧模型所給出的解釋 24
4.2. 模型效能及解釋品質評估 27
4.3. 解釋品質探討 28
4.4. 全局解釋生成結果 29
4.5. 局部解釋生成結果 32
4.6. 解釋分歧結果 35
4.7. 遇到解釋分歧時的解決方式 35
5. 結論與未來研究 36
參考文獻 38
附錄 45
1.自我調節學習問卷(MSLQ) 45
2.語言學習策略問卷(SILL) 48
參考文獻 Salas-Pilco, S. Z., Xiao, K., & Hu, X. (2022). Artificial intelligence and learning
analytics in teacher education: A systematic review. Education Sciences, 12(8),
569.
Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., & Arshad,
H. (2018). State-of-the-art in artificial neural network applications: A survey.
Heliyon, 4(11), e00938.
Agarwal, C., Johnson, N., Pawelczyk, M., Krishna, S., Saxena, E., Zitnik, M., &
Lakkaraju, H. (2022). Rethinking stability for attribution-based explanations.
arXiv preprint arXiv:2203.06877.
Agarwal, C., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Puri, I., Zitnik, M.,
& Lakkaraju, H. (2022). Openxai: Towards a transparent evaluation of model
explanations. Advances in neural information processing systems, 35, 15784-
15799.
Albreiki, B., Zaki, N., & Alashwal, H. (2021). A systematic literature review of
student’performance prediction using machine learning techniques. Education
Sciences, 11(9), 552.
Alvarez Melis, D., & Jaakkola, T. (2018). Towards robust interpretability with selfexplaining
neural networks. Advances in neural information processing
systems, 31.
Alwarthan, S., Aslam, N., & Khan, I. U. (2022). An explainable model for identifying
at-risk student at higher education. Ieee Access, 10, 107649-107668.
Ancona, M., Ceolini, E., Öztireli, C., & Gross, M. (2017). Towards better
understanding of gradient-based attribution methods for deep neural networks.
arXiv preprint arXiv:1711.06104.
Arias-Duart, A., Parés, F., Garcia-Gasulla, D., & Giménez-Ábalos, V. (2022). Focus!
Rating XAI Methods and Finding Biases. 2022 IEEE International Conference
on Fuzzy Systems (FUZZ-IEEE),
Arras, L., Osman, A., & Samek, W. (2020). Ground truth evaluation of neural network
explanations with clevr-xai. arXiv preprint arXiv:2003.07258.
Arya, V., Bellamy, R. K., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S. C.,
Houde, S., Liao, Q. V., Luss, R., & Mojsilović, A. (2019). One explanation
does not fit all: A toolkit and taxonomy of ai explainability techniques. arXiv
preprint arXiv:1909.03012.
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W.
(2015). On pixel-wise explanations for non-linear classifier decisions by layerwise
relevance propagation. PloS one, 10(7), e0130140.
Baker, R. S. (2019). Challenges for the future of educational data mining: The Baker
learning analytics prizes. Journal of Educational Data Mining, 11(1), 1-17.
Bhatt, U., Weller, A., & Moura, J. M. (2020). Evaluating and aggregating featurebased
model explanations. arXiv preprint arXiv:2005.00631.
Cahour, B., & Forzy, J.-F. (2009). Does projection into use improve trust and
exploration? An example with a cruise control system. Safety science, 47(9),
1260-1270.
Chalasani, P., Chen, J., Chowdhury, A. R., Wu, X., & Jha, S. (2020). Concise
explanations of neural networks using adversarial training. International
Conference on Machine Learning,
Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review.
Ieee Access, 8, 75264-75278.
Chui, K. T., Fung, D. C. L., Lytras, M. D., & Lam, T. M. (2020). Predicting at-risk
university students in a virtual learning environment via a machine learning
algorithm. Computers in Human Behavior, 107, 105584.
Collins, R. P., Litman, J. A., & Spielberger, C. D. (2004). The measurement of
perceptual curiosity. Personality and individual differences, 36(5), 1127-1141.
Das, A., & Rad, P. (2020). Opportunities and challenges in explainable artificial
intelligence (xai): A survey. arXiv preprint arXiv:2006.11371.
Dasgupta, S., Frost, N., & Moshkovitz, M. (2022). Framework for evaluating
faithfulness of local explanations. International Conference on Machine
Learning,
Došilović, F. K., Brčić, M., & Hlupić, N. (2018). Explainable artificial intelligence: A
survey. 2018 41st International convention on information and communication
technology, electronics and microelectronics (MIPRO),
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z.,
Shah, T., & Morgan, G. (2023). Explainable AI (XAI): Core ideas, techniques,
and solutions. ACM Computing Surveys, 55(9), 1-33.
Eckles, J. E., & Stradley, E. G. (2012). A social network analysis of student retention
using archival data. Social Psychology of Education, 15, 165-180.
Ferri, C., Hernández-Orallo, J., & Modroiu, R. (2009). An experimental comparison
of performance measures for classification. Pattern recognition letters, 30(1),
27-38.
Flanagan, B., & Ogata, H. (2017). Integration of learning analytics research and
production systems while protecting privacy. The 25th International
Conference on Computers in Education, Christchurch, New Zealand,
Gaikwad, S. K., Gawali, B. W., & Yannawar, P. (2010). A review on speech
recognition technique. International Journal of Computer Applications, 10(3),16-24.
Gao, J. (2014). Machine learning applications for data center optimization.
Guresen, E., & Kayakutlu, G. (2011). Definition of artificial neural networks with
comparison to other networks. Procedia Computer Science, 3, 426-433.
Hart, S. G. (2006). NASA-task load index (NASA-TLX); 20 years later. Proceedings
of the human factors and ergonomics society annual meeting,
Hedström, A., Weber, L., Krakowczyk, D., Bareeva, D., Motzkus, F., Samek, W.,
Lapuschkin, S., & Höhne, M. M.-C. (2023). Quantus: An Explainable AI
Toolkit for Responsible Evaluation of Neural Network Explanations and
Beyond. Journal of Machine Learning Research, 24(34), 1-11.
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018). Metrics for
explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608.
Holzinger, A., Saranti, A., Molnar, C., Biecek, P., & Samek, W. (2022). Explainable
AI methods-a brief overview. xxAI-Beyond Explainable AI: International
Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna,
Austria, Revised and Extended Papers,
Hossin, M., & Sulaiman, M. N. (2015). A review on evaluation metrics for data
classification evaluations. International journal of data mining & knowledge
management process, 5(2), 1.
Hsiao, J. H.-w., Ngai, H. H. T., Qiu, L., Yang, Y., & Cao, C. C. (2021). Roadmap of
designing cognitive metrics for explainable artificial intelligence (XAI). arXiv
preprint arXiv:2108.01737.
Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996). Artificial neural networks: A
tutorial. Computer, 29(3), 31-44.
Jang, Y., Choi, S., Jung, H., & Kim, H. (2022). Practical early prediction of students’
performance using machine learning and eXplainable AI. Education and
Information Technologies, 1-35.
Jayalakshmi, T., & Santhakumaran, A. (2011). Statistical normalization and back
propagation for classification. International Journal of Computer Theory and
Engineering, 3(1), 1793-8201.
Jones, L. (2017). Driverless cars: when and where? Engineering & Technology, 12(2),
36-40.
Kashdan, T. B., Gallagher, M. W., Silvia, P. J., Winterstein, B. P., Breen, W. E., Terhar,
D., & Steger, M. F. (2009). The curiosity and exploration inventory-II:
Development, factor structure, and psychometrics. Journal of research in
personality, 43(6), 987-998.
Katarya, R., Gaba, J., Garg, A., & Verma, V. (2021). A review on machine learning
based student’s academic performance prediction systems. 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS),
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S.,
Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable
artificial intelligence in education. Computers and Education: Artificial
Intelligence, 3, 100074.
Kohlbrenner, M., Bauer, A., Nakajima, S., Binder, A., Samek, W., & Lapuschkin, S.
(2020). Towards best practice in explaining neural network decisions with
LRP. 2020 International Joint Conference on Neural Networks (IJCNN),
Krishna, S., Gupta, R., Verma, A., Dhamala, J., Pruksachatkun, Y., & Chang, K.-W.
(2022). Measuring fairness of text classifiers via prediction sensitivity. arXiv
preprint arXiv:2203.08670.
Krishna, S., Han, T., Gu, A., Pombra, J., Jabbari, S., Wu, S., & Lakkaraju, H. (2022).
The disagreement problem in explainable machine learning: A practitioner′s
perspective. arXiv preprint arXiv:2202.01602.
Lipton, Z. C. (2018). The mythos of model interpretability: In machine learning, the
concept of interpretability is both important and slippery. Queue, 16(3), 31-57.
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017). A survey of
deep neural network architectures and their applications. Neurocomputing,
234, 11-26.
Lu, O. H., Huang, A. Y., Huang, J. C., Huang, C. S., & Yang, S. J. (2016). Early-Stage
Engagement: Applying Big Data Analytics on Collaborative Learning
Environment for Measuring Learners′ Engagement Rate. 2016 International
Conference on Educational Innovation through Technology (EITT),
Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model
predictions. Advances in neural information processing systems, 30.
Marbouti, F., Diefes-Dux, H. A., & Madhavan, K. (2016). Models for early prediction
of at-risk students in a course using standards-based grading. Computers &
Education, 103, 1-15.
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021). The role of explainability in
creating trustworthy artificial intelligence for health care: a comprehensive
survey of the terminology, design choices, and evaluation strategies. Journal
of Biomedical Informatics, 113, 103655.
Merlo, J., Chaix, B., Ohlsson, H., Beckman, A., Johnell, K., Hjerpe, P., Råstam, L., &
Larsen, K. (2006). A brief conceptual tutorial of multilevel analysis in social
epidemiology: using measures of clustering in multilevel logistic regression to
investigate contextual phenomena. Journal of Epidemiology & Community
Health, 60(4), 290-297.
Merritt, S. M. (2011). Affective processes in human–automation interactions. Human Factors, 53(4), 356-370.
Montavon, G., Samek, W., & Müller, K.-R. (2018). Methods for interpreting and
understanding deep neural networks. Digital signal processing, 73, 1-15.
Mood, C. (2010). Logistic regression: Why we cannot do what we think we can do,
and what we can do about it. European sociological review, 26(1), 67-82.
Nguyen, A.-p., & Martínez, M. R. (2020). On quantitative aspects of model
interpretability. arXiv preprint arXiv:2007.07584.
O′shea, T. J., & West, N. (2016). Radio machine learning dataset generation with gnu
radio. Proceedings of the GNU Radio Conference,
Ogata, H., Oi, M., Mohri, K., Okubo, F., Shimada, A., Yamada, M., Wang, J., &
Hirokawa, S. (2017). Learning analytics for e-book-based educational big data
in higher education. Smart sensors at the IoT frontier, 327-350.
Ogata, H., Yin, C., Oi, M., Okubo, F., Shimada, A., Kojima, K., & Yamada, M.
(2015). E-Book-based learning analytics in university education. International
conference on computer in education (ICCE 2015),
Oqaidi, K., Aouhassi, S., & Mansouri, K. (2022). Towards a Students’ Dropout
Prediction Model in Higher Education Institutions Using Machine Learning
Algorithms. International Journal of Emerging Technologies in Learning
(Online), 17(18), 103.
Osmanbegovic, E., & Suljic, M. (2012). Data mining approach for predicting student
performance. Economic Review: Journal of Economics and Business, 10(1), 3-
12.
Oxford, R. (1990). Language learning strategiesWhat every teacher should know.
Heinle & heinle Publishers.;.
Peng, C.-Y. J., Lee, K. L., & Ingersoll, G. M. (2002). An introduction to logistic
regression analysis and reporting. The journal of educational research, 96(1),
3-14.
Pintrich, P. R. (1991). A manual for the use of the Motivated Strategies for Learning
Questionnaire (MSLQ).
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). " Why should i trust you?"
Explaining the predictions of any classifier. Proceedings of the 22nd ACM
SIGKDD international conference on knowledge discovery and data mining,
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018). Anchors: High-precision modelagnostic
explanations. Proceedings of the AAAI conference on artificial
intelligence,
Rieger, L., & Hansen, L. K. (2020). Irof: a low resource evaluation metric for
explanation methods. arXiv preprint arXiv:2003.08747.
Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data mining and knowledge discovery, 3(1), 12-27.
Rong, Y., Leemann, T., Borisov, V., Kasneci, G., & Kasneci, E. (2022). A consistent
and efficient evaluation strategy for attribution methods. arXiv preprint
arXiv:2202.00449.
Rosenfeld, A. (2021). Better metrics for evaluating explainable artificial intelligence.
Proceedings of the 20th international conference on autonomous agents and
multiagent systems,
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Müller, K.-R. (2016).
Evaluating the visualization of what a deep neural network has learned. IEEE
transactions on neural networks and learning systems, 28(11), 2660-2673.
Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K.-R. (2019).
Explainable AI: interpreting, explaining and visualizing deep learning (Vol.
11700). Springer Nature.
Schneider, K.-M. (2003). A comparison of event models for naive bayes anti-spam email
filtering. 10th Conference of the European Chapter of the Association for
Computational Linguistics,
Shapley, L. S. (1953). A value for n-person games.
Singh, D., & Singh, B. (2020). Investigating the impact of data normalization on
classification performance. Applied Soft Computing, 97, 105524.
Sperandei, S. (2014). Understanding logistic regression analysis. Biochemia medica,
24(1), 12-18.
Theiner, J., Müller-Budack, E., & Ewerth, R. (2022). Interpretable semantic photo
geolocation. Proceedings of the IEEE/CVF Winter Conference on
Applications of Computer Vision,
Tomasevic, N., Gvozdenovic, N., & Vranes, S. (2020). An overview and comparison
of supervised data mining techniques for student exam performance
prediction. Computers & Education, 143, 103676.
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021). Evaluating
XAI: A comparison of rule-based and example-based explanations. Artificial
Intelligence, 291, 103404.
Wolff, A., Zdrahal, Z., Nikolov, A., & Pantucek, M. (2013). Improving retention:
predicting at-risk students by analysing clicking behaviour in a virtual learning
environment. Proceedings of the third international conference on learning
analytics and knowledge,
Yeh, C.-K., Hsieh, C.-Y., Suggala, A., Inouye, D. I., & Ravikumar, P. K. (2019). On
the (in) fidelity and sensitivity of explanations. Advances in neural
information processing systems, 32.
Zhang, J., Bargal, S. A., Lin, Z., Brandt, J., Shen, X., & Sclaroff, S. (2018). Top-down neural attention by excitation backprop. International Journal of Computer
Vision, 126(10), 1084-1102.
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021). Evaluating the quality of
machine learning explanations: A survey on methods and metrics. Electronics,
10(5), 593.
Zitnik, M., Nguyen, F., Wang, B., Leskovec, J., Goldenberg, A., & Hoffman, M. M.
(2019). Machine learning for integrating data in biology and medicine:
Principles, practice, and opportunities. Information Fusion, 50, 71-91.
Braunstein, A. W., Lesser, M. N., & Pescatrice, D. R. (2008). The Impact of a
Program for the Disadvantaged on Student Retention. College Student Journal,
42(1).
Eckles, J. E., & Stradley, E. G. (2012). A social network analysis of student retention
using archival data. Social Psychology of Education, 15(2), 165-180.
Hendel, D. D. (2007). Efficacy of participating in a first-year seminar on student
satisfaction and retention. Journal of College Student Retention: Research,
Theory & Practice, 8(4), 413-423.
Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied linear statistical
models. In: McGraw-Hill New York.
Cremer, C. Z. (2021). Deep limitations? Examining expert disagreement over deep
learning. Progress in Artificial Intelligence, 10, 449-464.
O. H.T. Lua, A. Y.Q. Huang, B. Flanagan, H. Ogata, and S. J.H. Yang, “A Quality
Data Set for Data Challenge: Featuring 160 Students’ Learning Behaviors and
Learning Strategies in a Programming Course,” Asia-Pacific Society for
Computers in Education, vol. 30, p. 10, 2022.
指導教授 楊鎮華(Jhen-Hua Yang) 審核日期 2023-7-5
推文 facebook   plurk   twitter   funp   google   live   udn   HD   myshare   reddit   netvibes   friend   youpush   delicious   baidu   
網路書籤 Google bookmarks   del.icio.us   hemidemi   myshare   

若有論文相關問題,請聯絡國立中央大學圖書館推廣服務組 TEL:(03)422-7151轉57407,或E-mail聯絡  - 隱私權政策聲明