深度 Q 網絡學習用於加護病房敗血症治療

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：85

、訪客IP：3.135.208.236

姓名

徐昊(Hao Hsu) 查詢紙本館藏

畢業系所

系統生物與生物資訊研究所

論文名稱

深度 Q 網絡學習用於加護病房敗血症治療
(Deep Q Network Learning for Sepsis Treatment in Intensive Care Unit)

相關論文

★ 發展酵素非限制性全基因體調控因子解析方法	★ 利用健保資料庫探討常見複雜疾病之中草藥處方研究
★ 主觀影響療癒的案例與主觀在醫療重要性的探討	★ 精神分裂症病患與正常人之DNA甲基化網絡的差異
★ 躁鬱症病患的精子之DNA 甲基化的網路分析	★ Cloud-R:以R軟體與雲端技術為基礎的生物統計應用網站
★ 中草藥藥性與中草藥遺傳演化樹之關係	★ 利用階層式叢集及不同分類方法分析人類正常組織特異性基因
★ 由ENCODE計畫分析脫氧核醣核酸酶I與組蛋白修飾	★ 皮膚痣圖片毛髮辨識去除
★ 中醫癌症處方多由癰瘍、和解之劑與寒方組成，並隨氣溫下降而更改組成	★ 主成分分析與叢集分析於DNA微陣列數據前處理的應用與實作
★ 確認與中醫處方有關的環境和社會經濟變數	★ 與中醫處方有關的社會經濟變量關係網絡的確認與分析
★ 開發CNN模型預測學生是否退學— 練習如何建立AI模型以從NGS短序列片段數據中偵測SNP	★ 比較線性模型、多層感知器和卷積神經網絡在回歸分析應用中的性能

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

敗血症是患者感染所引起全身性發炎的嚴重疾病，在重症加護病房是常見的死亡原因，隨著病程的發展，病患生理功能會逐漸受到損害而無法維持正常機能，最後演變成死亡，而不同敗血病患者的治療對於醫療措施會有不同的反應。目前敗血症在臨床上沒有普遍認可的治療方針與指引，治療敗血症患者是富有挑戰性的課題，所以了解患者在特定時間的生理狀態可能是製定有效治療政策的關鍵。現今深度強化學習的應用廣大，可以藉由電腦來執行人類智慧的判斷過程，用來輔助人類執行困難的工作。在我們的研究當中，提出了一種能夠推斷最佳敗血症治療的策略，係利用深度強化學習的方法，為敗血症患者的治療制定具有參考價值的醫療政策，學習到的治療政策可用於幫助重症加護病房的臨床醫生做出醫療決策並提高患者生存的可能性。我們發現與臨床醫師的決策相比，模型政策略優於臨床醫師的決策，且符合臨床醫師實際執行的政策特性分布，可用於為臨床醫生提供敗血症治療決策的輔助支持，協助醫師執行醫療策略。

摘要(英)

Sepsis is a fatal condition of systemic inflammation caused by infection of patients. Sepsis is prevalent a prevalent bring of death in the intensive care units (ICU), costing a hospital jillion. With the development of the sepsis disease, patients’ physiological functions will be gradually damaged, and they can not maintain normal functions well. Treatment of sepsis patients will respond diversely to clinical standards. Currently, there are no generally accepted treatment guidelines for sepsis patients, and treating patients with sepsis can be very challenging. Understanding a sepsis patient’s conditions and physiological state at a specific time may explain developing a worthwhile treatment policy. In our study, we proposed a strategy capable of inferring optimal treatment for sepsis patients, using a deep reinforcement learning method to create a reference medical policy for sepsis patients, and the learned treatment policy can be used to help clinicians in intensive care units make medical decisions and improve the likelihood of patient survival. Deep reinforcement learning is widely used in the medical field, and the algorithm can perform the judgment process of human intelligence to assist humans in performing complex tasks. Our policy is slightly better than the clinician′s policy compared to the clinician′s approach and our study. Finally, our policy conforms to the policy characteristic distribution implemented by the clinician, which can be used to provide the clinician with additional support for sepsis treatment and assist physicians in medical strategies.

關鍵字(中)

★ 機器學習
★ 強化學習
★ 深度 Q 網絡
★ 敗血症
★ 加護病房

關鍵字(英)

★ Machine Learning
★ Reinforcement Learning
★ Deep Q Network
★ Sepsis
★ Intensive Care Unit

論文目次

Chinese Abstrac ii
English Abstract iii
Table of Contents iv
List of Figures v
List of Tables vi
Explanation of Symbols vii
Chapter I Introduction 1
1-1 Machine Learning 1
1-2 Reinforcement Learning 2
1-3 Markov Decision Process 3
1-4 Deep Q Learning 3
1-5 Septicemia 6
Chapter 2 Materials and Methods 8
2-1 MIMIC-III Clinical Database 8
2-2 Process Flow Chart 9
2-3 Feature State Preprocessing 10
2-4 Feature Action Preprocessing 13
2-5 Feature Reward Preprocessing 16
Chapter 3 Results 17
3-1 Deep Q Network Models 17
3-2 Adjustments Appropriately to Increase Model Performance 18
3-3 Policies Evaluation of Different Models 19
3-4 Actions Learned by the Different Models 20
Chapter 4 Discussion and Conclusion 22
4-1 Discussion and Conclusion 22
4-2 Future Works 23
References 24
Appendix A 26
Appendix B 33

參考文獻

1. J. Cohen, et al. Sepsis: a roadmap for future research. Lancet Infectious Diseases, 15(5): 581614, 2006
2. Dellinger, R. Phillip, et al. Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup Surviving Sepsis Campaign, Critical Care Medicine, 41 (2): 580–637, 2013
3. Jason Waechter, et al. Cooperative Antimicrobial Therapy of Septic Shock Database Research Group, et al. Interaction between fluids and vasoactive agents on mortality in septic shock: a multicenter, observational study. Critical care medicine, 42 (10): 2158–2168, 2014.
4. Andrew Rhodes, et al. Surviving sepsis campaign: International guidelines for management of sepsis and septic shock: 2016. Intensive care medicine, 43 (3): 304–377, 2017.
5. D. Silver, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 529: 484–489, 2016.
6. David Polle, et al. Computational Intelligence: A Logical Approach. New York: Oxford University Press. 1998.
7. J. Hu, et al. Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning. IEEE Transactions on Vehicular Technology. 69 (12): 14413-14423, 2020.
8. Bellman, R. A Markovian Decision Process. Journal of Mathematics and Mechanics. 6 (5): 679–684, 1957.
9. Watkins, Christopher JCH, and Peter Dayan. Q-learning. Machine learning 8.3 279-292, 1992.
10. Raghu, Aniruddh, et al. Continuous state-space models for optimal sepsis treatment: a deep reinforcement learning approach. Machine Learning for Healthcare Conference. PMLR 147-163, 2017.
11. Martín Abadi, et al. TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. 2015.
12. Chollet, Fran, et al. Keras. GitHub. 2015
13. Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press. 2018.
14. Hausknecht, Matthew, and Peter Stone. Deep recurrent q-learning for partially observable mdps. 2015 AAAI fall symposium series. 2015.
15. Aniruddh Raghu, et al. Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters. International Conference on Machine Learning (ICML) Workshop on CausalML. 2018
16. Raghu, Aniruddh. Reinforcement learning for sepsis treatment: Baselines and analysis. 2019.
17. Angus, DC; van der Poll, T. Severe sepsis and septic shock. New England Journal of Medicine. 369 (9): 840–51, 2013.
18. Johnson, Alistair, et al. MIMIC-III Clinical Database (version 1.4). PhysioNet. 2016
19. Goldberger, A., et al. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23): e215–e220, 2000
20. Van Buuren, S., Groothuis-Oudshoorn, K. mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software. 45(3), 1-67, 2011.
21. VanValkinburgh, D., Kerndt, C. C., and Hashmi, M. F. Inotropes and vasopressors. StatPearls. 2018.

指導教授

王孫崇(Sun-Chong Wang)

審核日期

2022-7-7

推文