應用深度學習於藥品後市場監督：Twitter文本分類任務

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：29

、訪客IP：3.149.214.52

姓名

黄堅洽(Jian-Cia Huang) 查詢紙本館藏

畢業系所

資訊管理學系

論文名稱

應用深度學習於藥品後市場監督：Twitter文本分類任務

相關論文

★ 不動產仲介業銷售住宅類別之成交預測模型—以不動產仲介S公司為例	★ 應用文字探勘技術建構預測客訴問題類別機器學習模型
★ 以機器學習技術建構顧客回購率預測模型：以某手工皂原料電子商務網站為例	★ 以機器學習建構股價預測模型：以台灣股市為例
★ 以機器學習方法建構財務危機之預測模型：以台灣上市櫃公司為例	★ 運用資料探勘技術於股票填息之預測模型：以台灣股市上市公司為例
★ 運用資料探勘技術優化次世代防火牆規則之研究	★ 應用資料探勘技術於電子病歷文本中識別相關新資訊
★ 運用電子病歷與資料探勘技術建構腦中風病人心房顫動預測模型	★ 考量特徵選取與隨機森林之遺漏值填補技術
★ 電子病歷縮寫消歧與一對多分類任務	★ 運用Meta-path與注意力機制改善個人化穿搭推薦
★ 運用機器學習技術建構核保風險預測模型：以A公司為例	★ 風扇壽命預測使用大數據分析－以 X 公司為例
★ 使用文字探勘與深度學習技術建置中風後肺炎之預測模型	★ 利用文字探勘技術分析評論特徵因子對於體驗品評論有益性之影響：以IMDb 為例

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2026-8-1以後開放)

摘要(中)

處方用藥是醫生每天面對每位患者需要處理的任務，當醫生開立藥品給患者服用時需了解此類藥品潛在的副作用，而一項藥物以正常劑量用於人類疾病的預防、診斷治療或是改變其生理功能，但卻出現有害且非預期的結果即為藥物不良反應(Adverse Drug Reaction, ADR)。根據美國衛生公共福利部表示「藥物不良反應」佔每年住院總人數1/3。現今社群媒體使用者越來越多，Twitter每天發文數高達6,500萬則推文及Facebook用戶現今也已超過5億人在使用。
本研究針對44個治療過動症患者的品牌仿製藥及其他81種藥物做推文收集，共收錄5,729筆用戶推文，進行前處理後做特徵擷取，並以各單詞於詞嵌入模型中產生的文字向量作為自變數，由專家們對推文內容進行判斷是否含有藥物不良反應之訊號，以其結果作為依變數，搭配深度學習之架構與預訓練詞嵌入模型──BERT (Bidirectional Encoder Representations from Transformers)及其分支模型BioBERT (Biomedical Bidirectional Encoder Representations from Transformers)、Bio + Clinical BERT及RoBERTa進行BiLSTM (Bidirectional Long Short-Term Memory)模型的分類任務訓練。
進行資料平衡處理後，搭配不同的詞向量合併方式Average及Concat進行BiLSTM模型的訓練，結合Earlystop避免過擬合，找出應用於有關ADR推文最適合之預訓練詞嵌入模型。
本研究發現BERT、BioBERT、Bio + Clinical BERT及RoBERTa等預訓練詞嵌入模型即使於資料不平衡之資料集中建立ADR預測模型，其模型準確率均可接近55%甚至更高，且BERT預訓練詞嵌入模型以Concat方式合併詞向量，於Random Undersampling或是Random Oversampling進行模型訓練，均獲得更好的ADR預測能力。

摘要(英)

Prescription medication is a task that doctors face each patient every day. When doctors prescribe drugs to patients, they need to understand the potential side effects of such drugs. A drug is used in normal doses for the prevention, diagnosis and treatment of human diseases. It is to change its physiological function, but the harmful and unexpected result is named Adverse Drug Reaction (ADR). According to the US Department of Health and Public Welfare, "adverse drug reactions" account for 1/3 of the total number of hospitalizations each year. Nowadays, social media users are becoming more and more developed. Twitter posts up to 65 million tweets every day and more than 500 million Facebook users are using it now.
In this study, 44 brand-name generic drugs and 81 other drugs used for the treatment of patients with ADHD were collected for tweet dataset. A total of 5,729 tweets were collected for pre-processing and feature extraction to obtain the text generated by each word in the word embedding model was used as an independent variable. Experts judged whether the tweet content contained adverse drug reactions combined with deep learning architecture and pre-trained word embedding model－BERT (Bidirectional Encoder Representations from Transformers) and its branch models BioBERT (Biomedical Bidirectional Encoder Representations from Transformers), Bio + Clinical BERT and RoBERTa for BiLSTM (Bi-directional Long Short -Term Memory) model classification task training.
After the data is balanced, the BiLSTM model is trained with different word vector merging methods "Average" and "Concat", combined with Earlystop to avoid over-fitting mechanism to find out the most suitable pre-trained word embedding model for ADR tweets.
This study found that even if pre-trained word embedding models such as BERT, BioBERT, Bio + Clinical BERT, and RoBERTa build an ADR prediction model in a imbalanced dataset ,the precision of the model can be close to 55% or even higher, and BERT pre-trained words The embedding model merges word vectors in ′Concat′ method, and performs model training on Random Undersampling or Random Oversampling, both of which obtain better ADR prediction capabilities.

關鍵字(中)

★ 深度學習
★ 自然語言處理
★ 藥物警戒
★ 資訊分類
★ 藥物不良反應

關鍵字(英)

★ Deep Learning
★ Natural Language Processing
★ Pharmacovigilance
★ Information Classification
★ Adverse Drug Reaction

論文目次

摘要 VII
Abstract VIII
誌謝 X
目錄 XI
一、前言 1
1-1 研究背景 1
1-2 研究動機 3
1-3 研究目的 5
二、文獻探討 6
2-1 ADR相關研究演進 6
2-1-1 機器學習應用於ADR研究 6
2-1-2 深度學習應用於ADR研究 7
2-2 以社群文本進行ADR研究 8
三、研究方法 12
3-1 研究架構與流程 12
3-2 資料收集 13
3-3 資料前處理 14
3-4 產生文本的詞向量 15
3-4-1 Word2vec 16
3-4-2 BERT 18
3-4-3 BioBERT 20
3-4-4 Bio + Clinical BERT 21
3-4-5 RoBERTa 22
3-5 分類模型 23
3-5-1 長短期記憶(Long-Short Term Memory, LSTM) 23
3-5-2 雙向長短期記憶(Bidirectional Long-Short Term Memory, BiLSTM) 25
3-6 實驗設計 26
3-7 實驗驗證與評估指標 31
四、實驗建構與評估 34
4-1 實驗結果與評估 34
4-1-1 實驗一：資料類別不平衡、詞向量合併方式Average 34
4-1-2 實驗二：資料類別平衡處理(Random Undersampling)、詞向量合併方式Average 35
4-1-3 實驗三：資料類別平衡處理(Random Oversampling)、詞向量合併方式Average 37
4-1-4 實驗四：資料類別平衡處理(Random Undersampling)、詞向量合併方式Concat 38
4-1-5 實驗五：資料類別平衡處理(Random Oversampling)、詞向量合併方式:Concat 39
4-1-6 實驗六：資料類別平衡處理(Random Undersampling)、詞向量合併方式:Average／Concat 40
4-1-7 實驗七：資料類別平衡處理(Random Oversampling)、詞向量合併方式:Average／Concat 41
4-2 討論 44
五、結論與未來方向 46
5-1 研究結論 46
5-2 研究限制 47
5-3 未來研究方向 47
參考文獻 49

參考文獻

中文文獻
108 年度國內上市後藥品不良反應通報案例分析 (. (n.d.).
英文文獻
1972WHO.pdf. (n.d.).
Ahmad, S. R. (2003). Adverse drug event monitoring at the food and drug administration. Journal of General Internal Medicine, 18(1), 57–60. https://doi.org/10.1046/j.1525-1497.2003.20130.x
Akhtyamova, L., Cardiff, J., &Alexandrov, M. (2017). Adverse drug extraction in twitter data using convolutional neural network. Proceedings - International Workshop on Database and Expert Systems Applications, DEXA, 2017-Augus, 88–92. https://doi.org/10.1109/DEXA.2017.34
Alex Graves, A. M. and G. H. (2013). Speech Recognition with Deep Recurrent Neural Networks , Department of Computer Science, University of Toronto. Department of Computer Science, University of Toronto, 3(3), 45–49. Retrieved from https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?tp=&arnumber=6638947&ref=aHR0cHM6Ly9pZWVleHBsb3JlLmllZWUub3JnL2Fic3RyYWN0L2RvY3VtZW50LzY2Mzg5NDc/Y2FzYV90b2tlbj1OQUo1VFJxWk5JRUFBQUFBOmtPZmdDbS00NGhqaGI2N3dMd2JrU3lSaEdJREhBWnpMSkxoT201Um5YMXR0S0poUDAtM2hkbT
Alsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., &McDermott, M. (2019). Publicly Available Clinical. 72–78. https://doi.org/10.18653/v1/w19-1909
Alsentzer, E., Murphy, J. R., Boag, W., Weng, W.-H., Jin, D., Naumann, T., &McDermott, M. B. A. (2019). Publicly Available Clinical BERT Embeddings. Retrieved from http://arxiv.org/abs/1904.03323
Breden, A., &Moore, L. (2020). Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling. ArXiv, 2015.
Chandak, P., &Tatonetti, N. P. (2020). Using Machine Learning to Identify Adverse Drug Effects Posing Increased Risk to Women. Patterns, 1(7), 100108. https://doi.org/10.1016/j.patter.2020.100108
Cocos, A., Fiks, A. G., &Masino, A. J. (2017). Deep learning for pharmacovigilance: Recurrent neural network architectures for labeling adverse drug reactions in Twitter posts. Journal of the American Medical Informatics Association, 24(4), 813–821. https://doi.org/10.1093/jamia/ocw180
Devlin, J., Chang, M. W., Lee, K., &Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, 1(Mlm), 4171–4186.
Edwards, B. J., Bunta, A. D., Lane, J., Odvina, C., Rao, D. S., Raisch, D. W., …Stern, P. H. (2013). Bisphosphonates and Nonhealing Femoral Fractures: Analysis of the FDA Adverse Event Reporting System (FAERS) and International Safety Efforts. The Journal of Bone and Joint Surgery-American Volume, 95(4), 297–307. https://doi.org/10.2106/jbjs.k.01181
Fan, B., Fan, W., Smith, C., &Garner, H. “Skip.” (2020). Adverse drug event detection and extraction from open data: A deep learning approach. Information Processing and Management, 57(1), 102131. https://doi.org/10.1016/j.ipm.2019.102131
Gurulingappa, H., Mateen-Rajpu, A., &Toldo, L. (2012). Extraction of potential adverse drug events from medical case reports. Journal of Biomedical Semantics, 3(1), 1–10. https://doi.org/10.1186/2041-1480-3-15
Hakkarainen, K. M., Hedna, K., Petzold, M., &Hägg, S. (2012). Percentage of patients with preventable adverse drug reactions and preventability of adverse drug reactions - a meta-analysis. PLoS ONE, 7(3), 11–13. https://doi.org/10.1371/journal.pone.0033236
Hochreiter, S. (1997). Long Short-Term Memory. 1780, 1735–1780.
Korkontzelos, I., Nikfarjam, A., Shardlow, M., Sarker, A., Ananiadou, S., &Gonzalez, G. H. (2016). Analysis of the effect of sentiment analysis on extracting adverse drug reactions from tweets and forum posts. Journal of Biomedical Informatics, 62, 148–158. https://doi.org/10.1016/j.jbi.2016.06.007
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., &Kang, J. (2020). BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4), 1234–1240. https://doi.org/10.1093/bioinformatics/btz682
Li, F., Liu, W., &Yu, H. (2018). Extraction of Information Related to Adverse Drug Events from Electronic Health Record Notes: Design of an End-to-End Model Based on Deep Learning. JMIR Medical Informatics, 6(4), e12159. https://doi.org/10.2196/12159
Li, H., Guo, X. J., Ye, X. F., Jiang, H., Du, W. M., Xu, J. F., …He, J. (2014). Adverse drug reactions of spontaneous reports in Shanghai pediatric population. PLoS ONE, 9(2), 1–6. https://doi.org/10.1371/journal.pone.0089829
Lin, C., Miller, T., Dligach, D., Bethard, S., &Savova, G. (2019). A BERT-based Universal Model for Both Within- and Cross-sentence Clinical Temporal Relation Extraction. Proceedings of the 2nd Clinical Natural Language Processing Workshop, 2, 65–71.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., …Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. (1). Retrieved from http://arxiv.org/abs/1907.11692
Lopez-Gonzalez, E., Herdeiro, M. T., &Figueiras, A. (2009). Determinants of under-reporting of adverse drug reactions: A systematic review. Drug Safety, 32(1), 19–31. https://doi.org/10.2165/00002018-200932010-00002
McMaster, C., Liew, D., Keith, C., Aminian, P., &Frauman, A. (2019). A Machine-Learning Algorithm to Optimise Automated Adverse Drug Reaction Detection from Clinical Coding. Drug Safety, 42(6), 721–725. https://doi.org/10.1007/s40264-018-00794-y
Mikolov, T., Chen, K., Corrado, G., &Dean, J. (2013). Efficient estimation of word representations in vector space. 1st International Conference on Learning Representations, ICLR 2013 - Workshop Track Proceedings, 1–12.
Mohsen, A., Tripathi, L. P., &Mizuguchi, K. (2020). Deep Learning Prediction of Adverse Drug Reactions Using Open TG-GATEs and FAERS Databases. Retrieved from http://arxiv.org/abs/2010.05411
National Council of State Boards of Nursing. (2011). White paper: a nurse’s guide to the use of social media. The Journal of Practical Nursing, 61(3), 3–9.
Nikfarjam, A., Sarker, A., O’Connor, K., Ginn, R., &Gonzalez, G. (2015). Pharmacovigilance from social media: Mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. Journal of the American Medical Informatics Association, 22(3), 671–681. https://doi.org/10.1093/jamia/ocu041
Powell, G. E., Seifert, H. A., Reblin, T., Burstein, P. J., Blowers, J., Menius, J. A., …Dasgupta, N. (2016). Social Media Listening for Routine Post-Marketing Safety Surveillance. Drug Safety, 39(5), 443–454. https://doi.org/10.1007/s40264-015-0385-6
Sarker, A., Ginn, R., Nikfarjam, A., O’Connor, K., Smith, K., Jayaraman, S., …Gonzalez, G. (2015). Utilizing social media data for pharmacovigilance: A review. Journal of Biomedical Informatics, 54, 202–212. https://doi.org/10.1016/j.jbi.2015.02.004
Sarker, A., Nikfarjam, A., &Gonzalez, G. (2016). Social media mining shared task workshop. Pacific Symposium on Biocomputing, 581–592. https://doi.org/10.1142/9789814749411_0054
Scepanovic, S., Martin-Lopez, E., Quercia, D., &Baykaner, K. (2020). Extracting medical entities from social media. ACM CHIL 2020 - Proceedings of the 2020 ACM Conference on Health, Inference, and Learning, 170–181. https://doi.org/10.1145/3368555.3384467
Sousa, D., Lamurias, A., &Couto, F. M. (2019). A silver standard corpus of human phenotype-gene relations. NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, 1, 1487–1492. https://doi.org/10.18653/v1/n19-1152
Tapi Nzali, M. D., Bringay, S., Lavergne, C., Mollevi, C., &Opitz, T. (2017). What Patients Can Tell Us: Topic Analysis for Social Media on Breast Cancer. JMIR Medical Informatics, 5(3), e23. https://doi.org/10.2196/medinform.7779
Tatonetti, N. P., Denny, J. C., Murphy, S. N., Fernald, G. H., Krishnan, G., Castro, V., …Altman, R. B. (2011). Detecting drug interactions from adverse-event reports: Interaction between paroxetine and pravastatin increases blood glucose levels. Clinical Pharmacology and Therapeutics, 90(1), 133–142. https://doi.org/10.1038/clpt.2011.83
Viscounty, B. P., &Barry, J. L. (2010). How Discoverable Is Social Media Content?
Yang, C. C., Jiang, L., Yang, H., &Tang, X. (2012). Detecting Signals of Adverse Drug Reactions from Health Consumer Contributed Content in Social Media. Hi-Kkd ’12, (May 2016).
Yang, C. C., Yang, H., Jiang, L., &Zhang, M. (2012). Social media mining for drug safety signal detection. International Conference on Information and Knowledge Management, Proceedings, 33–40. https://doi.org/10.1145/2389707.2389714

指導教授

胡雅涵(Ya-Han Hu)

審核日期

2021-7-27

推文