以作者查詢圖書館館藏 、以作者查詢臺灣博碩士 、以作者查詢全國書目 、勘誤回報 、線上人數:235 、訪客IP:3.145.88.55
姓名 蘇俊儒(Jun-Ru Su) 查詢紙本館藏 畢業系所 資訊工程學系 論文名稱 動態多模型融合分析研究
(Dynamic Ensemble Learning Research)相關論文 檔案 [Endnote RIS 格式] [Bibtex 格式] [相關文章] [文章引用] [完整記錄] [館藏目錄] [檢視] [下載]
- 本電子論文使用權限為同意立即開放。
- 已達開放權限電子全文僅授權使用者為學術研究之目的,進行個人非營利性質之檢索、閱讀、列印。
- 請遵守中華民國著作權法之相關規定,切勿任意重製、散佈、改作、轉貼、播送,以免觸法。
摘要(中) 現今多模型的整合大多採用固定策略,在訓練過後,多個基礎模型將以「靜態」的方式作融合,即:不會因為待測樣本的特徵不同而改變基礎模型的融合方式。但在現實的訓練情境中,單一模型可能只擅長於預測特定特徵分佈的樣本。由於各個樣本的特徵分佈不盡相同,只採用「靜態」融合的策略可能是過於天真的。
主流多模型融合大多假設單一基礎模型對不同數據的預測的能力大致相同,本論文想嘗試設計「動態」的融合學習以彌補這個假設可能造成的缺陷。我們已經嘗試了五種不一樣的方法,分別根據(1) 基礎模型判斷類別的機率;(2) 將基礎模型判斷轉換成損失;(3) 根據樣本空間的判斷能力;(4) 根據樣本空間的答對個數;及(5) 加入分類器判斷正確屬
性,以這五種不同的方法來實做「動態」融合。
本文將說明我們設計的五種方法,並在人工生成資料集、車險資料集、Fahsion-MNIST 以及Kuzushiji-MNIST 上的實驗結果。我們設計的融合方法的預測準確度均優於基礎模型,這說明動態的多模型融合是可行的。然而,與理想Model 相比,結果相差甚遠,在訓練額外屬性學習器上還有加強的空間。摘要(英) Nowadays most of the ensemble learning methods apply a static strategy to integrate the base learners. After training, base learners are merged in a “static”manner, that is, the basic models will not adapt the fusion
strategy to the different feature distribution of the samples to be tested. However, in a realistic training scenario, a single model may only be good at predicting samples of a particular feature distribution. Since the features of each sample are distributed differently, the strategy of using only “static”fusion may be over-naïve.
The mainstream ensemble models mostly assume that the ability of a single base model to predict different data is roughly the same. This paper attempts to design a“dynamic”ensemble model to compensate for the shortcomings of this hypothesis. We have tried five different methods, based on (1) the category probability predicted by the base learners; (2)the loss of the base learners; (3) the percentage of correctness of the nearby
samples predicted by the base learners ; (4) the numbers of correctness of the nearby samples predicted by the base learners ; and (5) adding extra features about which base learner correctly predict the right label. These
five methods realize the “dynamic”ensemble.
This article will explain the five methods we designed and the experimental results based on a simulated dataset and three real datasets, including the Allstate dataset, the Fashion-MNIST dataset, and the Kuzushiji-MNIST dataset. We found that all five ensemble methods perform better than each of the single base learners. However, if we compare our method with an ideal model, the result is not good enough. Therefore, it may still be possible to improve our methods by training the leaner with extra
features.關鍵字(中) ★ 多模型融合
★ 動態多模型融合
★ 監督式學習關鍵字(英) ★ ensemble learning
★ dynamic ensemble learning
★ supervised learning論文目次 目錄
頁次
摘要iv
Abstract v
誌謝vii
目錄viii
圖目錄x
表目錄xi
一、緒論1
二、背景及相關論文3
三、動態多模型融合法10
3.1 根據基礎模型判斷類別機率做融合................................. 10
3.2 將基礎模型判斷損失做融合.......................................... 11
3.3 根據樣本空間的判斷能力做融合.................................... 13
3.4 根據樣本空間的答對個數做融合.................................... 14
3.5 加入分類器判斷正確屬性作融合.................................... 15
四、實驗結果18
4.1 資料集介紹............................................................... 18
4.1.1 人工生成資料集................................................ 18
4.1.2 車險資料集...................................................... 19
4.1.3 Fashion-MNIST 資料集....................................... 21
4.1.4 Kuzushiji-MNIST 資料集.................................... 23
4.2 實驗設定.................................................................. 24
4.3 人工生成資料集結果................................................... 25
4.4 車險料集結果............................................................ 27
4.5 Fashion-MNIST 結果................................................... 28
4.6 Kuzushiji-MNIST 結果................................................ 31
五、討論與未來發展33
參考文獻35
附錄一37參考文獻 [1] T. Chen, T. He, M. Benesty, V. Khotilovich, and Y. Tang, “Xgboost: extreme
gradient boosting,” R package version 0.4-2, pp. 1–4, 2015.
[2] R. M. Cruz, R. Sabourin, G. D. Cavalcanti, and T. I. Ren, “Meta-des: A dynamic ensemble
selection framework using meta-learning,” Pattern recognition, vol. 48, no. 5,
pp. 1925–1935, 2015.
[3] C.-C. Chang and C.-J. Lin, “Libsvm: A library for support vector machines,” ACM
transactions on intelligent systems and technology (TIST), vol. 2, no. 3, p. 27, 2011.
[4] S. R. Safavian and D. Landgrebe, “A survey of decision tree classifier methodology,”
IEEE transactions on systems, man, and cybernetics, vol. 21, no. 3, pp. 660–674,
1991.
[5] Y. Freund, R. E. Schapire, et al., “Experiments with a new boosting algorithm,” in
icml, vol. 96, pp. 148–156, Citeseer, 1996.
[6] T. G. Dietterich et al., “Ensemble learning,” The handbook of brain theory and neural
networks, vol. 2, pp. 110–125, 2002.
[7] H.-F. Yu, H.-Y. Lo, H.-P. Hsieh, J.-K. Lou, T. G. McKenzie, J.-W. Chou, P.-H.
Chung, C.-H. Ho, C.-F. Chang, Y.-H. Wei, et al., “Feature engineering and classifier
ensemble for kdd cup 2010,” in KDD Cup, 2010.
[8] A. Niculescu-Mizil, C. Perlich, G. Swirszcz, V. Sindhwani, Y. Liu, P. Melville,
D. Wang, J. Xiao, J. Hu, M. Singh, et al., “Winning the kdd cup orange challenge
with ensemble selection,” in KDD-Cup 2009 Competition, pp. 23–34, 2009.
[9] A. Liaw, M. Wiener, et al., “Classification and regression by randomforest,” R news,
vol. 2, no. 3, pp. 18–22, 2002.
[10] L. Breiman, “Bagging predictors,” Machine learning, vol. 24, no. 2, pp. 123–140,
1996.
[11] T. J. Hastie, “Generalized additive models,” in Statistical models in S, pp. 249–307,
Routledge, 2017.
[12] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning
and an application to boosting,” Journal of computer and system sciences, vol. 55,
no. 1, pp. 119–139, 1997.
[13] L. Breiman et al., “Arcing classifier (with discussion and a rejoinder by the author),”
The annals of statistics, vol. 26, no. 3, pp. 801–849, 1998.
[14] L. Breiman, “Prediction games and arcing algorithms,” Neural computation, vol. 11,
no. 7, pp. 1493–1517, 1999.
[15] T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings
of the 22nd acm sigkdd international conference on knowledge discovery and data
mining, pp. 785–794, ACM, 2016.
[16] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu,
“Lightgbm: A highly efficient gradient boosting decision tree,” in Advances in Neural
Information Processing Systems, pp. 3146–3154, 2017.
[17] A. V. Dorogush, V. Ershov, and A. Gulin, “Catboost: gradient boosting with categorical
features support,” arXiv preprint arXiv:1810.11363, 2018.
[18] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking
machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.
[19] T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha,
“Deep learning for classical japanese literature,” arXiv preprint arXiv:1812.01718,
2018.指導教授 陳弘軒(Hung-Hsuan Chen) 審核日期 2019-7-2 推文 facebook plurk twitter funp google live udn HD myshare reddit netvibes friend youpush delicious baidu 網路書籤 Google bookmarks del.icio.us hemidemi myshare