動態多模型融合分析研究

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：73

、訪客IP：3.133.141.71

姓名

蘇俊儒(Jun-Ru Su) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

動態多模型融合分析研究
(Dynamic Ensemble Learning Research)

相關論文

★ 透過網頁瀏覽紀錄預測使用者之個人資訊與性格特質	★ 透過矩陣分解之多目標預測方法預測使用者於特殊節日前之瀏覽行為變化
★ 擴展點擊流：分析點擊流中缺少的使用者行為	★ 關聯式學習：利用自動編碼器與目標傳遞法分解端到端倒傳遞演算法
★ 融合多模型排序之點擊預測模型	★ 分析網路日誌中有意圖、無意圖及缺失之使用者行為
★ 基於自注意力機制產生的無方向性序列編碼器使用同義詞與反義詞資訊調整詞向量	★ 探索深度學習或簡易學習模型在點擊率預測任務中的使用時機
★ 空氣品質感測器之故障偵測--基於深度時空圖模型的異常偵測框架	★ 以同反義詞典調整的詞向量對下游自然語言任務影響之實證研究
★ 結合時空資料的半監督模型並應用於PM2.5空污感測器的異常偵測	★ 藉由權重之梯度大小調整DropConnect的捨棄機率來訓練神經網路
★ 使用圖神經網路偵測 PTT 的低活躍異常帳號	★ 針對個別使用者從其少量趨勢線樣本生成個人化趨勢線
★ 基於雙變量及多變量貝他分布的兩個新型機率分群模型	★ 一種可同時更新神經網路各層網路參數的新技術— 採用關聯式學習及管路化機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

現今多模型的整合大多採用固定策略，在訓練過後，多個基礎模型將以「靜態」的方式作融合，即：不會因為待測樣本的特徵不同而改變基礎模型的融合方式。但在現實的訓練情境中，單一模型可能只擅長於預測特定特徵分佈的樣本。由於各個樣本的特徵分佈不盡相同，只採用「靜態」融合的策略可能是過於天真的。
主流多模型融合大多假設單一基礎模型對不同數據的預測的能力大致相同，本論文想嘗試設計「動態」的融合學習以彌補這個假設可能造成的缺陷。我們已經嘗試了五種不一樣的方法，分別根據(1) 基礎模型判斷類別的機率；(2) 將基礎模型判斷轉換成損失；(3) 根據樣本空間的判斷能力；(4) 根據樣本空間的答對個數；及(5) 加入分類器判斷正確屬
性，以這五種不同的方法來實做「動態」融合。
本文將說明我們設計的五種方法，並在人工生成資料集、車險資料集、Fahsion-MNIST 以及Kuzushiji-MNIST 上的實驗結果。我們設計的融合方法的預測準確度均優於基礎模型，這說明動態的多模型融合是可行的。然而，與理想Model 相比，結果相差甚遠，在訓練額外屬性學習器上還有加強的空間。

摘要(英)

Nowadays most of the ensemble learning methods apply a static strategy to integrate the base learners. After training, base learners are merged in a “static”manner, that is, the basic models will not adapt the fusion
strategy to the different feature distribution of the samples to be tested. However, in a realistic training scenario, a single model may only be good at predicting samples of a particular feature distribution. Since the features of each sample are distributed differently, the strategy of using only “static”fusion may be over-naïve.
The mainstream ensemble models mostly assume that the ability of a single base model to predict different data is roughly the same. This paper attempts to design a“dynamic”ensemble model to compensate for the shortcomings of this hypothesis. We have tried five different methods, based on (1) the category probability predicted by the base learners; (2)the loss of the base learners; (3) the percentage of correctness of the nearby
samples predicted by the base learners ; (4) the numbers of correctness of the nearby samples predicted by the base learners ; and (5) adding extra features about which base learner correctly predict the right label. These
five methods realize the “dynamic”ensemble.
This article will explain the five methods we designed and the experimental results based on a simulated dataset and three real datasets, including the Allstate dataset, the Fashion-MNIST dataset, and the Kuzushiji-MNIST dataset. We found that all five ensemble methods perform better than each of the single base learners. However, if we compare our method with an ideal model, the result is not good enough. Therefore, it may still be possible to improve our methods by training the leaner with extra
features.

關鍵字(中)

★ 多模型融合
★ 動態多模型融合
★ 監督式學習

關鍵字(英)

★ ensemble learning
★ dynamic ensemble learning
★ supervised learning

論文目次

目錄
頁次
摘要iv
Abstract v
誌謝vii
目錄viii
圖目錄x
表目錄xi
一、緒論1
二、背景及相關論文3
三、動態多模型融合法10
3.1 根據基礎模型判斷類別機率做融合................................. 10
3.2 將基礎模型判斷損失做融合.......................................... 11
3.3 根據樣本空間的判斷能力做融合.................................... 13
3.4 根據樣本空間的答對個數做融合.................................... 14
3.5 加入分類器判斷正確屬性作融合.................................... 15
四、實驗結果18
4.1 資料集介紹............................................................... 18
4.1.1 人工生成資料集................................................ 18
4.1.2 車險資料集...................................................... 19
4.1.3 Fashion-MNIST 資料集....................................... 21
4.1.4 Kuzushiji-MNIST 資料集.................................... 23
4.2 實驗設定.................................................................. 24
4.3 人工生成資料集結果................................................... 25
4.4 車險料集結果............................................................ 27
4.5 Fashion-MNIST 結果................................................... 28
4.6 Kuzushiji-MNIST 結果................................................ 31
五、討論與未來發展33
參考文獻35
附錄一37

參考文獻

[1] T. Chen, T. He, M. Benesty, V. Khotilovich, and Y. Tang, “Xgboost: extreme
gradient boosting,” R package version 0.4-2, pp. 1–4, 2015.
[2] R. M. Cruz, R. Sabourin, G. D. Cavalcanti, and T. I. Ren, “Meta-des: A dynamic ensemble
selection framework using meta-learning,” Pattern recognition, vol. 48, no. 5,
pp. 1925–1935, 2015.
[3] C.-C. Chang and C.-J. Lin, “Libsvm: A library for support vector machines,” ACM
transactions on intelligent systems and technology (TIST), vol. 2, no. 3, p. 27, 2011.
[4] S. R. Safavian and D. Landgrebe, “A survey of decision tree classifier methodology,”
IEEE transactions on systems, man, and cybernetics, vol. 21, no. 3, pp. 660–674,
1991.
[5] Y. Freund, R. E. Schapire, et al., “Experiments with a new boosting algorithm,” in
icml, vol. 96, pp. 148–156, Citeseer, 1996.
[6] T. G. Dietterich et al., “Ensemble learning,” The handbook of brain theory and neural
networks, vol. 2, pp. 110–125, 2002.
[7] H.-F. Yu, H.-Y. Lo, H.-P. Hsieh, J.-K. Lou, T. G. McKenzie, J.-W. Chou, P.-H.
Chung, C.-H. Ho, C.-F. Chang, Y.-H. Wei, et al., “Feature engineering and classifier
ensemble for kdd cup 2010,” in KDD Cup, 2010.
[8] A. Niculescu-Mizil, C. Perlich, G. Swirszcz, V. Sindhwani, Y. Liu, P. Melville,
D. Wang, J. Xiao, J. Hu, M. Singh, et al., “Winning the kdd cup orange challenge
with ensemble selection,” in KDD-Cup 2009 Competition, pp. 23–34, 2009.
[9] A. Liaw, M. Wiener, et al., “Classification and regression by randomforest,” R news,
vol. 2, no. 3, pp. 18–22, 2002.
[10] L. Breiman, “Bagging predictors,” Machine learning, vol. 24, no. 2, pp. 123–140,
1996.
[11] T. J. Hastie, “Generalized additive models,” in Statistical models in S, pp. 249–307,
Routledge, 2017.
[12] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning
and an application to boosting,” Journal of computer and system sciences, vol. 55,
no. 1, pp. 119–139, 1997.
[13] L. Breiman et al., “Arcing classifier (with discussion and a rejoinder by the author),”
The annals of statistics, vol. 26, no. 3, pp. 801–849, 1998.
[14] L. Breiman, “Prediction games and arcing algorithms,” Neural computation, vol. 11,
no. 7, pp. 1493–1517, 1999.
[15] T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings
of the 22nd acm sigkdd international conference on knowledge discovery and data
mining, pp. 785–794, ACM, 2016.
[16] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu,
“Lightgbm: A highly efficient gradient boosting decision tree,” in Advances in Neural
Information Processing Systems, pp. 3146–3154, 2017.
[17] A. V. Dorogush, V. Ershov, and A. Gulin, “Catboost: gradient boosting with categorical
features support,” arXiv preprint arXiv:1810.11363, 2018.
[18] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking
machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.
[19] T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha,
“Deep learning for classical japanese literature,” arXiv preprint arXiv:1812.01718,
2018.

指導教授

陳弘軒(Hung-Hsuan Chen)

審核日期

2019-7-2

推文