以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：12

、訪客IP：3.147.58.196

姓名	黃雅若(Ya-Jo Huang)  查詢紙本館藏	畢業系所	統計研究所
論文名稱	(Bayesian Optimization for Hyperparameter Tuning with Robust Parameter Design)
檔案	[Endnote RIS 格式]    [Bibtex 格式]    [相關文章]   [文章引用]   [完整記錄]   [館藏目錄]   [檢視]  [下載] 本電子論文使用權限為同意立即開放。 已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。 請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。
摘要(中)	在機器學習領域中，超參數調整對於深度學習演算法來說是一個很重要的步驟，不同的超參數設定可以直接影響模型效能。而貝氏優化一直是超參數調整的熱門方法，貝氏優化利用迭代的方式，不斷更新先驗與後驗分佈來找出最佳超參數組合。本研究利用貝氏優化與穩健參數設計的概念，提出了一種新的超參數優化方法。在優化過程中，該方法將控制因子及噪音因子(例如：初始權重、訓練樣本的選取)納入考量，以期提高求得最佳超參數組合之準確度。在模擬及實證例子中，依據不同類型的問題，發現所提出的方法會比傳統貝氏優化方法找到更接近真實超參數組合的設定。
摘要(英)	Tuning hyperparameters is crucial to the success of deep learning algorithms because it affects the model performance directly. Therefore, hyperparameter tuning has received great attention. Bayesian optimization has always been a popular option for hyperparameter tuning, which obtains optimal values of hyperparameters in a sequential manner. This thesis presents a new hyperparameter optimization method using the concept of robust parameter design. We identify several noise factors (e.g, initial weights or random splitting training samples) for optimization. Simulations show that the proposed method can find hyperparameter settings that are closer to the real hyperparameter setting.
關鍵字(中)	★ 類神經網路 ★ 超參數優化 ★ 貝氏優化 ★ 穩健參數設計	關鍵字(英)	★ Neural network ★ hyperparameter optimization ★ Bayesian optimization ★ expected improvement ★ robust parameter design
論文目次	中文摘要 i Abstract ii Contents iv List of Figures v List of Tables vi 1 Introduction 1 2 Literature Review 2 3 Research Methodology 4 3.1 Bayesian optimization and Gaussian process . . . . . . . . . . . . . . . . . 4 3.2 Proposed method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4 Simulation 8 4.1 Regression problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2 Classification problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5 Real-World Data Analysis 25 5.1 Real estate valuation dataset for regression problem . . . . . . . . . . . . . 25 5.2 Iris dataset for classification problem . . . . . . . . . . . . . . . . . . . . . 25 6 Conclusions and Future Study 27 References 28
參考文獻	Garrido-Merchán, E.C. and Hernández-Lobato, D. (2020). Dealing with Cate- gorical and Integer-valued Variables in Bayesian optimization with Gaussian Processes, Neurocomputing, 380(7), 20-35. Greenhill, S., Rana, S., Gupta, P., Vellanki, P., and Venkatesh, S. (2020). Bayesian optimization for Adaptive Experimental Design: A Review, IEEE Access, 8, 13937-13948. Hertel, L., Baldi, P., and Gillen, D.L. (2021). Reproducible Hyperparameter optimization, Computational and Graphical Statistics, 31, 84-99. Johnson, Richard A. and Wichern, Dean W. (2007). Applied Multivariate Statis- tical Analysis, Pearson, 149-208. Joseph, V.R. and Delaney, J.D.(2007). Functionally Induced Priors for the Analysis of Experiments, Technology, 49, 1-11. Khaw, J.F.C., Lim, B.S., and Lim, L.E.N. (1995). Optimal design of neural networks using the Taguchi method, Neurocomputing, 3(7), 225–245. Kim, Y.S. and Yum, B.J. (2004). Robust design of multilayer feedforward neural net- works: an experimental approach, Engineering Applications of Artificial Intelligence, 17(3), 249-263. Luong, P., Gupta, S., Nguyen, D., Rana, S., and Venkatesh, S. (2019). Bayesian optimization with Discrete Variables, In Australasian Joint Conference on Artificial Intelligence, 473-484. Midilli, Y.E. and Elevli, S. (2019). Optimization of Neural Networks with Response Surface Methodology: Prediction of Cigarette Pressure Drop, 60th International Scien- tific Conference on Information Technology and Management Science of Riga Technical University (ITMS). Murugan, P. (2017). Hyperparameters optimization in Deep Convolutional Neural Net- work / Bayesian Approach with Gaussian Process Priors, arXiv:1712.07233. Nazghelichi, T., Aghbashlo, M., and Kianmehr, M.H. (2011). Optimization of an artificial neural network topology using coupled response surface methodology and genetic algorithm for fluidized bed drying, Computers and Electronics in Agriculture, 75(1), 84-91. Packianather, M.S., Drake, P.R., and Rowlands, H. (2000). Optimizing the parameters of multilayered feedforward neural networks through Taguchi design of ex- periments, Quality and Reliability Engineering International, 16(6), 461-473. Sato, R., Tanaka, M., and Takeda, A. (2021). A Gradient Method for Multilevel optimization, NeurIPS, arXiv:2105.13954. Sukthomya, W. and Tannock, J. (2005). The training of neural networks to model manufacturing processes, Journal of Intelligent Manufacturing, 16(1), 39-51, 16, 39-51. Sukthomya, W. and Tannock, J. (2005). The optimization of neural network param- eters using Taguchi’s design of experiments approach-an application in manufacturing process modelling, Neural Computing and Applications, 14,337-344. Santner, Thomas J., Williams, Brian J., and Notz, William I. (2003). The Design and Analysis of Computer Experiments, Springer, 216-225. Tsai, J.-T., Chou, J.-H., and Liu, T.-K. (2006). Tuning the Structure and Parame- ters of a Neural Network by Using Hybrid Taguchi-Genetic Algorithm, IEEE Transac- tions on Neural Networks, 17(1), 69-80. Tarik, M.H.M., Omar, M., Abdullah, M.F., and Ibrahim, R. (2018). Optimiza- tion of Neural Network Hyperparameters for Gas Turbine Modeling Using Bayesian optimization, 5th IET International Conference on Clean Energy and Technology (CEAT2018), 1-5. Wang, L., Dernoncourt, F., and Bui, T. (2020). Bayesian optimization for Selecting Efficient Machine Learning Models, CIKM MoST-Rec Workshop. Yangi, S.M. and Lee, G.S. (1999). Neural Network Design by Using Taguchi Method, Journal of Dynamic Systems, Measurement, and Control, 121(3), 560-563. Zhang, X., Chen, X., Yao, L., Ge, C., and Dong, M. (2019). Deep Neural Network Hyperparameter optimization with orthogonal array tuning, Conference on Neural Information Processing,287–295.
指導教授	樊采虹 張明中(Tsai-Hung Fan Ming-Chung Chang)	審核日期	2022-7-21
推文	facebook   plurk   twitter   funp   google   live   udn   HD   myshare   reddit   netvibes   friend   youpush   delicious   baidu
網路書籤	Google bookmarks   del.icio.us   hemidemi   myshare