探索深度學習或簡易學習模型在點擊率預測任務中的使用時機

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：45

、訪客IP：3.135.183.190

姓名

楊易哲(Yi-Che Yang) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

探索深度學習或簡易學習模型在點擊率預測任務中的使用時機
(Exploring the usage scenarios of deep learning or simple learning models for click-through rate prediction)

相關論文

★ 透過網頁瀏覽紀錄預測使用者之個人資訊與性格特質	★ 透過矩陣分解之多目標預測方法預測使用者於特殊節日前之瀏覽行為變化
★ 動態多模型融合分析研究	★ 擴展點擊流：分析點擊流中缺少的使用者行為
★ 關聯式學習：利用自動編碼器與目標傳遞法分解端到端倒傳遞演算法	★ 融合多模型排序之點擊預測模型
★ 分析網路日誌中有意圖、無意圖及缺失之使用者行為	★ 基於自注意力機制產生的無方向性序列編碼器使用同義詞與反義詞資訊調整詞向量
★ 空氣品質感測器之故障偵測--基於深度時空圖模型的異常偵測框架	★ 以同反義詞典調整的詞向量對下游自然語言任務影響之實證研究
★ 結合時空資料的半監督模型並應用於PM2.5空污感測器的異常偵測	★ 藉由權重之梯度大小調整DropConnect的捨棄機率來訓練神經網路
★ 使用圖神經網路偵測 PTT 的低活躍異常帳號	★ 針對個別使用者從其少量趨勢線樣本生成個人化趨勢線
★ 基於雙變量及多變量貝他分布的兩個新型機率分群模型	★ 一種可同時更新神經網路各層網路參數的新技術— 採用關聯式學習及管路化機制

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

[檢視]

[下載]

本電子論文使用權限為同意立即開放。
已達開放權限電子全文僅授權使用者為學術研究之目的，進行個人非營利性質之檢索、閱讀、列印。
請遵守中華民國著作權法之相關規定，切勿任意重製、散佈、改作、轉貼、播送，以免觸法。

摘要(中)

點擊率的預測在許多內容導向為主的資訊服務中一直有著非常重要的應用，這類服務如電子商務網站、影音串流平台與社群媒體網站，都會盡可能的將使用者會點擊的內容展示在最顯眼的位子，目的即是為了增加使用者使用服務的時間，使用者使用服務的時間增加自然能夠提升服務帶來的商業效益。
要如何找出使用者感興趣並且會點擊的內容一直以來都是推薦系統領域的研究重點，隨著近年來深度學習的興起與成功，已有許多國際大型公司將自身所提供的內容服務改以基於深度學習架構的推薦系統進行推薦，並且提出自行研發的深度學習模型。
我們發現這些成功應用深度學習的服務往往都有著國際級的超大型規模，相對的區域性中小型規模的服務就鮮少有看到成功使用深度學習的案例，這讓我們不禁懷疑時下流行的深度學習模型用於中小型服務的可行性，於是我們開始使用不同的深層與簡易模型對不同規模的資料進行實驗，我們發現深層模型的確不全然適用於中小規模的服務，但其與簡易模型一樣，有著一種在特定條件下逐漸準確的趨勢，也就是說不同的模型有著各自準確的時機，發現這點後，我們開始於不同時機選擇不同模型進行預測，最終提升了點擊率預測任務整體的準確性。

摘要(英)

Click-through rate prediction has been an essential application in many content-oriented information services, such as e-commerce, video streaming platforms, and social media. These services display contents that users are likely to click in a prominent position. As a result, users may be attracted and spend more time on these services.
With the rise and success of deep learning in recent years, many large international companies have integrated their content services with recommendation systems based on the deep learning framework and proposed their deep learning models. However, it seems only the Internet giants reported successful stories on deep learning-based recommender systems. Consequently, we are suspicious of the feasibility of the deep learning models on small and medium-sized services, so we started experimenting with machine learning models with different complexity and datasets of different sizes. We found that deep learning models and simple models seem to appropriate in different cases. After discovering this, we proposed a model to select a recommendation algorithm based on the given scenario automatically. This selecting model improved the overall accuracy of the click-through rate prediction task.

關鍵字(中)

★ 點擊率預測
★ 推薦系統
★ 深度學習
★ 電子商務

關鍵字(英)

★ Click-Through Rate Prediction
★ Recommender System
★ Deep Learning
★ E-commerce

論文目次

摘要 iv
Abstract v
目錄 vii
圖目錄 ix
表目錄 xi

一、緒論 1

二、相關研究 3
2.1 推薦系統在使用者端的使用情境.............................3
2.2 推薦系統之架構...........................................7
2.2.1 候選(Candidate Generation).............................8
2.2.1.1 多路候選策略.........................................9
2.2.1.2 基於embedding的候選方法.............................10

三、研究方法與流程 13
3.1 候選....................................................14
3.1.1 Word2vec..............................................14
3.1.2 Item2vec..............................................15
3.1.3 生成商品embedding之流程...............................16
3.1.4 透過計算商品相似度進行Top-K的候選.....................17
3.2 多模型排序..............................................17
3.2.1 基於最近鄰居法(k-nearest neighbors)的排序.............18
3.2.2 使用簡易神經網路的排序................................18
3.2.3 使用DIN(Deep Interest Network)模型的排序..............20
3.2.4 使用DIEN(Deep Interest Evolution Network)模型的排序...21
3.3 Switch..................................................23

四、實驗結果與分析 26
4.1 資料集介紹..............................................26
4.1.1 淘寶用戶行為資料集....................................26
4.1.2 台灣電商用戶行為資料集................................27
4.2 實驗流程與細節..........................................29
4.3 實驗結果................................................31
4.3.1 評量指標..............................................31
4.3.2 排序模型實驗結果......................................32
4.3.3 Switch模型實驗結果....................................32
4.4 探討商品於訓練資料中出現次數與預測結果之關係............35
4.5 使用更多的資料去訓練DIN與DIEN...........................38

五、結論與未來展望 40
5.1 結論....................................................40
5.2 未來展望................................................40

參考文獻 41

附錄 A 實驗程式碼 45

參考文獻

[1] Rafael Alencar. 2017. Resampling strategies for imbalanced datasets. Kaggle. (2017). https://www.kaggle.com/rafjaa/resampling-strategies-for-imbalanceddatasets

[2] Alibaba. 2018. X-deeplearning. Github. (2018). https://github.com/alibaba/xdeeplearning

[3] Naomi S Altman. 1992. An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician, 46, 3, 175–185. doi: 10.2307/2685209

[4] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. (2014). arXiv: 1409.0473 [cs.CL]

[5] Oren Barkan and Noam Koenigstein. 2016. Item2vec: neural item embedding for collaborative filtering. 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), (September 2016). doi: 10.1109/mlsp.2016.7738886

[6] Tianqi Chen and Carlos Guestrin. 2016. Xgboost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16). Association for Computing Machinery, San Francisco, California, USA, 785–794. isbn: 9781450342322. doi: 10.1145/2939672.2939785

[7] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS 2016). Association for Computing Machinery, Boston, MA, USA, 7–10. isbn: 9781450347952. doi: 10.1145/2988450.2988454

[8] François Chollet et al. 2015. Keras. (2015). https://keras.io

[9] Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys ’16). Association for Computing Machinery, Boston, Massachusetts, USA, 191–198. isbn: 9781450340359. doi: 10.1145/2959100.2959190

[10] Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep session interest network for click-through rate prediction. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, (August 2019). doi: 10.24963/ijcai.2019/319

[11] David Goldberg, David Nichols, Brian M. Oki, and Douglas Terry. 1992. Using collaborative filtering to weave an information tapestry. Commun. ACM, 35, 12, (December 1992), 61–70. issn: 0001-0782. doi: 10.1145/138859.138867

[12] Mihajlo Grbovic and Haibin Cheng. 2018. Real-time personalization using embeddings for search ranking at airbnb. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). Association for Computing Machinery, London, United Kingdom, 311–320. isbn: 9781450355520. doi: 10.1145/3219819.3219885

[13] Long Guo, Lifeng Hua, Rongfei Jia, Binqiang Zhao, Xiaobo Wang, and Bin Cui. 2019. Buying or browsing?: predicting real-time purchasing intent using attentionbased deep network with multiple behavior. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19). Association for Computing Machinery, Anchorage, AK, USA, 1984–1992. isbn: 9781450362016. doi: 10.1145/3292500.3330670

[14] F. Maxwell Harper and Joseph A. Konstan. 2015. The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst., 5, 4, Article 19, (December 2015), 19 pages. issn: 2160-6455. doi: 10.1145/2827872

[15] Jeff Johnson, Matthijs Douze, and Herve Jegou. 2019. Billion-scale similarity search with gpus. IEEE Transactions on Big Data. issn: 2372-2096. doi: 10.1109/tbdata.2019.2921572

[16] Guillaume Lemaître, Fernando Nogueira, and Christos K. Aridas. 2017. Imbalancedlearn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18, 17, 1–5. issn: 1532-4435. http://jmlr.org/papers/v18/16-365.html

[17] G. Linden, B. Smith, and J. York. 2003. Amazon.com recommendations: item-toitem collaborative filtering. IEEE Internet Computing, 7, 1, 76–80. doi: 10.1109/MIC.2003.1167344

[18] Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H. Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). Association for Computing Machinery, London, United Kingdom, 1930–1939. isbn: 9781450355520. doi: 10.1145/3219819.3220007

[19] Martı́n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: large-scale machine learning on heterogeneous systems. (2015). https://www.tensorflow.org/

[20] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. (2013). arXiv: 1301.3781 [cs.CL]

[21] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (NIPS’13). Curran Associates Inc., Lake Tahoe, Nevada, 3111–3119.

[22] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: machine learning in python. Journal of Machine Learning Research, 12, 85, 2825–2830. issn: 1532-4435. http://jmlr.org/papers/v12/pedregosa11a.html

[23] Radim Řehůřek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of LREC 2010 workshop New Challenges for NLP Frameworks. University of Malta, Valletta, Malta, 46–50. isbn: 2-9517408-6-7

[24] Xin Rong. 2014. Word2vec parameter learning explained. (2014). arXiv: 1411.2738 [cs.CL]

[25] Ying Shan, T. Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep crossing: web-scale modeling without manually crafted combinatorial features. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16). Association for Computing Machinery, San Francisco, California, USA, 255–262. isbn: 9781450342322. doi: 10.1145/2939672.2939704

[26] M. Slaney and M. Casey. 2008. Locality-sensitive hashing for finding nearest neighbors [lecture notes]. IEEE Signal Processing Magazine, 25, 2, 128–131. doi: 10.1109/MSP.2007.914237

[27] 2006. Introduction to data mining. (1st edition). Addison-Wesley. Chapter 8, 500. isbn: 0321321367

[28] TIANCHI. 2018. User behavior data from taobao for recommendation. Website. (May 2018). https://tianchi.aliyun.com/dataset/dataDetail?dataId=649

[29] I. Tomek. 1976. Two modifications of cnn. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6, 11, 769–772. doi: 10.1109/TSMC.1976.4309452

[30] Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. Advances in Information Retrieval, 45–57. issn: 1611-3349. doi: 10.1007/978-3-319-30671-1_4

[31] Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending what video to watch next: a multitask ranking system. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys ’19). Association for Computing Machinery, Copenhagen, Denmark, 43–51. isbn: 9781450362436. doi: 10.1145/3298689.3346997

[32] Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 33, (July 2019), 5941–5948. issn: 2159-5399. doi: 10.1609/aaai.v33i01.33015941

[33] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’18). Association for Computing Machinery, London, United Kingdom, 1059–1068. isbn: 9781450355520. doi: 10.1145/3219819.3219823

指導教授

陳弘軒(Hung-Hsuan Chen)

審核日期

2020-7-30

推文