基於深度學習的長期週期性多變量時間序列預測模型

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：14

、訪客IP：3.12.41.25

姓名

林煥軒(Huan-Syuan Lin) 查詢紙本館藏

畢業系所

資訊工程學系

論文名稱

基於深度學習的長期週期性多變量時間序列預測模型
(Deep Learning-based Multivariate Long-term Periodic Time Series Forecasting Model)

相關論文

★ 影片指定對象臉部置換系統	★ 以單一攝影機實現單指虛擬鍵盤之功能
★ 基於視覺的手寫軌跡注音符號組合辨識系統	★ 利用動態貝氏網路在空照影像中進行車輛偵測
★ 以視訊為基礎之手寫簽名認證	★ 使用膚色與陰影機率高斯混合模型之移動膚色區域偵測
★ 影像中賦予信任等級的群眾切割	★ 航空監控影像之區域切割與分類
★ 在群體人數估計應用中使用不同特徵與回歸方法之分析比較	★ 以視覺為基礎之強韌多指尖偵測與人機介面應用
★ 在夜間受雨滴汙染鏡頭所拍攝的影片下之車流量估計	★ 影像特徵點匹配應用於景點影像檢索
★ 自動感興趣區域切割及遠距交通影像中的軌跡分析	★ 基於回歸模型與利用全天空影像特徵和歷史資訊之短期日射量預測
★ Analysis of the Performance of Different Classifiers for Cloud Detection Application	★ 全天空影像之雲追蹤與太陽遮蔽預測

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2028-7-16以後開放)

摘要(中)

時間序列預測有很高的實用價值。隨預測序列長度增長，預測難度也會越來越高，同時附加價值也愈高，因此提升預測序列的長度是非常顯要的研究議題。在許多實際應用當中，如銷售量預測、用電量預測等等，其目標序列都呈現非常明顯的週期特徵，若能有效利用這個特徵，我們可以提升預測遙遠未來的信心水準，進而增加預測模型實際應用價值。
由於近年深度學習方法在許多研究領域上的成功，我們結合了多個深度序列模型並提出了一個encoder-decoder架構的深度網路模型，希望透過融合不同序列模型所提取的多樣時序特徵以提升模型的綜合時序建模能力。
　　基於對幾個現實序列的觀察，我們發現除了短期的週期特徵以外，這些真實序列也表現出了難以一眼察覺的長期週期特徵或趨勢，這些長期的序列特徵之時間跨度可以長至超越輸入序列長度，使模型無法單從輸入序列提供之線索提取這些隱藏的長期特徵。我們提出了一種日期時間編碼方式，目的是為模型提供更宏觀的時間資訊，以幫助模型提升長期預測準確度。
　　由於深度網路先天的非線性特性，導致我們的模型對於序列的局部尺度不夠敏感，準確度不如預期。為了解決這個問題，我們參考傳統方法的自迴歸模型，增加了一個與模型並行的線性單元，該單元將歷史觀測經線性映射後產生目標預測序列，並保證預測序列內時間步之間的獨立性以維持模型的平行化特性，最後將此單元的輸出與模型相加以得到更準確的預測結果。
　　經過實驗，我們提出的架構在幾個現實時間序列資料集上都取得了不錯的成績，並超越了幾個參測基準方法。我們提出的兩大組件在經過消融實驗後證實能在特定情況下有效提升模型的準確度。

摘要(英)

Deep learning methods has shown superior performance in long-term time series forecasting. For this reason, we propose a deep encoder-decoder neutral network. The model combines multiple deep sequence models, under the hypothesis that integrating diverse temporal features extracted from different sequence models can improve the comprehensive time series modeling capability of the model.
　Upon analyzing some real-world time series, we found that many of them exhibit short-term periodic features plus long-term periodic features or trends that are not easily discernible at a glance. To model these long-term features, we propose a date-time encoding method to provide proposed model with macro time information to help improve long-term prediction accuracy.
　Due to the inherent non-linearity of deep neural networks, our model may not be sensitive enough to the local scale of the input sequence, leading to non-satisfactory performance. To address this issue, we introduce a parallel linear unit to the model. This unit linearly maps the historical observations to generate the target prediction sequence. Finally, the output of this unit is added to the output of the deep model for more accurate prediction results.
　Through experiments, our proposed architecture achieves good results on several real-world time series datasets and outperform several baseline methods. Our two proposed components are proven to effectively improve the accuracy of the model under specific conditions after ablation experiments.

關鍵字(中)

★ 深度學習
★ 時間序列預測
★ 多變量時間序列
★ 週期性時間序列

關鍵字(英)

★ Deep learning
★ Time series forecasting
★ Multivariate timer seires
★ Periodic time series

論文目次

Abstract I
摘要 II
目錄 III
圖目錄 V
表目錄 VI
1 緒論 1
1.1 研究動機 1
1.2 研究目標 2
2 相關研究 3
2.1 相關背景與發展 3
2.2 深度學習方法 4
2.3 神經網路模組 6
2.3.1 卷積層(Convolution Layer) 6
2.3.2 門控循環單元(Gated Recurrent Unit, GRU) 7
2.3.3 層正規化(Layer Normalization) 8
2.3.4 自注意力機制(Self-Attention Mechanism) 9
2.3.5 跨注意力機制(Cross-Attention Mechanism) 10
2.3.6 前饋網路(Feed-Forward Network) 11
2.4 比較與討論 12
3 研究方法 13
3.1 問題描述 13
3.2 模型架構 14
3.2.1 輸入表徵(Input Representation) 15
3.2.2 編碼器(Encoder) 17
3.2.3 解碼器(Decoder) 18
3.2.4 時域線性回歸(Temporal Linear Regression) 20
3.3 資料收集與分析 22
3.3.1 參測資料集 22
3.3.2 資料分析 23
3.3.3 資料前處理與序列分割 25
4 實驗結果 26
4.1 實驗設定 26
4.1.1 基準方法 26
4.1.2 實作細節 27
4.1.3 評估指標 27
4.2 主要結果 28
4.2.1 輸出長度 28
4.2.2 偏移量 31
4.3 消融實驗 32
4.3.1 日期時間編碼 32
4.3.2 旁通線性單元 35
4.4 實驗結果視覺化 38
4.4.1 在EC資料集上呈現 38
4.4.2 在SIC資料集上呈現 40
5 結論與未來展望 42
參考文獻 43
附錄1 Ours-GRU之參數敏感度 46
附錄2 Ours-SA之參數敏感度 49
附錄3 旁通線性單元之參數敏感度 53

參考文獻

[1] L. E. Jeffrey, "Finding structure in time," Cognitive Science, vol. 14, no. 2, pp. 179-211, 1990, doi: https://doi.org/10.1016/0364-0213(90)90002-E.

[2] Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 157-166, March , 1994, doi: 10.1109/72.279181 , ISSN= 1941-0093.

[3] R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training recurrent neural networks," presented at the Proceedings of the 30th International Conference on Machine Learning, Proceedings of Machine Learning Research, 2013. [Online]. Available: https://proceedings.mlr.press/v28/pascanu13.html.

[4] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, Nov , 1997, doi: 10.1162/neco.1997.9.8.1735 , ISSN= 0899-7667.

[5] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling," arXiv preprint arXiv:1412.3555, 2014.

[6] A. Vaswani et al., "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.

[7] G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time series analysis: forecasting and control. John Wiley & Sons, 2015.

[8] E. S. Gardner Jr, "Exponential smoothing: The state of the art," Journal of forecasting, vol. 4, no. 1, pp. 1-28, 1985.

[9] E. S. Gardner Jr, "Exponential smoothing: The state of the art—Part II," International journal of forecasting, vol. 22, no. 4, pp. 637-666, 2006.

[10] R. B. Cleveland, W. S. Cleveland, J. E. McRae, and I. Terpenning, "STL: A seasonal-trend decomposition," J. Off. Stat, vol. 6, no. 1, pp. 3-73, 1990.

[11] E. B. Dagum and S. Bianconcini, Seasonal adjustment methods and real time trend-cycle estimation. Springer, 2016.

[12] C. Meek, D. M. Chickering, and D. Heckerman, "Autoregressive tree models for time-series analysis," in Proceedings of the 2002 SIAM International Conference on Data Mining, 2002: SIAM, pp. 229-244.

[13] K.-j. Kim, "Financial time series forecasting using support vector machines," Neurocomputing, vol. 55, no. 1-2, pp. 307-319, 2003.

[14] N. I. Sapankevych and R. Sankar, "Time series prediction using support vector machines: a survey," IEEE computational intelligence magazine, vol. 4, no. 2, pp. 24-38, 2009.

[15] H. Tyralis and G. Papacharalampous, "Variable selection in time series forecasting using random forests," Algorithms, vol. 10, no. 4, p. 114, 2017.

[16] D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski, "DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, vol. 36, no. 3, pp. 1181-1191, 2020.

[17] A. v. d. Oord et al., "Wavenet: A generative model for raw audio," arXiv preprint arXiv:1609.03499, 2016.

[18] S. Bai, J. Z. Kolter, and V. Koltun, "An empirical evaluation of generic convolutional and recurrent networks for sequence modeling," arXiv preprint arXiv:1803.01271, 2018.

[19] G. Lai, W.-C. Chang, Y. Yang, and H. Liu, "Modeling long-and short-term temporal patterns with deep neural networks," in The 41st international ACM SIGIR conference on research & development in information retrieval, 2018, pp. 95-104.

[20] J. Cheng, K. Huang, and Z. Zheng, "Towards better forecasting by fusing near and distant future visions," in Proceedings of the AAAI Conference on Artificial Intelligence, 2020, vol. 34, no. 04, pp. 3593-3600.

[21] S. Li et al., "Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting," Advances in neural information processing systems, vol. 32, 2019.

[22] H. Zhou et al., "Informer: Beyond efficient transformer for long sequence time-series forecasting," in Proceedings of the AAAI conference on artificial intelligence, 2021, vol. 35, no. 12, pp. 11106-11115.

[23] H. Wu, J. Xu, J. Wang, and M. Long, "Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting," Advances in Neural Information Processing Systems, vol. 34, pp. 22419-22430, 2021.

[24] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017.

[25] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.

[26] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.

[27] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.

[28] C. Olah. "Understanding LSTM Networks." https://colah.github.io/posts/2015-08-Understanding-LSTMs/ (accessed.

[29] J. L. Ba, J. R. Kiros, and G. E. Hinton, "Layer normalization," arXiv preprint arXiv:1607.06450, 2016.

[30] S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International conference on machine learning, 2015: pmlr, pp. 448-456.

[31] Y. Wu and K. He, "Group normalization," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3-19.

[32] A. Trindade. ElectricityLoadDiagrams20112014, doi: https://doi.org/10.24432/C58C86.

[33] C. C. D. o. Transportation. Caltrans Performance Measurement System. [Online]. Available: https://pems.dot.ca.gov/

指導教授

鄭旭詠

審核日期

2023-7-19

推文