探討記憶體裝置中老化效應對深度神經網路預測準確性之影響：分析策略與解決方法

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：16

、訪客IP：3.15.34.228

姓名

蔡瑩靜(Ying-Jing Tsai) 查詢紙本館藏

畢業系所

電機工程學系

論文名稱

探討記憶體裝置中老化效應對深度神經網路預測準確性之影響：分析策略與解決方法
(Exploring the Impact of Aging Effects in Memory Devices on Deep Neural Networks: Analysis and Solutions)

相關論文

★ 晶圓圖之網格及稀疏缺陷樣態辨識	★ 晶圓圖提取特徵參數錯誤樣態分析
★ 使用聚類過濾策略和 CNN 計算識別晶圓圖瑕疵樣態	★ 新建晶圓圖相似性門檻以強化相似程度辨別能力
★ 一種可動態重新配置的4:2近似壓縮器用於補償老化	★ 一個可靠的靜態隨機存取記憶體內運算結構: 設計指南與耐老化策略研究
★ 一個高效的老化偵測器部屬策略: 基於生成對抗網路的設計方法	★ 考慮電壓衰退和繞線影響以優化電路時序之電源供應網絡精煉策略
★ 適用於提高自旋轉移力矩式磁阻隨機存取記憶體矩陣可靠度之老化偵測與緩解架構設計	★ 8T 靜態隨機存取記憶體之內積運算引擎的老化威脅緩解策略: 從架構及運算角度來提出解決的方法
★ 用於響應穩定性的老化感知平行掃描鏈PUF設計	★ 8T靜態隨機存取記憶體運算的老化檢測和容忍機制：適用於邏輯和 MAC 運算的應用
★ 使用擺置後的設計特徵及極限梯度提升演算法預測繞線後的繞線需求	★ 基於強化學習的晶片佈局規劃的卷積神經網路與圖神經網路融合架構
★ 用於佈線後階段電壓降優化的強化學習框架	★ 多核心系統的老化與瞬態錯誤感知任務部署策略：壽命延長且節能的框架

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2030-1-1以後開放)

摘要(中)

深度神經網絡（Deep Neural Network，DNN）已成為增強安全敏感系統（如自動駕駛汽車和醫療監控）的不可或缺的工具。然而，它們在面對硬體引發的錯誤，特別是老化效應時的可靠性仍然是一個緊迫的問題。本文對老化效應，包括負偏壓溫度不穩定性（Negative Bias Temperature Instability，NBTI）、正偏壓溫度不穩定性（Postivie Bias Temperature Instability，PBTI）和熱載子注入（Hot Carrier Injection，HCI）對深度神經網絡可靠性的影響進行了全面分析，同時提出了實用的解決方案來應對這些挑戰。
我們對記憶體裝置中的老化效應對深度神經網絡可靠性的影響進行了全面的分析，包括模擬老化條件、觀察老化誘發的錯誤（Aging-induced Error，AIE）以及此誘發之錯誤（AIE）發生在DNN上所造成的影響。透過這項研究，我們發現了老化誘發的錯誤對DNN準確性造成大幅度下降。為了解決這個問題，我們研究了兩個有效的解決方案，包括冗餘位元注入（RBI）和降低操作頻率（LOF）。RBI方法只需25%的冗餘位元即可顯著維持 DNN 效能，而LOF方法則能使用比原本多1%的時間成本，維持深度神經網絡的性能。這些解決方案為減輕老化效應所導致的準確性下降提供了實用的解決方案，從而增強了DNN在實際應用中的可靠性。
通過完整的實驗和分析，我們揭示了老化引發的錯誤，並提出了增強深度神經網絡穩固性的方法。我們除了介紹了上述提到的兩種方法來最小化老化效應對深度神經網絡準確性的影響，還提供了一些未來可能可以發展的方向，為在現實應用中保持模型完整性提供了寶貴的見解，特別是在安全關鍵和安全敏感領域。我們的研究結果強調了采取積極措施以增強深度神經網絡韌性的重要性，為未來研究提供了基礎，旨在確保DNN在面對不斷變化的挑戰時的穩定性。

摘要(英)

Deep Neural Networks (DNNs) have become indispensable in bolstering security-sensitive systems such as autonomous vehicles and medical monitoring. However, their reliability in the face of hardware-induced errors, particularly aging effects, remains a pressing concern. This work conducts a thorough investigation into the repercussions of aging effects, including Negative Bias Temperature Instability (NBTI), Positive Bias Temperature Instability (PBTI), and Hot Carrier Injection (HCI), on DNN reliability while proposing practical solutions to counteract these challenges.
We conduct a comprehensive analysis of aging effects within memory devices, involving simulations to replicate aging conditions, observation of aging-induced errors (AIE), and their subsequent application to DNNs. Through this examination, we identified the detrimental impact of aging-induced errors on DNN accuracy. To address this, we study effective remedies, such as redundant bit injection (RBI) and lowering operation frequency (LOF). The RBI method, requiring only 25% redundant bits, significantly sustained DNN performance, while the LOF method minimized performance degradation to just 1% timing overhead. These remedies offer practical solutions to mitigate accuracy degradation caused by aging effects, thereby enhancing the reliability of DNNs in real-world applications.
Through meticulous experimentation and analysis, we uncover the deterministic nature of AIE and propose strategies to enhance DNN resilience. In addition to introducing the two methods mentioned above to minimize the impact of aging effects on DNN accuracy, we also provide some possible future development directions, providing valuable insights for preserving model integrity in real-world applications, particularly in safety-critical and security-sensitive domains. Our findings underscore the critical importance of proactive measures to bolster DNN resilience, laying the groundwork for future research endeavors aimed at ensuring the steadfastness of DNNs in the face of evolving challenges.

關鍵字(中)

★ 老化效應
★ 記憶體

關鍵字(英)

★ Aging effects
★ Memory devices

論文目次

摘要 i
Abstract ii
致謝 iv
Table of Contents v
Table of Figures vii
Table of Tables viii
Chapter 1 Introduction 1
1.1 Robustness of DNNs 2
1.2 Performance of DNNs 3
1.3 Related Works 4
1.4 Contributions 6
Chapter 2 Background 9
2.1 Deep Neural Network 9
2.2 IEEE754 Standard for Floating-Point Arithmetic 11
2.3 8T SRAM Operation 11
2.4 Aging Effects 12
2.4.1 NBTI 13
2.4.2 PBTI 14
2.4.3 HCI 14
Chapter 3 Observations 16
3.1 Aging Effects in Memory Devices 16
3.2 Aging-Induced Error (AIE) 17
3.3 Observations 20
Chapter 4 Explorations 24
4.1 Analysis of Normal Weights 24
4.2 Analysis of Weight Data after Aging 25
Chapter 5 Remedies 28
5.1 Redundant Bit Injection 28
5.2 Lowering Operation Frequency 29
5.3 Discussion 32
Chapter 6 Conclusions 34
Reference 35

參考文獻

[1] Zheng, G., Xiong, Y., Zang, X., Feng, J., Wei, H., Zhang, H., et al. (2019). Learning phase competition for traffic signal control. arXiv preprint arXiv:1905.04722.
[2] Cheng, L., Zhang, C., & Liao, Z. (2018). Intrinsic image transformation via scale space decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 656-665).
[3] Zhuo, C., Luo, S., Gan, H., Hu, J., & Shi, Z. (2019). Noise-aware DVFS for efficient transitions on battery-powered IoT devices. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), 1-1.
[4] Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572.
[5] Barenghi, A., Breveglieri, L., Koren, I., & Naccache, D. (2012). Fault injection attacks on cryptographic devices: Theory, practice, and countermeasures. Proceedings of the IEEE, 100(11), 3056-3076.
[6] Stevenson, M., Winter, R., & Widrow, B. (1990). Sensitivity of feedforward neural networks to weight errors. IEEE Transactions on Neural Networks, 1(1), 71-80.
[7] Cheney, N., Schrimpf, M., & Kreiman, G. (2017). On the robustness of convolutional neural networks to internal architecture and weight perturbations. arXiv preprint arXiv:1703.08245.
[8] Donato, M., et al. (2018). On-chip deep neural network storage with multi-level eNVM. In Proceedings of the 55th Annual Design Automation Conference (p. 169). ACM.
[9] Reagen, B., et al. (2018). Ares: A framework for quantifying the resilience of deep neural networks. In Proceedings of the 55th ACM/ESDA/IEEE Design Automation Conference (DAC) (pp. 1-6). IEEE.
[10] Sha, E., Dong, H., Jiang, W., Zhuge, Q., Chen, X., & Yang, L. (2018). On the design of reliable heterogeneous systems via checkpoint placement and core assignment. In Proceedings of the 2018 Great Lakes Symposium on VLSI (GLSVLSI) (pp. 475-478). ACM.
[11] Liu, Z., Luo, S., Xu, X., Shi, Y., & Zhuo, C. (2018). A multi-level-optimization framework for FPGA-based cellular neural network implementation. Journal of Emerging Technologies in Computing Systems (JETC), 14, 47:1-47:17.
[12] Ziegler, J. F., & Lanford, W. A. (1979). Effect of cosmic rays on computer memories. Science, 206(4420), 776-788.
[13] Yan, Z., Shi, Y., Liao, W., Hashimoto, M., Zhou, X., & Zhuo, C. (2020). When single event upset meets deep neural networks: observations, explorations, and remedies. In 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC) (pp. 163-168). IEEE.
[14] Kerber, A., & Nigam, T. (2018). Bias temperature instability in scaled CMOS technologies: A circuit perspective. Microelectronics Reliability, 81, 31-40.
[15] Chen, K.-L., Saller, S. A., Groves, I. A., & Scott, D. B. (1985). Reliability effects on MOS transistors due to hot-carrier injection. IEEE Transactions on Electron Devices, 32(2), 386-393.
[16] Bansal, A., Rao, R., Kim, J., Zafar, S., Stathis, J., & Chuang, C. (2009). Impacts of NBTI and PBTI on SRAM static/dynamic noise margins and cell failure probability. Microelectronics Reliability, 49, 642-649.
[17] Khalid, U., Mastrandrea, A., & Olivieri, M. (2015). “Effect of NBTI/PBTI aging and process variations on write failures in MOSFET and FinFET flip-flops,” Microelectron. Reliab., 55, 2614-2626.
[18] Chenyaofo. (2021). “pytorch-cifar-models”, Github, https://github.com/chenyaofo/pytorch-cifar-models
[19] Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images. (pp. 32-33).
[20] IEEE. (2008). IEEE standard for floating-point arithmetic. IEEE Std 754-2008, 1-70.
[21] Tudor, B., et al. (2010). MOSRA: An efficient and versatile MOS aging modeling and reliability analysis solution for 45nm and below. In Proceedings of the 10th IEEE International Conference on Solid-State and Integrated Circuit Technology.
[22] Liu, W., & Chang, C.-H. (2019). Analysis of circuit aging on accuracy degradation of deep neural network accelerator. In 2019 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1-5). IEEE.
[23] “IEEE 754”. Wikipedia. Access: Apr 20, 2024. [Online]. Available: https://en.wikipedia.org/wiki/IEEE_754
[24] “10 intelligent dog breeds.” Nylabone. Access: Apr 19, 2024 [Online]. Available: https://www.nylabone.com/dog101/10-intelligent-dog-breeds
[25] “What are the most surprising fields that rely on artificial intelligence and most people are generally unaware of?” BBN TIMES. Access: May 8, 2024 [Online]. Available: https://www.bbntimes.com/science/what-are-the-most-surprising-fields-that-rely-on-artificial-intelligence-and-most-people-are-generally-unaware-of
[26] “Machine learning in self-driving cars” Medium. Access: Apr 29, 2024 [Online]. Available: https://medium.com/@katalesanket90/machine-learning-in-self-driving-cars-8b5d1c685d3b
[27] “The components of patient monitoring systems” Vantage Medtech. Access: Apr 29, 2024 [Online]. Available: https://vantagemedtech.com/the-components-of-patient-monitoring-systems/

指導教授

陳聿廣(Yu-Guang Chen)

審核日期

2025-1-6

推文