近年來由於人工智慧技術的發展加上行動運算日漸普及，因此終端裝置上支援神經網路運算功能的加速器逐漸成為配備選項之一。本整合型計畫將開發用於終端裝置的人工智慧運算加速器—可重組深度神經網路引擎，預計運用可重組設計來面對不同應用情境所需採用的運算模型；並搭配監督式學習與強化式學習的神經網路，可根據應用來轉換學習能力以提高智慧終端的應用範圍；同時結合類比神經網路與數位神經網路，依據資料精度與網路複雜度需求調配所使用的神經網路型態，以降低功耗。而本子計畫將發展『應用於強化式學習之可重組深度神經網路技術』，強化式學習近來因為在圍棋對弈中屢次擊敗人類的頂尖棋士而大放異彩，透過機械自身的經驗來強化決策的最佳性，因此強化式學習可探索人類專家未知的領域，可用於電腦科學、神經科學、心理學、經濟學、數學與工程等，是未來人工智慧發展不可或缺的一項利器。我們預計研究強化式學習下之基於深度Q網路演算法以及特性，並研究深度Q神經網路的架構以及優先經驗回放記憶體的存取設計技術，針對Q值運算、損失函數後向傳播運算、經驗回放記憶體的存取等技術進行開發，使訓練階段可以達到有效率的經驗記憶體使用方式，並針對以上模組開發可重組式的彈性架構設計。預計從硬體實作的角度來考量演算法的創新設計並評估整合於深度Q網路的可行性或必要性。最後將分析探討關於有限精度效應的影響，考慮正向傳遞與反向傳遞兼容於硬體加速器內，考量強化式學習之推論與學習皆在端點運算的可行性，預計設計一高產出率的強化式學習之可重組深度神經網路，最終將整合於一系統晶片中，實現高效能低功耗的運算引擎。 ;Recently because of the development of artificial intelligence techniques and prevailing mobile computing, incorporating a neural network computing acceleration unit becomes one of the features in the edge devices. This integrated project aims to realize the artificial intelligent computing accelerator, reconfigurable deep neural network engine, for edge devices so that the computing models can be adapted according to the different application scenarios. Both supervised learning and reinforcement learning processing elements are included. Thus, the learning capabilities can be changed and programmed to extend the possible applications. In addition, the combination of analog neuromorphic computing and digital neural processing elements offers the flexibility to alter the computation styles regarding the requirements of precision and network complexity to reduce the power consumption. This sub-project will design and develop the reconfigurable deep neural network techniques for reinforcement learning. Reinforcement learning plays an important role in recent AlphaGo games. It also offers the driving strength to enhance the self-learning capability of artificial intelligent devices. Many important techniques have been proposed and examined in Atari games. We aim to study the deep Q network algorithm and its related properties for reinforcement learning. Also, recent improvements, including double Q network, prioritized experience replay, and actor-critic techniques, will be evaluated to see the necessity and possibility to insert them into the hardware implementation. The read and write access to the experience replay memory will be studied so that the content in the replay memory of a constrained size can be utilized efficiently. A reconfigurable architecture will be designed to support applications with different models. Furthermore, the impact of finite precision effect of the deep Q network will be analyzed and discussed. The forward propagation and backward propagation are also considered in the accelerator to support the feasibility of training and inference in the edge devices. We target at designing a high-throughput reconfigurable deep neural network for reinforcement learning and integrate it in the system on chip so that a high-efficiency and low-power computing engine can be realized.
