| dc.description.abstract | In the meantime while the quality of life promotes continuously and the convenience increase constantly, so many uses and applications rely on the support of technology and exploitation behind. From image to video, and from gesture to action, what we need to face with the succeeding improvement of technology and hardware, is the much better function and effect.
Based on the architecture of deep learning of long short-term memory, we proposed the optical flow attention model. This model do action recognition for videos through the use of optical flow images. In the proposed architecture, each video is separated to frame images, and feed into CNN for feature extraction. Each feature input into the optical flow model followed by the time sequence. The attention model is mainly composed by LSTM, and the characteristic of optical flow attention is that the input feature weighted by the optical flow weight image firstly to highlight the important part of current feature. And the adjusted feature input into LSTM after weighted and produce the recognition result at that time step.
The thesis does dynamical tracing on the important area of image using optical flow image as weights to promote the weights at the important part of feature. In the experiment of action recognition, the optical flow image we proposed grows about 3.6% accuracy compared with the model only use LSTM, and get 2.4% higher compared with the visual attention model we referenced. And we combine the visual attention model with our optical flow attention model, getting 4.5% higher than LSTM and 3.6% higher than the visual attention model. The experiment result shows that using optical flow image as weights brings the effect to capture the discriminate area of action in video, and can complement with visual attention to reach better recognition effect. | en_US |