我們認為動物行為數據蒐集和深度學習對於更進一步之相關研究探討是非常重要的。;In this dissertation, we proposed a new approach for learning features of animal activity videos. To our best knowledge, there have not any previous work on this problem due to the lack of animal videos. Our first contribution is creating a whole new animal action data which must be difficult to classify in several aspects. The data does not contain only one pet type as some related works, but has two types of cat and dog in similar activities. Furthermore, the data natural videos which are recorded in daily life in variety conditions of light, weather and scene. Thus, our data really challenge any model to do prediction and classification task. Our second contribution is investigating abilities of two Deep Learning models on classifying our own dataset. We implement two spatial Long Short-term Memory architectures LRCN and Convolutional-LSTM in Torch 7 framework. Our design makes adapting the networks easy and convenient. We trained our architectures on animal action dataset and discovered that they have potentials to learn meaning features even of difficult pet videos. The networks get a quite high result on classification task with 65-70% of accuracy. We believe that the animal action dataset and Deep Learning are essential for further studying with more critical requirements.