The important ability of English proficiency grows in tend of globalization, English have obviously been one of the major international languages for communication. However, the English education in Taiwan neglect that human have a natural procedure in learning language, and the teaching method depart from the life-related situation. This phenomenon is made that learners cannot regard English as the important knowledge of the life-related and forms an attitude of exclusion in learning English, feeling of unpractical, lack the learning motivation and so on. Therefore, this study is based on Total Physical Response (TPR) theory and Cognitive Psychology theories as the core and combines the technology of Microsoft Kinect to build an English learning system which is interacting by kinesthetic. Through the feature of kinesthetic interaction in English like body motion or context recognition by Kinect Technology we expect it will effectively enhance students’ motivations and procedure of foreign language learning. This study conducted 39 senior high school students to participate, 20 students were employed as the control group and 19 students were employed as the experiment group. And further explore the relationship between behavior of using system and learning effect in experiment group. Moreover, we use the perceptual learning style which is proposed by Reid (1984) to understand whether students with different perception style would have any difference in learning effect. Finally, we investigated the acceptance and motivation for using this learning system, and propose some difficulty in implementing in the real teaching environment and some solutions. The result show that there are significant different in long-term retention between control group and experimental group. Moreover, the peer interaction activity which is designed by this study would affect the learner’s learning effect. Concerning perception learning style, no matter the learner prefer the Kinesthetic style or not , they can get the better learning effect by using this learning system. Finally, most of the participants who are in experiment group agreed unanimously that this learning system would effective enhance their learning motivation and have a strong intention to use this learning system in the future.