| dc.description.abstract | In healthcare, industrial, and biometrics fields where obtaining a large amount of data is difficult, how to perform classification through deep learning using neural networks and a limited amount of data has become a problem remaining to be resolved. The interpretability of a neural network is another imperative topic. Previous studies have achieved effective recognition through machine learning using a small amount of data. However, real-time recognition still requires a large amount of power and computation resources as well as expensive hardware. Various problems concerning optimization also exist. In addition, such recognition processes result in undesirable interpretability and convergence. Therefore, this study proposed a widely applicable and interpretable deep multimodal neural network. Colors, textures, and shapes were used as the main features, and statistical and quantization methods were adopted to extract simple and identifiable multimodal features. Finally, the complementary characteristics of the extracted features were employed to perform classification. A genetic algorithm was used to optimize the parameters used in the deep multimodal neural network and therefore improve the recognition accuracy and achieve incremental learning. The experimental results revealed that when a small number of samples was used, the recognition rate of the deep multimodal neural network exceeded that of a shallow convolutional neural network by 12% and that of ResNet50 by 9.33%. Moreover, the deep multimodal neural network also exhibited faster learning and computation speeds than did these two networks, in addition to demonstrating excellent convergence. Deep multimodal neural networks can be applied to embedded systems with limited hardware resources that require real-time operations, thereby achieving the goals of high computation speed, instantaneity, low energy consumption, and low costs. | en_US |