本研究不僅提升了AI音樂視頻生成技術的實用性，同時也為相關領域的未來研究提供了新的方向。我們的系統顯著提高了音樂和動畫之間的連接性，並能更準確地反映出用戶的文化和情感需求，這對於推動文化多樣性的表達和保護具有重要意義。 ;Although existing music generation platforms are capable of autonomously creating diverse musical compositions, they frequently fail to integrate music with animation effectively, particularly in accurately reflecting specific cultural attributes and emotions. To address this issue, we have employed Large Generative Pre-trained Models (LGPM) and Video Latent Diffusion Models (video LDM), both of which have shown considerable potential in technological innovation. At the heart of our system is a semantically-driven module for generating music and animations, which accurately produces culturally distinctive tracks and corresponding animations based on user text prompts.
Our experiments demonstrate that the enhanced capability of Large Language Models (LLMs) to analyze and understand natural language significantly improves the thematic and emotional accuracy of the generated content. Additionally, we focused on enhancing the visual quality of animations, particularly in terms of dynamic coherence and reducing visual distortions. To further optimize the quality and efficiency of generated animations, we integrated Latent Consistency Models (LCMs), which significantly reduce the steps required for generating keyframes from 20 to 4 while maintaining high visual quality.
This research not only advances the practicality of AI-driven music video generation technologies but also opens new directions for future research in the field. Our system significantly improves the connectivity between music and animations, and more accurately reflects users′ cultural and emotional needs, which is crucial for promoting the expression and preservation of cultural diversity.
