以非監督式表徵分離學習之邊緣運算裝置低延遲樂曲中人聲轉換架構;Low-latency Singing Conversion Architecture based on Unsupervised Disentanglement Representation Learning for Edge Computing Devices

NCU Institutional Repository > 資訊電機學院 > 通訊工程研究所 > 博碩士論文 > Item 987654321/92345

jsp.display-item.identifier=請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/92345

题名:	以非監督式表徵分離學習之邊緣運算裝置低延遲樂曲中人聲轉換架構;Low-latency Singing Conversion Architecture based on Unsupervised Disentanglement Representation Learning for Edge Computing Devices
作者:	羅崇訓;Lo, Chung-Hsun
贡献者:	通訊工程學系
关键词:	低延遲樂曲中人聲轉換;邊緣運算;分離表徵學習;時序卷積網路;Low-latency Singing Conversion;Edge Computing;Disentanglement Representation Learning;Temporal Convolutional Networks
日期:	2023-08-15
上传时间:	2023-10-04 15:27:27 (UTC+8)
出版者:	國立中央大學
摘要:	在語音處理領域，深度學習技術的進步日益迅速。語音轉換技術是一項結合了語音處理技術的創新，它能夠將一位說話者的聲音特徵轉換成另一位說話者的聲音，同時保持說話內容的不變，從而實現更自然、生動且多樣化的合成語音。這項技術不僅在自動語音系統和虛擬角色等應用中扮演重要角色，還能在情感表達領域中應用，通過改變聲音特徵來傳達不同的情感，從而增強使用者的體驗。　　以音樂為例，音樂作為一種普遍存在且具有強烈情感觸動力的藝術形式，不僅為人們提供娛樂，還能促進交流，反映社會價值觀和文化。然而，音樂中蘊含著大量的資訊，音樂後期製作在保證音樂質量的過程中具有至關重要的作用。所以，本文以音樂轉換技術為核心，通過實驗和技術探討，研究了不同的音樂處理應用。　　然而，音樂轉換技術面臨著一些挑戰，包括如何實現高品質的合成語音，以及準確提取和轉換語音特徵等。儘管大型語言模型能夠有效提取特徵，但在資源有限的設備上應用時，需要考慮模型的輕量化處理。　　因此，本研究深入探討了「音樂後期製作與低延遲樂曲中的人聲轉換」，運用現代科技解決傳統音樂製作中的限制。通過表徵分離學習方式，對聲學資訊進行細節分析，以實現聲音轉換後的結果生成。在低延遲的要求下，我們引入時序卷積網路來取代傳統的循環系列神經網路，以實現低運算且快速的轉換效果。實驗結果顯示，這種架構在轉換品質和處理速度方面均優於傳統的循環系列神經網路。　　通過引入這樣的低延遲樂曲中人聲轉換技術，我們有望解決歌手音高的問題，提高音樂品質和製作效率，從而使錄音室專業人員受益，同時節省時間和資源，增強音樂製作的靈活性。此外，對音樂技術的不斷推進有助於我們更好地了解其潛力和限制，也為未來的音樂製作工具和方法提供新的啟發。 ;In the field of speech processing, the rapid advancement of deep learning technology is increasingly evident. Voice conversion technology is an innovative application that combines speech processing techniques. It allows for the transformation of the acoustic features of one speaker′s voice into those of another speaker, while maintaining the unchanged content of speech. This achievement brings about a more natural, vivid, and diverse synthesized speech. This technology not only plays a crucial role in applications such as automatic speech systems and virtual characters but also finds utility in emotional expression. By altering vocal characteristics, it conveys different emotions, thus enhancing the user experience. 　　Taking music as an example, it is a universally present art form with powerful emotional impact. Music not only provides entertainment but also fosters communication and reflects societal values and culture. However, music contains a wealth of information, and post-production processes play a vital role in ensuring the quality of music. Therefore, this paper centers around music conversion technology, conducting experiments and technical exploration to examine various applications in music processing. 　　Yet, music conversion technology faces several challenges, including achieving high-quality synthetic speech and accurately extracting and converting vocal characteristics. While large-scale language models are effective at feature extraction, their application on resource-constrained devices requires consideration of model light weighting. 　　Hence, this study delves deeply into "Post-Production in Music and Low-Latency Singing Conversion," utilizing modern technology to address constraints in traditional music production. Through disentanglement representation learning, detailed analysis of acoustic information is conducted to achieve the generation of transformed audio. To meet low-latency requirements, we introduce temporal convolutional networks as a replacement for traditional recurrent neural networks, achieving low-computation and rapid transformation effects. Experimental results demonstrate that this architecture outperforms traditional recurrent neural networks in both transformation quality and processing speed. 　　By introducing such low-latency vocal conversion technology in music, we aspire to resolve singer intonation issues, enhance music quality and production efficiency, benefiting studio professionals while saving time and resources, and enhancing music production flexibility. Furthermore, the continuous advancement of music technology contributes to a better understanding of its potential and limitations, providing fresh inspiration for future music production tools and methods.
显示于类别:	[通訊工程研究所] 博碩士論文

文件中的档案:

档案	描述	大小	格式	浏览次数
index.html		0Kb	HTML	127	检视/开启

在NCUIR中所有的数据项都受到原著作权保护.

社群 sharing

数据加载中.....