YuE Pioneers New Model of Open Source Generation of Lyrics to Full Songs
YuE does represent an important breakthrough in current music generation technology. As an open source base model, it realizes for the first time the end-to-end generation capability from lyrics to full songs. While traditional music generation models often only produce short snippets or purely accompanied music, YuE is capable of generating complete songs of up to several minutes in length, including lead vocals and full accompaniment.
This innovation addresses three key challenges in the field of music AI: first, the problem of long contextual coherence, which is maintained by a dual-splitting technique and a phased training scheme to maintain the consistency of the music structure; second, the problem of distorted linguistic content, which is ensured by the use of lyrics-chained thought generation to ensure a natural combination of lyrics and melody; and lastly, the problem of data scarcity, which is reduced by the use of semantically-enhanced audio splitters to reduce the reliance on parallel data.
The open source nature of the model makes it extremely valuable for both scientific research and commercial applications. Developers can use the pre-trained model directly for creation, as well as secondary development based on the open source code, which is important for promoting the ecological development of music AI.
This answer comes from the articleYuE: Transforms lyrics into a base model of a complete song, supporting a wide range of musical stylesThe
