Smart Audio to MIDI conversion technology
OpenUtau's audio transcription module employs deep learning algorithms to analyze vocal frequencies and automatically generate corresponding note sequences. At the technical level, the system first determines the fundamental frequency contour through FFT spectral analysis, then uses a pre-trained CNN network to identify phoneme boundaries, and finally outputs MIDI data with lyrics markers. Tests have shown that the transcription accuracy of 85% for clean singing audio exceeds the base mode of Melodyne and other professional tools. The latest transcription model can be installed via "Tools > Install Dependency", and it takes an average of 60 seconds (depending on CPU performance) to process 1 minute of audio. This feature is especially useful for digitizing singing from old recordings, rapid score learning, and assisted composition for musicians with disabilities. Future versions are planned to include polyphonic separation technology to further enhance the processing of complex audio.
This answer comes from the articleOpenUtau: free open source song synthesis editing toolThe
English 
简体中文 
日本語 
Deutsch 
Português do Brasil