Roformer Models STILL broken on Windows - Tested on Multiple Computers

[jet082]

As it says in the title.

I tested on two separate Windows computers now and it does not work properly. It DOES use the GPU, but it takes HOURS for a simple clip, so there's some serious problem with how the data is being moved between GPU/CPU. If you look at the GPU usage below, you'll see the dips in GPU usage in the current version compared to the much more stable usage in my PR back this theory up.

With my PR (#74) it still works on BOTH computers. There is clearly a problem with the current code.

Testing Code

from audio_separator.separator import Separator
separator = Separator()
#model = 'UVR-MDX-NET-Inst_HQ_4.onnx'
model = 'model_bs_roformer_ep_317_sdr_12.9755.ckpt'
separator.load_model(model_filename=model)
output_files = separator.separate('1.wav')
print(output_files)

CURRENT VERSION

Output

INFO - separator - Separator version 0.21.1 instantiating with output_dir: None, output_format: WAV
INFO - separator - Output directory not specified. Using current working directory.
INFO - separator - Operating System: Windows 10.0.22635
INFO - separator - System: Windows Node: NewPC Release: 11 Machine: AMD64 Proc: Intel64 Family 6 Model 151 Stepping 2, GenuineIntel
INFO - separator - Python Version: 3.12.3
INFO - separator - PyTorch Version: 2.3.0+cu121
INFO - separator - FFmpeg installed: ffmpeg version N-116603-gfaf7d37961-g43cde54fc1+4 Copyright (c) 2000-2024 the FFmpeg developers
INFO - separator - ONNX Runtime GPU package installed with version: 1.17.1
INFO - separator - ONNX Runtime CPU package installed with version: 1.17.1
INFO - separator - CUDA is available in Torch, setting Torch device to CUDA
INFO - separator - ONNXruntime has CUDAExecutionProvider available, enabling acceleration
INFO - separator - Loading model model_bs_roformer_ep_317_sdr_12.9755.ckpt...
INFO - mdxc_separator - MDXC Separator initialisation complete
INFO - separator - Load model duration: 00:00:03
INFO - separator - Starting separation process for audio_file_path: 1.wav
  0%|▏  | 2/836 [02:34<17:42:32, 76.44s/it]

Note the estimate of 17:42:32. At this time the GPU usage was almost 100% in Task Manager. However, look at how unstable the usage is.

MY PR (#74)

Output

INFO - separator - Separator version 0.21.1 instantiating with output_dir: None, output_format: WAV
INFO - separator - Output directory not specified. Using current working directory.
INFO - separator - Operating System: Windows 10.0.22635
INFO - separator - System: Windows Node: NewPC Release: 11 Machine: AMD64 Proc: Intel64 Family 6 Model 151 Stepping 2, GenuineIntel
INFO - separator - Python Version: 3.12.3
INFO - separator - PyTorch Version: 2.3.0+cu121
INFO - separator - FFmpeg installed: ffmpeg version N-116603-gfaf7d37961-g43cde54fc1+4 Copyright (c) 2000-2024 the FFmpeg developers
INFO - separator - ONNX Runtime GPU package installed with version: 1.17.1
INFO - separator - ONNX Runtime CPU package installed with version: 1.17.1
INFO - separator - CUDA is available in Torch, setting Torch device to CUDA
INFO - separator - ONNXruntime has CUDAExecutionProvider available, enabling acceleration
INFO - separator - Loading model model_bs_roformer_ep_317_sdr_12.9755.ckpt...
INFO - mdxc_separator - MDXC Separator initialisation complete
INFO - separator - Load model duration: 00:00:03
INFO - separator - Starting separation process for audio_file_path: 1.wav
  1%|█▏   | 6/836 [00:05<12:40,  1.09it/s]

Note the estimate of 12:40 - almost 100 times faster. At this time the GPU usage was almost 100% in Task Manager. However, look at how stable the usage was compared to the current code.

GPU Usage Instability vs Stability

Current Version

My PR

My suspicion is that data is getting copied to the GPU, then needlessly back to the CPU, then back to the GPU again.

[beveradb]

Hey, sorry to hear you're still not able to get GPU acceleration working on your machine.

I don't have (and don't want to have) a Windows computer to test on, so I'm not really sure what to do here.

The current code works with GPU acceleration on Macs with Apple Silicon and Linux with CUDA, and it's surprising to me that CUDA would work on Linux but not Windows - though there's all sorts of weird platform-specific things with PyTorch so who knows 🤷

If you want to try and fix it for Windows, you're welcome to open an updated PR with a fix (or merge in the latest changes into your older branch from PR #74), but if I test it and it breaks the acceleration on Mac or Linux I won't merge it.

Also I'm curious why you say "Roformer Models" in the title but your PR #74 only updated the BS-Roformer code; do you actually have the same issue with both BS-Roformer and Mel-Roformer models? (they have different code paths)

[jet082]

I do have that problem with mel roformer as well, if I recall. I've only been testing on the bs-roformer model though. I'll take a look and see if I can merge the changes to get them to work across all operating systems.