The GPU acceleration status can be verified by following the steps below:
- Ensure that the dependencies have been installed in the NVIDIA GPU way:
pip install onnxruntime-gpuand the corresponding version of PyTorch - Run the detect command in an activated virtual environment:
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')" - When the output
CUDA available: TrueThe configuration is successful when - The actual run can be done by opening Task Manager (Windows) or using the
nvidia-smiCommand (Linux) Observe GPU Utilization
Caveats:
- If False is displayed, please check if the CUDA driver version matches the PyTorch version
- Make sure NVIDIA Container Toolkit is installed for Docker deployments.
- There may be a delay in initializing the CUDA context for the first run
This answer comes from the articleKitten-TTS-Server: a self-deployable lightweight text-to-speech serviceThe
May not be reproduced without permission:AI productivity tools " How to verify if GPU acceleration is successfully enabled?
English 
简体中文 
日本語 
Deutsch 
Português do Brasil