TensorRT extension works after MANUALLY installing correct version with CuDNN

[ccarmatic]

CUDA 11.8, Python 3.10.14, Windows 11
Installed as per instructions:
pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu118
pip install git+https://github.com/cumulo-autumn/StreamDiffusion.git@main#egg=streamdiffusion[tensorrt]
python -m streamdiffusion.tools.install-tensorrt

I managed to bypass #157 (comment) and make Streamdiffusion run by uninstalling nvidia-cudnn-cu11 after running the Streamdiffusion TensorRT installation script

This allows Streamdiffusion to finish building the engines, but when is doing the img2img process with image resolution of 512x512, it gives these error messages for every frame it generates:

[E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2] for bindings[1] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 2.
    )
[E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2,77,768] for bindings[2] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 2.
    )
[E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2,4,64,64] for bindings[0] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 2.
    )
[E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::validateInputBindings::2045, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [2] for bindings[1] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 2.
    )

and the image looks very corrupted, like it is made of rectangular tiles, and text prompts have no effect:

for reference this is the input image:

[ccarmatic]

I have fixed the problem, here is how:

1. Install CUDA 12.1
2. Install PyTorch for CUDA 12.1 pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu121
3. This step is important: Install the CUDA 12 version of TensorRT : python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post12.dev4 --no-cache-dir notice that the version is 9.01.post12.dev4 , unlike the version installed by the StreamDiffusion install script which is 9.01.post11.dev4
4. Install the version of nvidia-cudnn-cu12 which StreamDiffusion uses: python -m pip install nvidia-cudnn-cu12==8.9.4.25 --no-cache-dir
5. Install StreamDiffusion : pip install streamdiffusion[tensorrt] And run its TensorRT installation script: python -m streamdiffusion.tools.install-tensorrt ; The script should detect the presence of TensorRT and nvidia-cudnn-cu12 , and only install supporting components such as pywin32 etc (I'm not sure about this step, but if the install script installed TensorRT or CuDNN , uninstall them and repeat the 2 previous steps to install the correct versions of TensorRT and CuDNN)
6. This step is important: Copy the contents of "Python310\Lib\site-packages\nvidia\cudnn\bin" to "Python310\Lib\site-packages\torch\lib" ; You will be replacing the cudnn dll files which came with Torch , with the version you manually installed in the previous step
7. Run a Python script which engages the TensorRT extension, such as in my previous post TensorRT Extension problem : Entry Point Not Found #157 (comment) , and let it build the engines under the conditions you have made in the previous steps

StreamDiffusion should now work under CUDA 12.1 with correct functioning of the TensorRT extension

[potesd]

You're a lifesaver!! stupid freaking defunct post11 link :(

[ccarmatic]

Does anyone know how to update the install script? So that it has these corrections , as well as fixing this #185