TensorRT failed to build with dynamic shapes

[0xprincess]

Describe the bug

I'm getting the following error when trying to build TensorRT engine with dynamic shape flag enabled. Tried combination with other flags, but it doesn't seem to matter and dynamic shape is probably the reason. Builds fine without it, but I need dynamic shape to work to use Highres fix.

[I] Building engine with configuration:
    Flags                  | [FP16, REFIT]
    Engine Capability      | EngineCapability.DEFAULT
    Memory Pools           | [WORKSPACE: 8406.00 MiB, TACTIC_DRAM: 24563.38 MiB]
    Tactic Sources         | []
    Profiling Verbosity    | ProfilingVerbosity.DETAILED
    Preview Features       | [FASTER_DYNAMIC_SHAPES_0805, DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]
[E] 10: Could not find any implementation for node {ForeignNode[onnx::LayerNormalization_9032 + (Unnamed Layer* 1229) [Shuffle].../down_blocks.0/attentions.0/Reshape_1 + /down_blocks.0/attentions.0/Transpose_1]}.
[E] 10: [optimizer.cpp::nvinfer1::builder::cgraph::LeafCNode::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[onnx::LayerNormalization_9032 + (Unnamed Layer* 1229) [Shuffle].../down_blocks.0/attentions.0/Reshape_1 + /down_blocks.0/attentions.0/Transpose_1]}.)
[!] Invalid Engine. Please ensure the engine was built correctly
Traceback (most recent call last):
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\routes.py", line 399, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\blocks.py", line 1299, in process_api
    result = await self.call_function(
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\blocks.py", line 1036, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\utils.py", line 488, in async_iteration
    return next(iterator)
  File "C:\Users\Princess\projects\Radiata\modules\tabs\tensorrt.py", line 173, in build_engine
    builder.build()
  File "C:\Users\Princess\projects\Radiata\modules\acceleration\tensorrt\engine.py", line 99, in build
    build_engine(
  File "C:\Users\Princess\projects\Radiata\lib\tensorrt\utilities.py", line 402, in build_engine
    engine.build(
  File "C:\Users\Princess\projects\Radiata\lib\tensorrt\utilities.py", line 241, in build
    engine = engine_from_network(
  File "<string>", line 3, in func_impl
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\base\loader.py", line 42, in __call__
    return self.call_impl(*args, **kwargs)
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 526, in call_impl
    return engine_from_bytes(super().call_impl)
  File "<string>", line 3, in func_impl
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\base\loader.py", line 42, in __call__
    return self.call_impl(*args, **kwargs)
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 550, in call_impl
    buffer, owns_buffer = util.invoke_if_callable(self._serialized_engine)
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\util\util.py", line 661, in invoke_if_callable
    ret = func(*args, **kwargs)
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 484, in call_impl
    G_LOGGER.critical("Invalid Engine. Please ensure the engine was built correctly")
  File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\logger\logger.py", line 597, in critical
    raise PolygraphyException(message) from None

Reproduction

Fresh clean install

Expected behavior

Building engine without an error

System Info

• Windows 10 22h2
• RTX 4090
• 32gb RAM
• Python 3.10.8
• TensorRT-8.6.1.6
• CUDNN-v8.9 (v8.8 also tried)
• Nvidia drivers 531.41

Additional context

No response

Validations

• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.