Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU delegation to OpenCL backend failing for certain models #506

Open
AD-lite24 opened this issue Jan 18, 2025 · 1 comment
Open

GPU delegation to OpenCL backend failing for certain models #506

AD-lite24 opened this issue Jan 18, 2025 · 1 comment
Assignees
Labels
status:awaiting user response awaiting user response status:need more data need more data type:gpu delegate Issue with GPU delegation

Comments

@AD-lite24
Copy link

AD-lite24 commented Jan 18, 2025

We have been using the LiteRT runtime for a while on our qualcomm devices to use the GPU delegation. However with certain models (namely Superpoint, Lightglue, YoloV11) the delegation keeps failing. After setting up more verbose error logs we get the following errors

ERROR: Following operations are not supported by GPU delegate:
CAST: Not supported Cast case.
CAST: Not supported cast case
CEIL: Operation is not supported.
CONCATENATION: OP is supported, but tensor type/shape isn't compatible.
EQUAL: Not supported logical op case
EQUAL: Not supported logical op case.
FLOOR_DIV: OP is supported, but tensor type/shape isn't compatible.
FLOOR_MOD: OP is supported, but tensor type/shape isn't compatible.
GATHER_ND: Operation is not supported.
LOGICAL_AND: Operation is not supported.
LOGICAL_NOT: Operation is not supported.
LOGICAL_OR: Operation is not supported.
RESHAPE: OP is supported, but tensor type/shape isn't compatible.
SCATTER_ND: Operation is not supported.
SELECT: Operation is not supported.
STRIDED_SLICE: OP is supported, but tensor type/shape isn't compatible.
TOPK_V2: Operation is not supported.
TRANSPOSE: Max version supported: 2. Requested version 4.
28 operations will run on the GPU, and the remaining 116 operations will run on the CPU.
ERROR: No selector for floor
ERROR: Falling back to OpenGL
ERROR: TfLiteGpuDelegate Init: OpenGL-based API disabled
INFO: Created 0 GPU delegate kernels.
ERROR: TfLiteGpuDelegate Prepare: delegate is not initialized
ERROR: Node number 144 (TfLiteGpuDelegateV2) failed to prepare.
ERROR: Restored original execution plan after delegate application failure.
GPU delegate error.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.

The unsupported OP errors are one issue, but there the other issue is the failure to apply the OpenCL backend

28 operations will run on the GPU, and the remaining 116 operations will run on the CPU.
ERROR: No selector for floor
ERROR: Falling back to OpenGL
ERROR: TfLiteGpuDelegate Init: OpenGL-based API disabled
INFO: Created 0 GPU delegate kernels.

For our other models we only use OpenCL backends and they seem to work fine, but for some reason it doesn't work here and there don't seem to be any more verbose errors I can obtain. The only thing that might could be a hint is ERROR: No selector for floor that is still rather cryptic (perhaps the floor opt?).

@pkgoogle
Copy link

Hi @AD-lite24, can you show us an example of how you are applying the delegation? The more context, the better for us.

@pkgoogle pkgoogle self-assigned this Jan 24, 2025
@pkgoogle pkgoogle added status:need more data need more data status:awaiting user response awaiting user response type:gpu delegate Issue with GPU delegation labels Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:awaiting user response awaiting user response status:need more data need more data type:gpu delegate Issue with GPU delegation
Projects
None yet
Development

No branches or pull requests

2 participants