Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how can the ai-edge-torch converter convert the operators supported on edgeTPU devices #450

Open
JinniPi opened this issue Jan 7, 2025 · 1 comment
Assignees
Labels
status:awaiting ai-edge-developer type:feature For feature requests type:performance An issue with performance, primarily inference latency

Comments

@JinniPi
Copy link

JinniPi commented Jan 7, 2025

Description of the bug:

  • I converted the pytorch model format to tflite format with p2e quantization static. Then I compile the tflite file to run the EdgeTPU. But the problem is that almost operators aren't supported running on TPU. Instead, they will run on CPU.
  • My question is: how can the ai-edge-torch converter convert the operators supported on edgeTPU devices?
  • For example, here is my log:
  • Model compiled successfully in 70 ms.

Input model: ZenNet-SA-M/rafdb_quant-m-sa_int8_pte.tflite
Input size: 1.57MiB
Output model: rafdb_quant-m-sa_int8_pte_edgetpu.tflite
Output size: 1.60MiB
On-chip memory used for caching model parameters: 0.00B
On-chip memory remaining for caching model parameters: 8.09MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 243
Operation log: rafdb_quant-m-sa_int8_pte_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 1
Number of operations that will run on CPU: 242
See the operation log file for individual operation details.
Compilation child process completed within timeout period.
Compilation succeeded!

Actual vs expected behavior:

The operators are supported on edgeTPU (not CPU)

Any other information you'd like to share?

No response

@pkgoogle
Copy link
Contributor

pkgoogle commented Jan 7, 2025

Thanks for pointing this out. I don't believe this is 100% supported right now but would be a good optimization to have.

@pkgoogle pkgoogle added type:feature For feature requests status:awaiting ai-edge-developer type:performance An issue with performance, primarily inference latency and removed type:bug Bug labels Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:awaiting ai-edge-developer type:feature For feature requests type:performance An issue with performance, primarily inference latency
Projects
None yet
Development

No branches or pull requests

2 participants