Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latency improvements with the new manual VAD algorithm #368

Open
1 of 6 tasks
clemlesne opened this issue Dec 7, 2024 · 1 comment
Open
1 of 6 tasks

Latency improvements with the new manual VAD algorithm #368

clemlesne opened this issue Dec 7, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@clemlesne
Copy link
Collaborator

clemlesne commented Dec 7, 2024

Done:

  • Implement monitoring metrics to follow improvements

Todo:

  • Parallelize more TTS and database calls (study OTEL traces for opportunity confirmation)
  • Reduce dependency calls before sending call to the LLM or defer them
  • Compress the prompt (LLMlingua?)
  • Use a LLM with a lower latency (Phi 4?)
  • Trace the code executions with local debugger to pin points unseen optimizations
@clemlesne clemlesne added the enhancement New feature or request label Dec 7, 2024
@clemlesne
Copy link
Collaborator Author

clemlesne commented Jan 9, 2025

OTEL gauges and counters has been added to follow technical and functional metrics.

Notably:

  • call.aec.droped, number of times the echo cancellation dropped the voice completely.
  • call.aec.missed, number of times the echo cancellation failed to remove the echo in time.
  • call.answer.latency, time between the end of the user voice and the start of the bot voice.

See 59e0d68, 34ca384, a2b484c, 24144a9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant