add openai completion API to SQL and daemon

[var77]

• Add SQL function to call OpenAI completion API
• Refactor embedding jobs, remove CSV exporter, make db_exporter more generic

[greptile-apps]

PR Summary

Hi! Looks like you've reached your API usage limit. You can increase it from your account settings page here: app.greptile.com/settings/usage

_{17 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile}

[github-actions]

Benchmarks

metric	old	new	pct change
recall (after create)	0.950	0.955	+0.53%
recall (after insert)	0.000	0.000	-
select tps	26607.539	24202.087	-9.04%
select bulk(100) tps	35.451	35.637	+0.52%
select latency (ms)	0.722 ± 1.642𝜎	0.705 ± 1.647𝜎	-2.35%
select bulk(100) latency (ms)	874.591 ± 110.647𝜎	865.651 ± 114.940𝜎	-1.02%
create latency (ms)	393139.712	395969.102	+0.72%
insert tps	471.130	466.257	-1.03%
insert bulk(100) tps	5.017	5.048	+0.62%
insert latency (ms)	67.257 ± 14.674𝜎	68.013 ± 15.906𝜎	+1.12%
insert bulk(100) latency (ms)	6230.209 ± 137.708𝜎	6208.913 ± 133.992𝜎	-0.34%
disk usage (bytes)	8192008192.000	8192008192.000	-

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

Attention: Patch coverage is 43.92202% with 489 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
lantern_extras/src/daemon.rs	1.61%	244 Missing ⚠️
lantern_extras/src/embeddings.rs	0.76%	129 Missing ⚠️
lantern_cli/src/embeddings/core/openai_runtime.rs	77.94%	30 Missing ⚠️
lantern_extras/src/lib.rs	0.00%	18 Missing ⚠️
lantern_cli/src/embeddings/mod.rs	91.87%	16 Missing ⚠️
lantern_cli/src/embeddings/core/cohere_runtime.rs	0.00%	15 Missing ⚠️
lantern_cli/src/embeddings/core/mod.rs	64.28%	10 Missing ⚠️
lantern_cli/src/embeddings/cli.rs	38.46%	8 Missing ⚠️
lantern_cli/src/embeddings/measure_speed.rs	0.00%	8 Missing ⚠️
lantern_cli/src/embeddings/core/ort_runtime.rs	0.00%	4 Missing ⚠️
... and 5 more

📢 Thoughts on this report? Let us know!

[dqii]

Was this intended to be included?

[var77]

it was by mistake thanks for noticing!