Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move lantern_extras to lantern repo #339

Merged
merged 281 commits into from
Oct 9, 2024
Merged

Move lantern_extras to lantern repo #339

merged 281 commits into from
Oct 9, 2024

Conversation

var77
Copy link
Collaborator

@var77 var77 commented Oct 8, 2024

No description provided.

var77 added 30 commits October 30, 2023 16:29
@var77
Add lantern-cli daemon mode (#24)
d44c0d5 
* Add lantern-cli daemon mode

* Add memory optimizations and comment code

* Add Dockerfile for cli

* Add logger, add task id when logging

* Fix db exporter

* Fix logger.warn

* Update README add logger in create index

* Add dockerfile for CUDA

* Add print_raw method for logger

* Update onnxruntime version in README

* Update GPU dockerfile

* Update README for docker usage

* Add update listener, continous embedding generation, startup checks

* Fix startup query

* Optimize tokenizer

* Implement stop handler, optimize code, make fault tolerant

* Remove unnecessary clones on references

* Set data path based on OS

* Add restarts on connection loss

* Implement row locking and batching for client table inserts

* Rename client to client_jobs

* Check write permission on target table, add streaming option in embeddings

* Send pending jobs only to update channel

* Fix model column name typo

* Fix startup job init

* Fix init_finished_at update on startup

* Add comment

* Change gh runner ubuntu version

* Update versions and README

* Fix updates for rows with non-numeric ids

* Update BERT large model to v1.5

* Add more BERT models

* Make tokenizer and inputs for bert model dynamic
@var77
Add docker publish action for CLI (#28)
d505ca7 
* Add docker publish action for CLI

* Optimize dockerfiles
@var77
Add tests for lantern_embeddings and lantern_embeddings core (#27)
91b1e17
@var77
Better handling for sql query formatting (#29)
c78a27b 
* Better handle sql quotting, update README for docker run

* Bump cli version
@var77
Varik/skip column creation (#30)
62ff52a 
* Skip column creation if specified, check permissions from grants, improve error logging for client task

* Add schema name to table permission check
@var77
Varik/fix client identifier names (#31)
5a20fd7 
* Fix handling names for client identifier

* Update cli version
@var77
Fix typo in trigger name (#32)
51e4af5
@var77
Fix notification channel for client (#33)
dd32004
@var77
Add dst_column to trigger and function names (#34)
613b103
@var77
Explicitly specify ort version (#36)
279f72a
@var77
Create lock table in lantern schema hash trigger names (#37)
fd804df 
* - Create lock table in lantern schema
- Hash client trigger function and trigger names so they won't exceed the character limit

* Bump cli version
@var77
Create client function and trigger names with job_id instead of hash (#…
018f804 
…38)
@var77
Collect update jobs non-blocking (#39)
2e9573c 
* Make row locking non-blocking

* Remove unnecessary deref
@var77
Add progress and usage tracking (#40)
ef4e299
@var77
Add model speed measurement script (#41)
e13d286 
* Add command to measure model generation speed

* Fix GPU memory leak, merge process_text and process_image into one function

* Add min/max/avg speeds

* Update min time for test

* Update version
@var77
Fix data_producer deadlock, remove unwraps (#42)
7d53268 
* Fix data_producer deadloc, remove unwraps

* Bump versions
@var77
Add db connect timeout (#43)
4e86a66
@var77
Index autotune and import (#44)
8b0858e 
* Index autotune WIP

* Add import functionality for lantern index

* add index autotune to cli

* Update README
@var77
Optimize batch size for models (#46)
084b28e
@var77
Check memory usage to avoid OOM Errors (#45)
d82daa0 
* Check memory usage before running model. references #26

* Fix checks for GPU #26

* Add info message #26

* Print more informative error messages

* Bump version

* Refactor naming

* Fix return type

* Bump versions
@var77
Index Autotune Improvements (#47)
9d334e3 
* Fix find_bes_variant, add comments, add existing result reuse functionality

* Refactor daemon, add autotune to daemon

* Make float to 2 decimal points, change internal schema name

* Bump version
@var77
Remove index file after import (#48)
5098036 
* Remove index file from file system

* Bump version

* Use index params when importing external inedx
@var77
Add external index functions to SQL and make index creation in one tr…
03acdfc 
…ansaction with lock (#49)

* Add _lantern_reindex_external_index, lantern_create_external_index functions and make index creation in one transaction with lock

* Check column dimension while creating index #19

* Remove regex package
@var77
Take relation instead of index id, encapsulate reindex function in sc…
55d3bea 
…hema (#50)
@var77
Gracefully exit ongoing embedding job if the job is canceled from db (#…
eb8bb10 
…51)

* Cancel running embedding job with tokio tasks

* Gracefully exit ongoing embedding job if the job is canceled from db

* Delete collected insert rows on job cancel
@var77
Add checks for downloading images (#52)
629adfd
@var77
Autotune bugfix and reduce memory consumption for index creation (#53)
e10373d 
* Fix column names for index autotune

* Try to reduce memory usage on index creation

* Drop variables as soon as they are not needed anymore

* Drop values outside and remove unnecessary cursor close statement

* Change internal schema name
@var77
External index jobs (#54)
b9de6da 
* Add external index jobs, refactor codes, add cancel handlers for all jobs

* Fix progress tracking for external index

* Added tests for daemon
@var77
Added CI tests (#55)
21a0e29 
* Added CI tests

* Fix lantern build

* Fix cargo path

* Skip pgrx tests

* Optimize tests
@var77
Fix sample size to take from test table count (#56)
c6c2ca5 
* Fix sample size to take from test table count

* Fix data type
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 8, 2024 •
edited
Loading

Benchmarks

metric old new pct change
recall (after create) 0.952 0.950 -0.21%
recall (after insert) 0.000 0.000 -
select tps 23737.698 26373.626 +11.10%
select bulk(100) tps 35.389 34.115 -3.60%
select latency (ms) 0.837 ± 1.783𝜎 0.722 ± 1.437𝜎 -13.74%
select bulk(100) latency (ms) 866.247 ± 112.589𝜎 911.508 ± 148.490𝜎 +5.22%
create latency (ms) 395466.232 408962.859 +3.41%
insert tps 465.338 480.187 +3.19%
insert bulk(100) tps 5.015 5.050 +0.68%
insert latency (ms) 68.166 ± 15.101𝜎 65.800 ± 13.093𝜎 -3.47%
insert bulk(100) latency (ms) 6233.738 ± 136.212𝜎 6209.783 ± 120.369𝜎 -0.38%
disk usage (bytes) 8192008192.000 8192008192.000 -

@var77
combine build action for lantern and lantern_extras
19ff9c1 
fix test action on CI

fix paths in CI scripts

update ubuntu version for extras runner

fix workflow syntax

fix test workflow

update version, fix test_updates script

fix pgvector build

fix update script for new version

fix onnx installation on arm instances
@var77 var77 force-pushed the varik/merge-lantern-extras branch from 05bfa68 to 19ff9c1 Compare October 8, 2024 16:18
@var77 var77 marked this pull request as ready for review October 8, 2024 16:19
@var77 var77 requested review from Ngalstyan4 and dqii October 8, 2024 16:22
@var77 var77 changed the title Merge lantern extras Move lantern_extras to lantern repo Oct 8, 2024
@codecov Codecov
Copy link

codecov bot commented Oct 9, 2024

Codecov Report

Attention: Patch coverage is 65.21833% with 3234 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
lantern_extras/src/daemon.rs 1.44% 410 Missing ⚠️
lantern_cli/src/pq/gcp_batch.rs 0.00% 298 Missing ⚠️
lantern_cli/src/embeddings/core/ort_runtime.rs 68.23% 243 Missing ⚠️
lantern_extras/src/embeddings.rs 1.22% 242 Missing ⚠️
lantern_cli/src/embeddings/core/openai_runtime.rs 0.00% 185 Missing ⚠️
lantern_cli/src/daemon/mod.rs 50.14% 170 Missing ⚠️
lantern_extras/src/dotvecs.rs 0.00% 159 Missing ⚠️
lantern_cli/src/embeddings/measure_speed.rs 0.00% 142 Missing ⚠️
lantern_cli/src/embeddings/core/cohere_runtime.rs 0.00% 139 Missing ⚠️
lantern_extras/src/external_index.rs 0.00% 133 Missing ⚠️
... and 32 more

📢 Thoughts on this report? Let us know!

@var77 var77 merged commit 2f7f42c into main Oct 9, 2024
55 of 58 checks passed
@var77 var77 deleted the varik/merge-lantern-extras branch October 9, 2024 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ngalstyan4 Ngalstyan4 Awaiting requested review from Ngalstyan4

@dqii dqii Awaiting requested review from dqii

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@var77 @Ngalstyan4