Chore: the big bump #433

nnansters · 2024-12-18T17:36:24Z

This is the one: the big dependency bump!

As Python 3.8 is end-of-life, this release drops the support for it. This allows us to bump a lot of our core dependencies:

numpy = "^2.0"

scipy = "^1.13"

pandas = "^2.2"

scikit-learn = "^1.6"

pyarrow = "^18.1"

This should be a lot less restrictive for people with up-to-date installations.
Working with these versions means we now support Python 3.12 as well!

Rewrote the logic that performs value counting to simplify grouping behavior.

…` after the `squared` parameter was removed.

…at all big compute libraries were majorly bumped.

…calculation methods. Direct consequence of numpy/numpy@c63969c

codecov · 2024-12-19T10:43:10Z

Codecov Report

Attention: Patch coverage is 29.93421% with 213 lines in your changes missing coverage. Please review.

Project coverage is 74.75%. Comparing base (c6d326e) to head (60f9fdf).

Files with missing lines	Patch %	Lines
...performance_estimation/confidence_based/metrics.py	29.72%	104 Missing ⚠️
...e_calculation/metrics/multiclass_classification.py	2.85%	34 Missing ⚠️
...mance_calculation/metrics/binary_classification.py	53.33%	21 Missing ⚠️
...nyml/performance_calculation/metrics/regression.py	10.00%	18 Missing ⚠️
...mance_estimation/direct_loss_estimation/metrics.py	40.00%	12 Missing ⚠️
...rformance_estimation/direct_loss_estimation/dle.py	0.00%	6 Missing ⚠️
nannyml/stats/avg/calculator.py	0.00%	4 Missing ⚠️
nannyml/stats/median/calculator.py	0.00%	4 Missing ⚠️
nannyml/stats/std/calculator.py	0.00%	4 Missing ⚠️
nannyml/stats/sum/calculator.py	0.00%	4 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #433      +/-   ##
==========================================
- Coverage   74.77%   74.75%   -0.02%     
==========================================
  Files         109      109              
  Lines        9703     9706       +3     
  Branches     1537     1183     -354     
==========================================
+ Hits         7255     7256       +1     
- Misses       1966     1969       +3     
+ Partials      482      481       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

nnansters · 2024-12-19T12:19:47Z

Finally got round to it. Take a peek at the dependencies in pyproject.toml and see if it covers your needs!

#431
#430
#428
#427
#424

davisthomas-aily · 2024-12-19T12:58:23Z

pyproject.toml

+
+pandas = "^2.2"
+
+scikit-learn = "^1.6"


isn't this the very latest version released dec 9? I think ideally deps shouldn't be too constrained, as this would make it harder for users to integrate nannyml into their model monitoring.

davisthomas-aily · 2024-12-19T13:05:06Z

pyproject.toml

 s3fs = ">=2022.8.2"
 sqlmodel = {version = "^0.0.19", optional = true}
 APScheduler = "^3.9.1"
 psycopg2-binary = {version = "^2.9.3", optional = true}
 segment-analytics-python = "^2.3.2"
-python-dotenv = "^0.21.0"
+python-dotenv = "^1.0"


might not be a bad idea to maintain support to some more older versions of dotenv here, the latest version is just 1.0.1, a lot of users might not have upgraded this yet

davisthomas-aily · 2024-12-19T13:09:02Z

pyproject.toml

 s3fs = ">=2022.8.2"
 sqlmodel = {version = "^0.0.19", optional = true}
 APScheduler = "^3.9.1"
 psycopg2-binary = {version = "^2.9.3", optional = true}
 segment-analytics-python = "^2.3.2"
-python-dotenv = "^0.21.0"
+python-dotenv = "^1.0"
 types-pyyaml = "^6.0.12.8"


isn't this for type checking? you could put such dependencies in a lint subgroup. This does add some work - you would need to identify which deps you actually need to run the main package, and which ones you need for the tests, etc
here's the relevant documentation: https://python-poetry.org/docs/managing-dependencies/#dependency-groups

davisthomas-aily · 2024-12-19T13:15:30Z

Great work! Always nice to see that the package is being maintained.
I've left some comments-I think the pyproject might benefit from a little division of dependencies into main and dev (and additional ones if you really want to clean house) - this keeps it from being too restrictive, as users importing this into their own env will never actually need to actively modify the library.

davisthomas-aily · 2024-12-19T13:55:05Z

pyproject.toml

+
+scipy = "^1.13"
+
+pandas = "^2.2"


would be nice to still support atleast some pandas 1. versions

davisthomas-aily · 2024-12-19T13:57:12Z

pyproject.toml

+
+scikit-learn = "^1.6"
+
+pyarrow = "^18.1"


this is also the latest version released nov, I think you should support it from atleast 13.0 or 14.0, those versions were released just last year.

davisthomas-aily · 2024-12-19T14:51:43Z

pyproject.toml

-]
+python = ">=3.9,<3.13"
+
+numpy = "^2.0"


This seems too restrictive as well, numpy 1.26 was just released in Sep 2023

davisthomas-aily · 2024-12-19T15:16:32Z

pyproject.toml

@@ -51,13 +56,13 @@ PyYAML = "^6.0"
 Jinja2 = "<3.1"
 pyfiglet = "^0.8.post1"
 lightgbm = "^3.3.2"


the latest lightgbm version is at 4.5, would be nice to have support for this

canoadri · 2025-01-02T09:21:49Z

Hi all, any updates on this PR?

nnansters added 11 commits December 17, 2024 19:09

Replace Pydantic validator by the new field_validator

e8623af

Dropped 3.8 support for now, updated major dependencies

94c3e24

Updated FLAML for numpy compatibility

ed76029

Replace np.NaN by np.nan

3c96296

Fix pearsonr for empty arrays

5f80785

Fix issue in stacked bar plots (not sure where this came from).

22a018c

Rewrote the logic that performs value counting to simplify grouping behavior.

Changed to root_mean_squared_error and `root_mean_squared_log_error…

6ae8184

…` after the `squared` parameter was removed.

Changed some leftover np.NAN values to np.nan

1e05a02

Some values changed on the fifth decimal, we can accept that given th…

9de66dc

…at all big compute libraries were majorly bumped.

Merge branch 'main' into chore/update-dependencies

a8ac374

Update actions workflows: drop 3.8 support, add 3.12 support.

cc51010

nnansters force-pushed the chore/update-dependencies branch from 3e0baa4 to cc51010 Compare December 18, 2024 18:26

Fix issue with binning behavior in continuous Hellinger and JS drift …

1cd1b55

…calculation methods. Direct consequence of numpy/numpy@c63969c

Fix flake8 issues + use python 3.11 for linting

60f9fdf

nnansters marked this pull request as ready for review December 19, 2024 12:14

nnansters requested a review from nikml as a code owner December 19, 2024 12:14

nnansters mentioned this pull request Dec 19, 2024

Updating dependency - python-dotenv #430

Closed

This was referenced Dec 19, 2024

Pyproject too restrictive #431

Open

Update all dependencies #428

Open

davisthomas-aily reviewed Dec 19, 2024

View reviewed changes

pyproject.toml

scipy = "^1.13"

pandas = "^2.2"

Copy link

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to still support atleast some pandas 1. versions

davisthomas-aily reviewed Dec 19, 2024

View reviewed changes

pyproject.toml

]

python = ">=3.9,<3.13"

numpy = "^2.0"

Copy link

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems too restrictive as well, numpy 1.26 was just released in Sep 2023

davisthomas-aily reviewed Dec 19, 2024

View reviewed changes

nnansters mentioned this pull request Jan 8, 2025

Dependencies - a suggestion #434

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Chore: the big bump #433

Chore: the big bump #433

nnansters commented Dec 18, 2024

codecov bot commented Dec 19, 2024 •

edited

Loading

nnansters commented Dec 19, 2024

davisthomas-aily Dec 19, 2024

davisthomas-aily Dec 19, 2024 •

edited

Loading

davisthomas-aily Dec 19, 2024

davisthomas-aily commented Dec 19, 2024

davisthomas-aily Dec 19, 2024

davisthomas-aily Dec 19, 2024

davisthomas-aily Dec 19, 2024

davisthomas-aily Dec 19, 2024 •

edited

Loading

canoadri commented Jan 2, 2025

Chore: the big bump #433

Are you sure you want to change the base?

Chore: the big bump #433

Conversation

nnansters commented Dec 18, 2024

codecov bot commented Dec 19, 2024 • edited Loading

Codecov Report

nnansters commented Dec 19, 2024

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

davisthomas-aily Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

davisthomas-aily commented Dec 19, 2024

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

davisthomas-aily Dec 19, 2024

Choose a reason for hiding this comment

davisthomas-aily Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

canoadri commented Jan 2, 2025

codecov bot commented Dec 19, 2024 •

edited

Loading

davisthomas-aily Dec 19, 2024 •

edited

Loading

davisthomas-aily Dec 19, 2024 •

edited

Loading