Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: pylibcugraph: declare cupy and numpy hard dependencies #4854

Draft
wants to merge 3 commits into
base: branch-25.02
Choose a base branch
from

Conversation

jameslamb
Copy link
Member

@jameslamb jameslamb commented Jan 8, 2025

While testing stuff for #4804, I found that pylibcugraph has a hard runtime dependency on cupy and numpy, but isn't declaring them

docker run \
    --rm \
    --gpus 0 \
    -it rapidsai/ci-wheel:latest \
    bash

python -m pip install 'pylibcugraph-cu12==25.2.*,>=0.0.0a0'
python -c "import pylibcugraph"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/pyenv/versions/3.12.7/lib/python3.12/site-packages/pylibcugraph/__init__.py", line 18, in <module>
    from pylibcugraph.graphs import SGGraph, MGGraph
  File "graphs.pyx", line 1, in init pylibcugraph.graphs
  File "utils.pyx", line 20, in init pylibcugraph.utils
ModuleNotFoundError: No module named 'cupy'

Notes for Reviewers

Evidence that pylibcugraph already has a hard dependency on these libraries

They're used unconditionally here:

import numpy
import cupy

But have import guards in other places:

# FIXME: import these modules here for now until a better pattern can be
# used for optional imports (perhaps 'import_optional()' from cugraph), or
# these are made hard dependencies.
try:
import cupy
except ModuleNotFoundError:
raise RuntimeError("sssp requires the cupy package, which could not "
"be imported")
try:
import numpy
except ModuleNotFoundError:
raise RuntimeError("sssp requires the numpy package, which could not "
"be imported")

So this PR doesn't introduce new hard dependencies... it just makes them explicit, to make it easier to install and run pylibcugraph.

How was this not caught in CI?

Import tests aren't run here for conda packages, because conda builds happen on CPU-only nodes.

rapids-conda-retry mambabuild \
--no-test \
--channel "${CPP_CHANNEL}" \
conda/recipes/pylibcugraph

And numpy and cupy are probably getting pulled in by some of the wheels' test dependencies, like cudf, here:

python -m pip install $(ls ./dist/${python_package_name}*.whl)[test]

Should we just make the other unconditional cases conditional with try-catching?

No. Talked with @rlratzel, @ChuckHastings, and @eriknw offline, and we agreed to declare these as hard runtime dependencies (and remove the try-catching in places that had it).

@jameslamb jameslamb added bug Something isn't working non-breaking Non-breaking change labels Jan 8, 2025
Copy link

copy-pr-bot bot commented Jan 8, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@jameslamb jameslamb changed the title WIP: pylibcugraph: declare cupy dependency WIP: pylibcugraph: declare cupy and numpy hard dependencies Jan 9, 2025
@jameslamb
Copy link
Member Author

/ok to test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working conda non-breaking Non-breaking change python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant