Packaging a python project with cpp backend built with bazel

[eugifa]

Hi all, I am trying to package a Python API that exploits cpp backend with pybind11. I would like to build the cpp code with bazel.
As of now I have seen there are two possible ways:

• Using py_wheel from rules_python and having as dependencies the .so files coming from pybind11
• Using setuptools external dependencies, this second possibility seems more complex.

Do you know which is the right way to proceed ?
Do you have further suggestions ?

[meteorcloudy]

/cc @rickeylev

[rickeylev]

I don't have experience with setuptools, so can't speak to how that is done in comparison. I have only a small amount of knowledge of pybind and pybind_bazel.

py_wheel should allow you to package up arbitrary things into a wheel and include arbitrary data files.

I don't know how abi-independent pybind is, which will determine how generic of a .so you can build. In pybind_bazel, I believe it uses cc_binary, which has a tendency for creating statically linked shared libraries, so you'll have to be careful in what dependencies it has to prevent including extra stuff that would cause and ODR violation for your downstream users. (cc_shared_library better addresses this, but I'm not sure if that's out of experimental yet; I think tensorflow uses it).

HTH