private or public functions in _accelerate

[gadamc]

Hi Qiskiters. We rely on a number of classes/functions that are quickly moving from python to rust and sometimes exposed in _accelerate. For example, in some current work we use from qiskit._accelerate.convert_2q_block_matrix import blocks_to_matrix. This is available in 1.2.4, but appears to be private in 1.3 because convert_2q_block_matrix was removed from qiskit/__init__.py. Could this be restored for public access?

Also, I'm confused about the private/public status of _accelerate in general, given it's leading underscore. In this comment, does "people" refer to external users or does it refer to developers of qiskit? Should we assume that anything imported in __init__.py, including _accelerate is public?

Thank you.

[mtreinish]

As of right now the entire _accelerate module is treated as a private module in Qiskit this is why it is prefixed with a _ which is the Python coding convention to indicate an API is internal: https://peps.python.org/pep-0008/#descriptive-naming-styles (since python as a language doesn't actually have a concept of private vs public). The only exceptions to that are where we have re-export functions to a public documented name. An example (the only one I can think of off the top of my head) of this is CircuitInstruction (https://docs.quantum.ibm.com/api/qiskit/qiskit.circuit.CircuitInstruction) which is originally imported from _accelerate but is re-exported as a public documented interface.

The Qiskit stability policy is documented here (unfortunately the docs theme doesn't make directly linking easy, but it's in that page): https://docs.quantum.ibm.com/open-source/qiskit-sdk-version-strategy#qiskit-sdk-version-strategy and the Qiskit project policy is that an API is only considered public if we document it.

The intent of _accelerate is it's an internal module that provides the inner Rust functions to accelerate certain pieces of Qiskit. If you have a specific use case that a function from _accelerate is providing which isn't public I'd suggest opening a feature request issue and we can discuss potentially adding a public API to meet the needs you have.

[jakelishman]

an API is only considered public if we document it

Just to really highlight: the absence of an underscore in any function, even one defined in Python space, is not sufficient to make an API public. It must be documented in the online documentation to be public, which typically means it is explicitly exported by some __init__.py file whose module is also public and documented online.

[gadamc]

Thanks for the quick reply @jakelishman and @mtreinish. I'm curious then, why all the imports of _accelerate in qiskit\__init__.py?

[jakelishman]

It's so Qiskit files can write

from qiskit._accelerate.convert_2q_blocks import convert_2q_blocks

or such like. The way C extensions modules work in CPython, qiskit._accelerate isn't actually a package, and the import system can only drill down like import package.subpackage.subpackage2.subpackage_or_module. It can't drill down import package.module.submodule; it'd error because package.module isn't actually a "package". In a from a.b.c import d line, the object a.b.c is retrieved using the package import rule, then d is tried with the regular import system, and if that fails, then it tries attribute-lookup semantics.

The trick (and what that __init__.py file is up to) is that package imports are cached in sys.modules to ensure that the import system only needs to do the filesystem lookup once, and that it executes the initialisation code of each module only once. So if we put the "not actually a package but a module" submodules of _accelerate into the right places in sys.modules, then the import system doesn't try to lookup qiskit._accelerate.convert_2q_matrices as a package, which would fail because it isn't an importable package (it's just a types.ModuleType object that's attached to another module).

It's getting into subtleties in the distinction between "packages" and "modules" in Python, which usually isn't visible to the Python programmer.

[jakelishman]

You could argue that the way we should have done that, if we really wanted, was to play with sys.meta_path, but we didn't do it on the initial commit that added to Rust, and it just never seemed worth it since then.

[gadamc]

Makes sense now. Thanks for the replies. We'll probably create an Issue asking to make blocks_to_matrix public.