Conceptual intro

[ricardoV94]

What is PyTensor

A library to define and manipulate array-based computational graphs

• Represent numpy-like computations in a graph composed of Python objects
• Manipulate graph at the Python level for:
  • Optimization and stabilization
  • Automatic differentiation
  • Vectorization and "Scalarization"
  • Anything else: Shape inference, logp inference
• Compile to performant non-python backends: C, Numba, JAX, PyTorch (or just vanilla numpy)

How does it compare with other frameworks

• Analogy with Sympy
• Graph is specified explicitly and with placeholder inputs
• It is focused on array (tensor) operations (dense and sparse). Narrow/hidden support for other types like scalars, lists, slices, random Generators, strings, None (although easy to extend)
• Side-effects free from the user perspective
• EVERYTHING IS LAZY!
• Trivial support for "partialling" / subgraph extraction.
• Strong focus on hackability / user-defined rewrites
• It's completely ours! (And nobody else uses it)
• Evolved from:
  1. Theano which strongly inspired Tensorflow 1.x and JAX. Many concepts stood the test of time. Others have aged and provide some drag.
  2. Aesara, which cleaned up the codebase, added alternative backends (Numba and JAX) and proved there's some interest out there in a library like this.