Evaluable to Python code compiler #859
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
2 times, most recently
from
520fd9a to
5abc0bc
Compare
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
6 times, most recently
from
0c5c56b to
8d6f738
Compare
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
4 times, most recently
from
444d9b7 to
bf0ef32
Compare
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
4 times, most recently
from
a48ac1f to
5195e81
Compare
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
2 times, most recently
from
5cd774f to
432146f
Compare
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
10 times, most recently
from
28e9f8f to
f821fdf
Compare
gertjanvanzwieten
previously approved these changes
May 31, 2024
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
from
9a336f4 to
b725ef1
Compare
This patch adds an alternative for `Evaluable.eval`: an Evaluable to Python code generator. This patch adds a new module `nutils._pyast` which provides a small subset of the Python language in the form of AST nodes, and a function `evaluable.compile` which compiles `Evaluable`s to AST, translates AST to Python code and returns the compiled Python code as a callable.
The previous patch added the basic compile infrastructure. This patch implements several `Evaluable._compile_expression`s, shadowing the default implementation which uses `self.evalf`. This brings more readable generated code and slightly faster execution as we skip a call to `self.evalf`.
This patch adds support for compiling `Evaluable`s using inplace additions or copy operations. For example `Inflate` in `LoopSum(Inflate(...), ...)` is compiled using inplace additions, yielding a large performance improvement if the `Inflate` is sparse.
This patch implements an alternative for commit f914a19 "replace const Arrays with Constants in optimize".
When replacing `Evaluable.eval` with a parallel `evaluable.compile` several assertions in the caching mechanism of `solver` and corresponding unittests fail because a repeated call to `evaluable.compile` may produce slightly different results. This is caused by the non-deterministic order of inplace adds in the different threads. This patch removes assertions and rewrites unittests using tolerances.
This patch replaces `Evaluable.eval` with `evaluable.compile` in `evaluable.eval_sparse`.
This patch replaces `Evaluable.eval` with `evaluable.compile` in `function.Basis`.
This patch replaces `Evaluable.eval` with `evaluable.compile` in `Topology.locate`.
This patch replaces `Evaluable.eval` with `evaluable.compile` in `Topology.trim`.
The new `evaluable.compile`, or more specifically, `evaluable._define_loop_block_structure` can't digest `LoopTuple`s. To prepare for using `evaluable.compile` in `Evaluable.eval`, this patch removes the combine loops features from `_optimized_for_numpy`, temporarily reducing the performance of `Evaluable.eval`.
This patch rewrites `Evaluable.eval` using `evaluable.compile` and removes `Evaluable.eval_withtimes`.
In preparation for the removal of `Evaluable.serialized` and `EVALARGS`, and because `Argument` is the only dependent on `EVALARGS`, this patch removes `EVALARGS not in self.dependencies` check from `Evaluable.isconstant`.
With `Evaluable.eval` rewritten in terms of `evaluable.compile` the `EVALARGS` are now unused. This patch removes `EVALARGS` from `evaluable.Argument` and `evaluable._LoopIndex`.
The `Loop._invariants` are quite expensive to build. This patch aims to reduce the usage of `Loop._invariants` by implementing `Loop.arguments` without `Loop._invariants`, shadowing the default implementation `Evaluable.arguments` which uses the `Loop._invariants` as part of the eval args.
The `Loop._invariants` are quite expensive to build. This patch aims to reduce the usage of `Loop._invariants` by reimplementing `Loop._node` without `Loop._invariants`.
`evaluable.compile` adds debug checks to output of every compiled `Array`. If an `Evaluable` compiles itself using `Evaluable.evalf` the debug checker meta class double checks the output. This patch removes the latter.
With `Evaluable.eval` rewritten using `evaluable.compile`, many evaluable methods and properties are now unused. This patch removes them.
To reduce the unnecessary usage of `evaluable.Tuple`, this patch replaces the loop init and body arg (singular) with init and body args (plural), similar to the `args` parameter of `Evaluable`.
joostvanzwieten
force-pushed
the
eval-cache-consts
branch
from
b725ef1 to
8077d62
Compare
gertjanvanzwieten
approved these changes
May 31, 2024
joostvanzwieten
added a commit
that referenced
this pull request
May 31, 2024
Previously `Evaluable`s were evaluated by looping over all dependencies and
calling `Evaluable.evalf`. For `1 + 2 + 3` (not simplified) this boils down to
```python
serialized = [
[Constant(1).evalf],
[Constant(2).evalf],
[Constant(3).evalf],
[Add.evalf, 0, 1],
[Add.evalf, 3, 2],
]
def eval():
values = []
for op, *args in serialized:
values.append(op(*map(values.get, args)))
return values[-1]
```
This PR replaces this mechanism by generating Python code via the new
`evaluable.compile()` function, which produces:
```python
c0 = 1
c1 = 2
c2 = 3
def eval():
v3 = numpy.add(c0, c1)
v4 = numpy.add(v3, c2)
return v4
```
In addition, subtrees that are constant are cached for a second call, providing
a slight performance improvement.
This was referenced Oct 23, 2024
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously
Evaluables were evaluated by looping over all dependencies and callingEvaluable.evalf. For1 + 2 + 3(not simplified) this boils down toThis PR replaces this mechanism by generating Python code via the new
evaluable.compile()function, which produces:In addition, subtrees that are constant are cached for a second call, providing a slight performance improvement.