The outer layer of dev docs. #23
Conversation
Additional more detailed docs on the basic structure of scie-pants and its main process execution flows is coming seperately. Work towards pantsbuild#7
|
|
||
| There are just a few guiding principles to keep in mind as alluded to above: | ||
| + The `scie-pants` hot path should be fast: It currently launches Pants in ~one millisecond and that | ||
| should hold steady. |
There was a problem hiding this comment.
Could we have/are there any perf tests that may run automatically/in CI to ensure this will hold true over time?
There was a problem hiding this comment.
We should. I'm not sure exactly how to do it yet. Pants gets in the way since its slow. There are 3 separate binary executions in the hot path:
- The scie-jump launches the embedded scie-pants.bin binary as the default command.
- The scie-pants.bin binary finds pants.toml, parses the pants_version and determines the proper Python version and then re-launches the scie-pants scie specifying a few env vars to convey what it found and SCIE_BOOT=pants to launch the pants command contained in the scie.
- The scie-jump launches the pants command.
- The pants console script in the previously installed Pants venv runs.
Each of the 1st 3 steps takes ~300us. The 4th step takes 500ms min (for pants -V for example).
Perhaps there is a way to shim out 4 with something other than the pants console script that runs in consistent time on the order of 1ms or less (like bash -V). Perhaps even that could be avoided.
Right now I just run with RUST_LOG=debug which gives the timings I just described for the 1st 3 steps in log lines.
There was a problem hiding this comment.
#24 now tracks this.
There was a problem hiding this comment.
I will say though to not really worry about this if it feels scary when considering contribution. This is both because the actual scie-pants binary does almost nothing; so its hard to make it slow, and because most feature work will just be, at most, plumbing reading of new toml values from an already priced in pants.toml parse and setting new env vars for sub-processes - both of which are really hard to make slow in non-obvious ways.
There was a problem hiding this comment.
The slowest thing it does actually, by almost an order of magnitude, is handle .pants.bootstrap! So this is step 2 which takes ~300us with no .pants.bootstrap and adding in a ~minimal GIT_COMMIT ./pants.bootstrap adds >2000us.
Additional more detailed docs on the basic structure of scie-pants and
its main process execution flows is coming separately.
Work towards #7