[CI] Add CI workflow to run compute-benchmarks on incoming syclos PRs

[ianayl]

This PR:

• adds a "benchmark" mode to sycl-linux-run-tests.yml, which benchmarks a given SYCL branch/build using compute-benchmarks
  • stores benchmark results in a git repo, and
  • aggregates benchmark results in order to produce a median, which is used to pass or fail the benchmark workflow

The current plan is to enable this benchmark to run nightly in order to catch regressions, although there is potential for this workflow to be used in precommit. As a result, a lot of components in this workflow are either separate reusable components, or directly written with precommit in mind. The current benchmarking workflow functions as so:

1. An "aggregate" workflow is ran, which aggregates historic benchmark results in the aforementioned git repo, and produces a historical median
   • This calls upon aggregate.py to handle the actual compute heavy-lifting
2. The core benchmarking workflow is ran:
   • This calls upon benchmark.sh, which handles the logic for building and running compute-benchmarks
   • Then, compare.py is called upon for the actual comparing of benchmark data against the historical median generated prior

The workflows are fully configurable via benchmark-ci.conf; enabled compute-benchmarks tests can be configured via enabled_tests.conf.

Feel free to test out the workflow via manual dispatches of sycl-linux-run-tests.yml on branch benchmarking-workflow, but be aware that the run currently will always fail, as Github repository secrets are not yet added.

[aelovikov-intel]

That isn't very useful. I'd hope there are some GH actions that can draw some graphs with performance data of the project over time (if that data is "reported" properly to it).

[ianayl]

Thanks for the review Andrei! Udit and I were talking about graphing this a while back: As of right now this workflow has not yet been approved, as many details regarding the workflow needs to be hashed out. However, we do plan on graphing this data if this workflow gets approved.

The current idea that I have is to store/cache benchmarks periodically over time as long-term artifacts, and then graphing it using either a python or js action. Although, I have yet to sort out the details. This may potentially be janky, so I'm open to new ideas.

[uditagarwal97]

Yes, we are yet to decide whether we will keep sycl-bench in intel/llvm CI. I guess we will also be exploring other micro benchmarking suites to find the ones appropriate for our use. This POC PR was just to evaluate the time it takes to run sycl-bench in CI.