Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.0] Cache ClangSharp outputs & use cache if we shouldn't run ClangSharp #1868

Merged
merged 20 commits into from
Jan 5, 2024

Conversation

Perksey
Copy link
Member

@Perksey Perksey commented Dec 28, 2023

This introduces a caching mechanism in SilkTouch 3.0 so that we can cache the raw outputs from ClangSharp to allow the rest of the SilkTouch pipeline to be run & developed on platforms other than the ones in which ClangSharp can run. For example, we'd cache raw outputs for the Windows SDK bindings so that if we don't have access to the Windows SDK for example (e.g. we're on macOS), we'd open the cache and feed the cached outputs through the pipeline.

The way I envision this looking is:

  • The bindings that can't be regenerated outside of a specific environment are cached upon regeneration in that environment.
  • We'd have a CI job that will run in that specific environment to regenerate the cache when there's an update to the underlying dependency, similar to how dependabot works.
  • This keeps all developers regardless of platform to be able to clone Silk.NET and regenerate all the bindings, cached or uncached as appropriate.

This was also how the BuildTools cache was intended to work, but I don't know of a single instance of it working properly and even if it did, the cache was output at a pretty late stage in the pipeline such that the only changes observable without regenerating the cache were pretty much just overloads.

@Perksey
Copy link
Member Author

Perksey commented Dec 28, 2023

Oh and this also lets us use Git repos & NuGet packages as ClangSharp inputs, alleviating the need for submodules which have to be manually synchronised.

Copy link
Contributor

github-actions bot commented Dec 28, 2023

File Coverage
All files 10%

Minimum allowed coverage is 0%

Generated by 🐒 cobertura-action against 89dcc7a

@Perksey
Copy link
Member Author

Perksey commented Jan 1, 2024

I think this is ready for review, I'll do the NUKE/CI stuff in a later PR.

The LOC count may seem excessive, but this is because TerraFX has updated and as a result so has our Windows bindings.

The bit I'm interested in scrutiny for is the stuff in sources/SilkTouch as this has been written over many sleep-deprived evenings.

@Perksey Perksey marked this pull request as ready for review January 1, 2024 19:58
@Perksey Perksey requested a review from a team as a code owner January 1, 2024 19:58
Copy link
Contributor

@curin curin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from what I can tell. Didn't really look at the d3d12 code, since I'm assuming that is just raw clangsharp output atm. The actual code for Caching, Sources, and the new AddIncludes looks all pretty sound to me.
The generated OpenGL code also looks a bit better formed.

@Perksey
Copy link
Member Author

Perksey commented Jan 5, 2024

Thank you all for reviewing and your continued support with the 3.0 effort!

@Perksey Perksey merged commit 54d6af3 into develop/3.0 Jan 5, 2024
13 checks passed
@Perksey Perksey deleted the feature/clangsharp-cache branch January 5, 2024 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants