-
Notifications
You must be signed in to change notification settings - Fork 859
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Autobuild python wheels on release tags and create github releases with them #969
base: master
Are you sure you want to change the base?
Conversation
@christian-rauch check it out, it builds!!!! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this. This will be quite useful.
It already looks like a standard pipeline for artifact handling. But I would like to see the following changes:
- Simply use
build/pypangolin-*.whl
as the path to the wheel file instead of moving the files around. Each build will only generate a single wheel file and, therefore, thebuild/pypangolin-*.whl
pattern will only match one file. - I suggest using
ncipollo/release-action
to create the release with the artifacts since it requires less configuration and, above all, does not require a token. I cannot add or update tokens. So this will be a very string limitation that we can easily avoid withncipollo/release-action
. - Please squash your commits together since they are changing the same things multiple times back and forth.
|
||
- name: Add prefix to python_wheel artifact | ||
run: | | ||
for file in ./build/pypangolin-*.whl; do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are you looping over multiple wheel files? The build process should only create a single wheel file for the target platform.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cuz i am unknowing of how to rename one single file with unknown name :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, if you can do ls ./build/pypangolin-*.whl
and get a result, you can also do something like mv ./build/pypangolin-*.whl /tmp/
and will find the file there. Or do you mean something other? In any case, see my other comments, you won't need to move that wheel file around.
- name: Add prefix to python_wheel artifact | ||
run: | | ||
for file in ./build/pypangolin-*.whl; do | ||
mv $file ./release_artifacts/${{matrix.os}}_shared_libs_${{matrix.shared_libs}}.$(basename $file); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this required? The wheel file already exists at build/pypangolin-*.whl
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I experienced that the shared libs on/off wheels ended up with the same name. So renaming them to unique names is a requirement for the later download and merge artifacts step, else they will just overwrite eachother
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean .so
files within the wheel file .whl
? That does not matter. You only need to upload the wheel file and that one usually has a unique name encoded by the OS and architecture. Can you provide a concrete example where this happens?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean that for example the wheels for sharedlibs on and sharedlibs on for ubuntu 22.04 will have the same name. The "shared libs" doesnt affect the name of the wheel. So in order to keep both the shared libs on/off versions i add the prefix containing os and sharedlibs boolean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. You are right. But we would only upload one of the two variants, shared or static, to the release page.
uses: actions/upload-artifact@v4 | ||
with: | ||
name: "release_artifacts_${{matrix.os}}_${{matrix.shared_libs}}" | ||
path: release_artifacts/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not directly use the path build/pypangolin-*.whl
here? This would save you from the renaming above.
You should also add a if-no-files-found: error
here to make sure that the CI fails if there is no file.
.github/workflows/release.yml
Outdated
|
||
steps: | ||
- name: Download build artifact from triggered workflow | ||
uses: dawidd6/action-download-artifact@v2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use the more official actions/download-artifact
here? I think this would be better as it from the same origin that also maintains actions/upload-artifact
and I simply expect that they will always work well together.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be easily swapped. No reason from my side
.github/workflows/release.yml
Outdated
name_is_regexp: true | ||
path: ./release_artifacts | ||
- name: Upload release | ||
uses: softprops/action-gh-release@v2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It probably does not make a huge difference, but I used ncipollo/release-action
before, which is as simple as:
- uses: ncipollo/release-action@v1
with:
artifacts: "release.tar.gz,foo/*.txt"
so you do not have to set the other parameters or care about the token.
@christian-rauch thanks for very good feedback. However i am unable to follow this through due to stress. So this pr is more seen as my limited try at contributing to a solution. Kind regards |
Thanks so far. I will have a look at this and will adjust myself later, if you cannot work on this. |
Did you actually try to install and use your wheel files? pip will complain that the file name is not correct: |
I tried to install the ones on my other repo. And it worked great on ubuntu24.04 migjt be the renaming that doesnt work |
As i mention in the readme you have to install openni firat, check here for the example i succesfully ran: |
Which README? There is no such change in this (your) PR. In any case, it is not just OpenNI that you have to install. Since the CI builds with all dependencies, the resulting binaries also have all the libraries linked. A user of the wheel files therefore has to install all these dependencies. Based on this PR, I created #970. That is the cleaned up version with the fixes I proposed in your PR. The resulting wheel files still have the dependant libraries not included, hence they cannot be used out-of-the-box. |
issues touched by this:
i had to rename the GITHUB_TOKEN to REPO_TOKEN due to github policies, see attached screenshot