Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Generating the e2e test matrix #5763

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

CI: Generating the e2e test matrix #5763

wants to merge 2 commits into from

Conversation

kantai
Copy link
Member

@kantai kantai commented Jan 28, 2025

This PR is a first pass at trying to generate the test matrix automatically. Right now, I'm just trying to make sure that it gets a single test to run using the excludes functions. Once I'm convinced that works, I'll update it to match the current exclusion set.

@kantai kantai requested a review from a team as a code owner January 28, 2025 22:02
@wileyj
Copy link
Collaborator

wileyj commented Jan 29, 2025

interesting!

have you considred nextest's filtersets here as well? we'd still need a matrix, but the pattern matching might be easier than what you're doing here

https://nexte.st/docs/filtersets/

@kantai
Copy link
Member Author

kantai commented Jan 29, 2025

have you considred nextest's filtersets here as well? we'd still need a matrix, but the pattern matching might be easier than what you're doing here

After looking into it, I don't think it would help that much. The filterset language doesn't support matching "ignored" only (which is how we indicate these tests), so it would only be helpful to match the binary stacks-node. But it wouldn't change the jq filter at all, because the JSON still needs to be altered to match the matrix spec.

* use nextest list to generate the set of e2e tests for bitcoin-tests (this switches the workflow from being an "inclusion list" to an "exclusion list").
* this changeset matches the current set of tests run in `develop`. it does not try to enable tests that were perhaps overlooked.
@kantai kantai force-pushed the ci/generate-tests branch from ef5045d to 4d96cf9 Compare January 29, 2025 15:28
@kantai kantai self-assigned this Jan 29, 2025
jferrant
jferrant previously approved these changes Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants