Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of hard-coded expected values in unit testing #44

Open
apulsipher opened this issue Nov 14, 2024 · 0 comments
Open

Improve handling of hard-coded expected values in unit testing #44

apulsipher opened this issue Nov 14, 2024 · 0 comments
Labels
CI/CD Creates or updates workflows

Comments

@apulsipher
Copy link
Contributor

At the moment, many unit tests produce different output depending on the machine they are run on, likely caused by how different systems handle random values (even with the same seed). So, a test might succeed locally but fail during the automated check with GitHub Actions. We have added a tolerance parameter to the tests where this has been a problem, but in some cases that tolerance is large enough that it isn't helpful for verifying output. This behavior needs to be updated to make our tests more robust without impeding the development experience.

One possible solution is testing for out of bounds instead of matching to expected values. That way we check for valid output, rather than specific output.

@apulsipher apulsipher added the CI/CD Creates or updates workflows label Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD Creates or updates workflows
Projects
None yet
Development

No branches or pull requests

1 participant