You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, many unit tests produce different output depending on the machine they are run on, likely caused by how different systems handle random values (even with the same seed). So, a test might succeed locally but fail during the automated check with GitHub Actions. We have added a tolerance parameter to the tests where this has been a problem, but in some cases that tolerance is large enough that it isn't helpful for verifying output. This behavior needs to be updated to make our tests more robust without impeding the development experience.
One possible solution is testing for out of bounds instead of matching to expected values. That way we check for valid output, rather than specific output.
The text was updated successfully, but these errors were encountered:
At the moment, many unit tests produce different output depending on the machine they are run on, likely caused by how different systems handle random values (even with the same seed). So, a test might succeed locally but fail during the automated check with GitHub Actions. We have added a
tolerance
parameter to the tests where this has been a problem, but in some cases that tolerance is large enough that it isn't helpful for verifying output. This behavior needs to be updated to make our tests more robust without impeding the development experience.One possible solution is testing for out of bounds instead of matching to expected values. That way we check for valid output, rather than specific output.
The text was updated successfully, but these errors were encountered: