Reusing snapshot with different values that match dirty-equals causes tests to fail when fixing

[alexmojaki]

As a simple example, this test passes normally but fails with --inline-snapshot=fix:

def test_bug() -> None:
    for x in 'ab':
        assert x == snapshot(IsStr())

I assume this is because snapshot expects its contents to be the same in each assert ... == snapshot(...). There's no helpful error message, just AssertionError.

As a more practical example, this test fails when fixing:

https://github.com/pydantic/logfire/blob/3c521842496dd110357c086abff8930cb9b446e2/tests/test_cli.py#L221-L222

@pytest.mark.parametrize('webbrowser_error', [False, True])
def test_auth(tmp_path: Path, webbrowser_error: bool) -> None:
    ...
            assert console_calls == snapshot(
            [
                IsStr(regex=r'^\(file=.*'),

The parametrize causes the test to run twice, so the snapshot which is in the same place is expected to contain the same thing, even though there's no loops. But its exact contents depend on the value of tmp_path which changes each time. The IsStr(regex=r'^\(file=.*') is supposed to make this not matter.

[15r10nk]

Thank you for the report.

I assume this is because snapshot expects its contents to be the same in each assert ... == snapshot(...). There's no helpful error message, just AssertionError.

You are right. The new value should pass all comparisons. There is no helpful error message because I have to disable pytest-assert-rewriting because I need executing to fix the bugs.

But the actual problem is that the first value "a" is not equal to "b" in the second iteration. This should be fixable.

[15r10nk]

@alexmojaki, you can try this branch #114 and this samuelcolvin/dirty-equals#105.
But a new release should come soon (in both cases).

[alexmojaki]

This is not urgent, I'll wait for the release. Thanks for your quick response!

[15r10nk]

fixed in 0.13.2