Duplicate tabs when using pyfakefs

[Aran-Fey]

Something must be going wrong internally with the handling of file paths. The effects I've observed are:

1. Hitting a breakpoint in the "current file" opens a duplicate tab.

   Before starting the debugger:
   

   After:
   

   The file path is exactly the same.

2. Creating a new breakpoint in this duplicate tab creates a disabled breakpoint. (See line 89.)
   

3. Breakpoints in other files are never hit. For example, if I set a breakpoint inside my load_user_app function, it's completely ignored.

All in all, the debugger is completely unusable.

What's also interesting is that everything works as expected if I try to reproduce it with a new, minimal file:

P.S.: No symlinks or hardlinks or directory junctions are involved. I do have a directory junction set up (from C:\ to D:), but these issues occur even if I open the project directly from the D:\ drive.

[rchiodo]

There's two things I can think of that might cause the problem.

1 - You're doing remote debugging and the pathmapping is the wrong case
2 - You're running your code as a script through like exec

[rchiodo]

A log might help to diagnose where debugpy is attempting to bind the breakpoints. See the directions here:
https://github.com/microsoft/debugpy/wiki/Enable-debugger-logs

[Aran-Fey]

The issue where it doesn't hit breakpoints seems to have solved itself, oddly enough. It's still opening duplicate tabs though.

The directions for obtaining the log seem to be out of date? I can't see any log files in ms-python.python-*. There is also a ms-python.debugpy-2024.12.0-win32-x64 folder, but that one contains 10 log files and I'm not sure which one you need.

[rchiodo]

Every time you run it generates a bunch log files. The easiest way to get us the log files is to delete them all and run once more. I'll update the directions to look for ms-python.debugpy.

If the breakpoints are binding, the problem may be with VS code then. Debugpy just gives back a file path to open for a stack frame. VS code would decide if the file was a new tab or not. But the path in the log files might give us some clue.

[rchiodo]

Similar VS code issue:
microsoft/vscode#225290

[Aran-Fey]

It seems to create 5 log files with each run. There's debugpy.adapter-, debugpy.launcher-, etc. and also a debugger.vscode_*.log, which I assume is the one you need.

debugger.vscode_839fb24b-acb8-4d5b-ac71-31bc2d9f37e2.log

Not using remote debugging or exec by the way. I am using pytest though, which I guess is somewhat similar to exec? I did a quick test and if I don't use pytest then the duplicate tab issue seems to go away. (Though, since all of these issues seem to come and go as they like, that may or may not be the reason.)

[rchiodo]

Here's the response we give to VS code:

14214 Client <-- Adapter:
{
    "seq": 41,
    "type": "response",
    "request_seq": 18,
    "success": true,
    "command": "stackTrace",
    "body": {
        "stackFrames": [
            {
                "id": 2,
                "name": "import_app_module",
                "line": 113,
                "column": 1,
                "source": {
                    "path": "D:\\Users\\Aran-Fey\\Desktop\\folder\\coding\\python\\rio\\rio\\cli\\run_project\\app_loading.py",
                    "sourceReference": 7
                }
            }
        ],
        "totalFrames": 3
    }
}

Is that casing the same as what VS code is showing in the tab?

[Aran-Fey]

Yep, exactly the same. But I just noticed that the new tab is in read-only mode. That's probably the reason why it opens a 2nd one.

[rchiodo]

Yeah, right after that VS code asks the debugger for the source. Like it's using that to create the tab. Super weird.

14226 Client --> Adapter:
{
    "command": "source",
    "arguments": {
        "sourceReference": 7,
        "source": {
            "path": "D:\\Users\\Aran-Fey\\Desktop\\folder\\coding\\python\\rio\\rio\\cli\\run_project\\app_loading.py",
            "sourceReference": 7
        }
    },
    "type": "request",
    "seq": 19
}

Why would it need to ask the debugger? @connor4312 do you know what circumstances VS code asks the debugger for the source? In a normal debug session, we only get a stack trace request.

[connor4312]

Your stackframe has a source with a non-zero sourcereference, which means it doesn't exist on disk, so the DA is asked for it:

        "stackFrames": [
            {
                "id": 2,
                "name": "import_app_module",
                "line": 113,
                "column": 1,
                "source": {
                    "path": "D:\\Users\\Aran-Fey\\Desktop\\folder\\coding\\python\\rio\\rio\\cli\\run_project\\app_loading.py",
                    "sourceReference": 7 // <-
                }
            }
        ],

See docs for the properties in the Source type

[rchiodo]

Okay that's here in our code then:

debugpy/src/debugpy/_vendored/pydevd/pydevd_file_utils.py

Line 816 in 4d86a42

def _map_file_to_client(filename, cache=norm_filename_to_client_container):

@Aran-Fey can you upload the pydevd log. It should have more information.

[Aran-Fey]

debugpy.pydevd.29588.log

[rchiodo]

Sorry this environment variable needs to be set globally first:

DEBUG_PYDEVD_PATHS_TRANSLATION=True

That adds more logging output for path translation.

[Aran-Fey]

Is there a way to do it without having to restart my PC? (Or logging out and back in.) I have a lot of stuff open...

[rchiodo]

I think this should work in your launch.json:

      "env": {
        "DEBUG_PYDEVD_PATHS_TRANSLATION": "True"
      }

It seems to have been picked up when I did it.

When the app runs, it should output something from this code if it worked:

    if DEBUG_CLIENT_SERVER_TRANSLATION:
        print("pydev debugger: client OS: %s" % (os,))

[Aran-Fey]

I had to play around with the launch.json until the bug happened again, so these logs are from running pytest test_app_loading.py.

debugpy.pydevd.18648.log
debugger.vscode_c95fc07b-4791-4100-a2ae-c992438b224a.log

[rchiodo]

Did you get this output?

pydev debugger: client OS: WINDOWS

I don't see any extra translation logging. Maybe the source reference is being set somewhere else. We'd likely need a copy of the test for this so we can debug it locally.

[Aran-Fey]

I did see pydev debugger: client OS: WINDOWS, yes.

You can find the code here. The file is tests/test_cli/test_app_loading.py.

[rchiodo]

It's your code's fault. It's overwriting the file system.

Internally we get this error when trying to detect if tests/test_cli/test_app_loading.py is a file or not:

FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: 'C:\Users\rchiodo\source\testing\rio\tests\test_cli'

That indicates to the debugger to return a source_reference.

[Aran-Fey]

Great. Looks like we'll be playing a game of "it's not this tool's fault, the other one should fix it".

[rchiodo]

The test you have is overwriting the file system. That's what I meant. It's your specific test. It's using the pyfakefs which patches the file system at the OS module level.

[rchiodo]

This module:
https://pypi.org/project/pyfakefs/

[Aran-Fey]

And of course it's pyfakefs's responsibility to add support for debugpy, and not the other way round.

[rchiodo]

Yes debugpy can't determine how pyfakefs is patching the os module. Pyfakefs would need to have a way to let some file paths use the normal OS and you'd specify that in your test.