Hey @gb119, thanks for reporting. You said:

Needless to say there is a "conda run" in the environment and most of the time spyder seems to find it ok!

Do you have several Anaconda/Miniconda/Miniforge installations in your system? Maybe in one of them conda is too old and doesn't support the conda run command. And sometimes Spyder is using it.

I've tried start spyder from the command line with the debugging output on but I can't (easily) see where it decides that conda run doesn't exist.

conda run is executed automatically when starting a new console (when we detect the interpreter is in a conda env), so that error is reported automatically by conda, not by Spyder.

From the "sticky" nature of the problem and the fact it is intermittent (and not predictable as far as I can tell) makes me think that there must be a race condition or a timeout or similar at the point where spyder is deciding it is working in a conda environment - but I've not looked at the code enough to know where to look and what to look for.

Well, the "is in conda env" check is very simple (just verifying that a directory exists) and it runs before calling conda run. The logic is in spyder/plugins/ipythonconsole/utils/kernelspec.py, in case you want to check it.

Sorry, that's not the most useful bug report!

Yep, I know. However, I saw that you have the Automatically load Numpy and Matplotlib modules enabled. Just to be sure, please disable it (along with any other custom option for the IPython console you have, one by one) and see if you can reproduce the problem. Maybe one of those options is interfering with our conda detection mechanism somehow.

By chance I noticed that after I had this problem I sometimes got the internal error dialog box and spotted that in utils/conda.py in get_spyder_conda_channel() there is a possible condition in which you index None. The test in line 196 needs to check first that channel_url is not None before looking for "".

Good catch! I'll submit a PR to fix it.

I have no idea if this is actually the cause of my problem (it seems to involve network paths not being consistently present), but it's definitely not right....

What do you mean by this? Is your conda command placed in a network drive? Or perhaps your environments?

Hey @gb119, any follow-up about my questions above?

Otherwise, I'm afraid I'll have to close this issue.

Sorry - end of academic term means I get a bit sidetracked!

Hey @gb119, thanks for reporting. You said:

Needless to say there is a "conda run" in the environment and most of the time spyder seems to find it ok!

Do you have several Anaconda/Miniconda/Miniforge installations in your system? Maybe in one of them conda is too old and doesn't support the conda run command. And sometimes Spyder is using it.

Yes, I've got a very ancient Miniconda as well as a current Anaconda. I need the ancient miniconda becuse of other projects that require a Python 3.6 dll that is compiled with particular VS C++ version (!). However, the intermittent nature of this problem seems to be not straightforwardly that I pick up the ancient conda.

I've tried start spyder from the command line with the debugging output on but I can't (easily) see where it decides that conda run doesn't exist.

conda run is executed automatically when starting a new console (when we detect the interpreter is in a conda env), so that error is reported automatically by conda, not by Spyder.

From the "sticky" nature of the problem and the fact it is intermittent (and not predictable as far as I can tell) makes me think that there must be a race condition or a timeout or similar at the point where spyder is deciding it is working in a conda environment - but I've not looked at the code enough to know where to look and what to look for.

Well, the "is in conda env" check is very simple (just verifying that a directory exists) and it runs before calling conda run. The logic is in spyder/plugins/ipythonconsole/utils/kernelspec.py, in case you want to check it.

I;ll take a look once I've got 6.0.3 installed and fixed my version to deal with the known issue with the Projects plugin widget and remote drives. Hope to spend asome time with this this weekend.

Sorry, that's not the most useful bug report!

Yep, I know. However, I saw that you have the Automatically load Numpy and Matplotlib modules enabled. Just to be sure, please disable it (along with any other custom option for the IPython console you have, one by one) and see if you can reproduce the problem. Maybe one of those options is interfering with our conda detection mechanism somehow.

I've had this disabled and I still get the problem periodically. It's very irritating because it is intermittent!

So a bit of further investigation ow I know where to look. I think I'm just a very weird corner case because I have an ancient conda at the end of my PATH. It seems that under some circumstances (and this is the bit I don't understand yet), Spyder is getting launched without CONDA_EXE being set in the environment variables and then in spyder/utils/conda.py:find_conda I fall through to searching the PATH whereupon it finds the incorrect conda.

The workaround that should work consistently is to manually make sure I have CONDA_EXE set in my environment variables, but I still don't understand why it is an intermittent error and most of the time it will start fine and then sometimes it falls over.

I also wonder whether it would be sensible to do a version check on conda before assuming that conda run will work - that at least would give a less confusing failure.

If you want to close this issue, then fine I'll accept I'm just a rogue data point - the main (new) reason for this issue was the problem with spyder_conda_channel() that you've already fixed.

Actually, before closing this one - there's another bad code smell....

spyder/utils/programs.py line 121
def find_program(basename, extra_paths=[]):

Probably a bad idea to pass a mutable empty list as the default value of a keyword argument. Shouldn't this be:
def find_program(basename, extra_paths=None): ... if extra_paths=[] if extra_paths is None?

(...and just spotted also in is_program_installed()

Finally in is_program_installed(), it looks like we search in he PATH before we look in the paths passed in via extra_paths - I would have thought that the contents of extra_paths were more likely to be correct places to look and therefore we should look in them before the system PATH?

I don't know if this is why I'm getting inconsistencies, but I could believe a race condition with default arguments getting mutated when accessing project files off very slow file systems might do the trick.

Ok, just done a ^\s+def\s.*=([\{\}\[\]]{2}) search across the Spyder code base and got multiple hits. Probably they don't matter in practice but perhaps they should be cleaned up? Happy to have shot at a PR so I'm not just a user complaining...

I also wonder whether it would be sensible to do a version check on conda before assuming that conda run will work - that at least would give a less confusing failure.

This was reported in #22554 and we'll try to fix it in one of the coming 6.0.x releases.

Finally in is_program_installed(), it looks like we search in he PATH before we look in the paths passed in via extra_paths - I would have thought that the contents of extra_paths were more likely to be correct places to look and therefore we should look in them before the system PATH?

I don't know if this is why I'm getting inconsistencies, but I could believe a race condition with default arguments getting mutated when accessing project files off very slow file systems might do the trick.

Yeah, I think that's a good idea. @mrclary, could you take a look at that?

Ok, just done a ^\s+def\s.*=([{}[]]{2}) search across the Spyder code base and got multiple hits. Probably they don't matter in practice but perhaps they should be cleaned up? Happy to have shot at a PR so I'm not just a user complaining...

Thanks for your help with that! I agree it's a good idea to fix those cases.

So a few questions.

Yes, I've got a very ancient Miniconda as well as a current Anaconda. I need the ancient miniconda becuse of other projects that require a Python 3.6 dll that is compiled with particular VS C++ version (!). However, the intermittent nature of this problem seems to be not straightforwardly that I pick up the ancient conda.

Can you tell me what is the conda version in your ancient miniconda installation?
As near as I can tell, conda run was introduced (reintroduced?) at version 4.6.0. However, the --live-stream and --no-capture-output flags were introduced at version 4.9.0.

Nevertheless, I'm interested in the intermittent nature of the problem because, even with the extra_paths at the end of the search path list, the search list path should be deterministic. Are you running Spyder under different circumstances or environments?

I agree that we can move the extra_paths to the "front of the line", but that may not change the underlying intermittency. We may want to find the first version of conda >=4.9, or maybe the latest version available. Checking the conda version, if there are multiple available, may be expensive, so maybe this could be done once at startup and cached.

So a few questions.

Can you tell me what is the conda version in your ancient miniconda installation? As near as I can tell, conda run was introduced (reintroduced?) at version 4.6.0. However, the --live-stream and --no-capture-output flags were introduced at version 4.9.0.

I'll need to confirm this when I back in front of the right computer - but it could well be older than 4.6.x! (to explain, I'm also maintaining a large NI LabVIEW 2018 codebase - that links to a Python 3.6.5 dll and is limited to the versions it will accept, can't cope without the dll being in the PATH and that codebase also needs a few other packages installed! Legacy code fun...).

Nevertheless, I'm interested in the intermittent nature of the problem because, even with the extra_paths at the end of the search path list, the search list path should be deterministic. Are you running Spyder under different circumstances or environments?

So I think it is correlated to, but not deterministically, to having spyder projects on network drives that are either very slow or sometimes missing. From the poking I've done and somewhat limited understanding of how spyder works, it seemed to me that with some of the functions used in locating conda taking default arguments that were mutable, it would be possible that in the process of loading a project from a slow network drive there might be a race condition that sometime resulted in the default argument being mutated in such a way as to locate a conda in the path before locating conda in the environment from which Spyder is running.

I think my setup is some really weird corner case, but perhaps it is serving to show that the conda location is not quite as robust as it could be....

I agree that we can move the extra_paths to the "front of the line", but that may not change the underlying intermittency. We may want to find the first version of conda >=4.9, or maybe the latest version available. Checking the conda version, if there are multiple available, may be expensive, so maybe this could be done once at startup and cached.

Okay, I found the problem. The relevant code is here:

The problem is the use of set which ensures that there are no duplicate paths, but does not guarantee order! This would be a problem even if req_paths (which contains extra_paths) is moved in front of PATHs. I'll fix this and place extra_paths at the front.

@gb119, how are you launching Spyder? If you can explicitly set the CONDA_EXE environment variable to a known conda executable >4.9.0, then Spyder should always find that first because Spyder does not bother searching through paths if this variable is set.

Okay, I found the problem. The relevant code is here:

spyder/spyder/utils/programs.py

Line 115 in 3f64ee3

for path in set(os.environ['PATH'].split(os.pathsep) + req_paths):

The problem is the use of set which ensures that there are no duplicate paths, but does not guarantee order! This would be a problem even if req_paths (which contains extra_paths) is moved in front of PATHs. I'll fix this and place extra_paths at the front.

Ah, sorry didn't realise you didn't know where I was looking - there's already a pull request from me to remove the default mutable arguments from that function (and a bunch of others).

Guess it just needs:
for path in {path:None for path in req_paths+ os.environ['PATH'].split(os.pathsep)}

Although in practice set() seems often to preserve item order in modern cpython.

@gb119, how are you launching Spyder? If you can explicitly set the CONDA_EXE environment variable to a known conda executable >4.9.0, then Spyder should always find that first because Spyder does not bother searching through paths if this variable is set.

I have now put CONDA_EXE explicitly int he system environment variables having crawled over that bit of code (it wasn't there as a side effect of the weirdness of keeping the ancient miniconda in PATH and the modern anaconda3 out of it.

Ah, sorry didn't realise you didn't know where I was looking - there's already a pull request from me to remove the default mutable arguments from that function (and a bunch of others).

Technically, the mutable argument is unrelated to this issue.

Guess it just needs: for path in {path:None for path in req_paths+ os.environ['PATH'].split(os.pathsep)}

This will result in pyenv + conda + extra_paths + system paths, not extra_paths + conda + pyenv + system paths.