Variable Explorer error when class has attribute that references a generator

[puastro]

Description

What steps will reproduce the problem?

I hit the 'reload' button in the corner of the variable explorer. It was open to the class I was looking at. It crashed as soon as I hit it.

Traceback

Traceback (most recent call last):
  File "C:\Users\cgburnsi\AppData\Local\anaconda3\Lib\site-packages\spyder\plugins\ipythonconsole\widgets\namespacebrowser.py", line 54, in get_value
    value = cloudpickle.loads(value)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cgburnsi\AppData\Local\anaconda3\Lib\site-packages\cloudpickle\cloudpickle.py", line 457, in subimport
    __import__(name)
ModuleNotFoundError: No module named 'convert'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\cgburnsi\AppData\Local\anaconda3\Lib\site-packages\spyder\plugins\variableexplorer\widgets\objectexplorer\objectexplorer.py", line 428, in refresh_editor
    data = self.data_function()
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cgburnsi\AppData\Local\anaconda3\Lib\site-packages\spyder\widgets\collectionseditor.py", line 1875, in get_data
    return parent.get_value(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cgburnsi\AppData\Local\anaconda3\Lib\site-packages\spyder\widgets\collectionseditor.py", line 1929, in get_value
    value = self.shellwidget.get_value(name)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\cgburnsi\AppData\Local\anaconda3\Lib\site-packages\spyder\plugins\ipythonconsole\widgets\namespacebrowser.py", line 67, in get_value
    raise ValueError(msg % reason_other)
ValueError: <br><i>An unkown error occurred. Check the console because its contents could have been printed there.</i><br><br><br><b>Note</b>: Please don't report this problem on Github, there's nothing to do about it.

Versions

• Spyder version: 6.0.1 (conda)
• Python version: 3.12.3 64-bit
• Qt version: 5.15.2
• PyQt5 version: 5.15.10
• Operating System: Windows-11-10.0.22631-SP0

Dependencies

# Mandatory:
aiohttp >=3.9.3                  :  3.10.11 (OK)
asyncssh >=2.14.0,<3.0.0         :  2.17.0 (OK)
atomicwrites >=1.2.0             :  1.4.0 (OK)
chardet >=2.0.0                  :  4.0.0 (OK)
cloudpickle >=0.5.0              :  3.0.0 (OK)
cookiecutter >=1.6.0             :  2.6.0 (OK)
diff_match_patch >=20181111      :  20200713 (OK)
github >=2.3.0                   :  2.4.0 (OK)
importlib_metadata >=4.6.0       :  8.5.0 (OK)
intervaltree >=3.0.2             :  3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1 :  8.30.0 (OK)
jedi >=0.17.2,<0.20.0            :  0.19.2 (OK)
jellyfish >=0.7                  :  1.0.1 (OK)
jsonschema >=3.2.0               :  4.23.0 (OK)
keyring >=17.0.0                 :  24.3.1 (OK)
nbconvert >=4.0                  :  7.16.4 (OK)
numpydoc >=0.6.0                 :  1.7.0 (OK)
parso >=0.7.0,<0.9.0             :  0.8.4 (OK)
pexpect >=4.4.0                  :  4.8.0 (OK)
pickleshare >=0.4                :  0.7.5 (OK)
psutil >=5.3                     :  5.9.0 (OK)
pygments >=2.0                   :  2.15.1 (OK)
pylint >=3.1,<4                  :  3.2.7 (OK)
pylint_venv >=3.0.2              :  3.0.3 (OK)
pyls_spyder >=0.4.0              :  0.4.0 (OK)
pylsp >=1.12.0,<1.13.0           :  1.12.0 (OK)
pylsp_black >=2.0.0,<3.0.0       :  2.0.0 (OK)
pyuca >=1.2                      :  1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0        :  3.2.3 (OK)
qstylizer >=0.2.2                :  0.2.2 (OK)
qtawesome >=1.3.1,<1.4.0         :  1.3.1 (OK)
qtconsole >=5.6.0,<5.7.0         :  5.6.0 (OK)
qtpy >=2.4.0                     :  2.4.1 (OK)
rtree >=0.9.7                    :  1.0.1 (OK)
setuptools >=49.6.0              :  75.1.0 (OK)
sphinx >=0.6.6                   :  7.3.7 (OK)
spyder_kernels >=3.0.0,<3.1.0    :  3.0.0 (OK)
superqt >=0.6.2,<1.0.0           :  0.6.7 (OK)
textdistance >=4.2.0             :  4.6.2 (OK)
three_merge >=0.1.1              :  0.1.1 (OK)
watchdog >=0.10.3                :  4.0.1 (OK)
yarl >=1.9.4                     :  1.18.0 (OK)
zmq >=24.0.0                     :  25.1.2 (OK)

# Optional:
cython >=0.21                    :  None (NOK)
matplotlib >=3.0.0               :  3.9.2 (OK)
numpy >=1.7                      :  1.26.4 (OK)
pandas >=1.1.1                   :  2.2.3 (OK)
scipy >=0.17.0                   :  1.13.1 (OK)
sympy >=0.7.3                    :  1.13.2 (OK)

[puastro]

I've continued to look at the code I wrote. This seems to happen when I have a method inside of another method. I pulled the inner method out and made it a staticmethod. As soon as I did that, the variable explorer was able to see the class again. I don't know if that is helpful for others or not, but I know this is an issue outside of Spyder.

[ccordoba12]

@puastro, thanks for taking a closer look at this problem. Could you post a simple code example that generates it?

That way we'll be able to reproduce and try to fix it on our side.

[puastro]

The following is a code that I was working on. It gives me the 'TypeError: cannot pickle 'generator' object' error.

import numpy as np

class Geometry:
    def __init__(self, L, r_max):
        self.L      = L         # [m] Length of Nozzle
        self.r_max  = r_max     # [m] Maximum radius of Nozzle
        
    def generate_radius_profile(self, n_x):
        self.r  = np.array(self.nozzle_radius(x) for x in np.linspace(0, self.L, n_x))
    
    # Define nozzle geometry function (e.g., a simple converging-diverging nozzle)
    def nozzle_radius(self,x):
        if x < 5.0:
            return 0.5 + 0.1 * x
        else:
            return 1.0 + 0.05 * (x - 5.0)
        
    def get_radius_at(self, x):
        return self.nozzle_radius(x)
    
class Grid:
    def __init__(self, L, r_max, n_x, n_r, geometry):
        self.L = L
        self.r_max = r_max
        self.n_x = n_x
        self.n_r = n_r
        self.geometry = geometry

        self.dx = L / (n_x - 1)
        self.dr = r_max / (n_r - 1)
        
        # Create grid arrays for x and r
        self.x = np.linspace(0, L, n_x)
        self.r = np.linspace(0, r_max, n_r)
        
        # Generate the radius profile based on nozzle geometry
        self.geometry.generate_radius_profile(n_x)
        
        # Generate grid points using the nozzle geometry
        self.X, self.R = np.meshgrid(self.x, self.geometry.r)
    
if __name__ == '__main__':
    
    geom = Geometry(L=10.0, r_max=1.2)
    grid = Grid(L=10.0, r_max=1.2, n_x=100, n_r=50, geometry=geom)

[puastro]

The only way I found to solve this was to convert generator expression in the generate_radius_profile(self, n_x) to a list using the following line:

self.r = np.array(list(self.nozzle_radius(x) for x in np.linspace(0, self.L, n_x)))

after doing that, it allowed me to use the variable explorer as normal.

[ccordoba12]

Thanks for the reproducible example @puastro! I got a similar error on Linux too.

After a bit of googling I found that generators can't be pickled, and that's the cause of this problem:

https://stackoverflow.com/questions/7180212/why-cant-generators-be-pickled

We'll report a better message about this limitation in a future version.