WIP: Populate tox with a script

[sentrivana]

WIP

• Fetch releases for packages to test from PYPI and pick a handful to test against
• Remove -latest tests; instead, we can rerun the script to regenerate tox.ini as it will now always also include the latest release
  • TODO: actually make an action for this to happen on PRs
• Add a CI check for making sure the script/template is in sync with the contents of tox.ini (so that folks get notified if they edit tox.ini directly instead of the template/script)
• ❗Implement an automatic deprecation policy: only test releases that are at most X years old

[codecov]

❌ 284 Tests Failed:

Tests completed	Failed	Passed	Skipped
20845	284	20561	6399

View the top 3 failed tests by shortest run time

 tests.integrations.cohere.test_cohere

Stack Traces | 0s run time

ImportError while importing test module '.../integrations/cohere/test_cohere.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../site-packages/_pytest/python.py:599: in _importtestmodule
    mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../site-packages/_pytest/pathlib.py:533: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.6.15.../x64/lib/python3.6/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:994: in _gcd_import
    ???
<frozen importlib._bootstrap>:971: in _find_and_load
    ???
<frozen importlib._bootstrap>:955: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:665: in _load_unlocked
    ???
.tox/py3.6-cohere-v0.0.8/lib/python3.6.../_pytest/assertion/rewrite.py:171: in exec_module
    exec(co, module.__dict__)
.../integrations/cohere/test_cohere.py:5: in <module>
    from cohere import Client, ChatMessage
E   ImportError: cannot import name 'Client'

 tests.integrations.django.test_basic

Stack Traces | 0s run time

ImportError while importing test module '.../integrations/django/test_basic.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.10.16.../x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
.tox/py3.10-django-v5.1.4/lib/python3.10.../_pytest/assertion/rewrite.py:184: in exec_module
    exec(co, module.__dict__)
.../integrations/django/test_basic.py:40: in <module>
    from tests.integrations.django.utils import pytest_mark_django_db_decorator
.../integrations/django/utils.py:4: in <module>
    import pytest_django
E   ModuleNotFoundError: No module named 'pytest_django'

 tests.integrations.django.test_cache_module

Stack Traces | 0s run time

ImportError while importing test module '.../integrations/django/test_cache_module.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
.tox/py3.10-django-v5.1.4/lib/python3.10.../site-packages/_pytest/pathlib.py:587: in import_path
    importlib.import_module(module_name)
.../hostedtoolcache/Python/3.10.16.../x64/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
.tox/py3.10-django-v5.1.4/lib/python3.10.../_pytest/assertion/rewrite.py:184: in exec_module
    exec(co, module.__dict__)
.../integrations/django/test_cache_module.py:18: in <module>
    from tests.integrations.django.utils import pytest_mark_django_db_decorator
.../integrations/django/utils.py:4: in <module>
    import pytest_django
E   ModuleNotFoundError: No module named 'pytest_django'

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

[sentrivana]

Closing, continuing in #3920