Bug 732400 – sys-apps/portage-2.3.103 does not pull in depends with --newuse on package rebuild: (was: dev-python/oct2py-4.0.6 fails tests: ModuleNotFoundError: No module named 'markupsafe')

• Status: CONFIRMED
• Product: Portage Development

[Description Sergei Trofimovich (RETIRED) 2020-07-13 06:55:56 UTC]

"""
...
python3.6 setup.py build
...
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/lib64/python3.6/site-packages/_pytest/assertion/rewrite.py", line 304, in load_module
    exec(co, mod.__dict__)
  File "/usr/lib64/python3.6/site-packages/pytest_httpbin/plugin.py", line 3, in <module>
    from httpbin import app as httpbin_app
  File "/usr/lib64/python3.6/site-packages/httpbin/__init__.py", line 3, in <module>
    from .core import *
  File "/usr/lib64/python3.6/site-packages/httpbin/core.py", line 18, in <module>
    from flask import Flask, Response, request, render_template, redirect, jsonify as flask_jsonify, make_response, url_for, abort
  File "/usr/lib64/python3.6/site-packages/flask/__init__.py", line 14, in <module>
    from jinja2 import escape
  File "/usr/lib64/python3.6/site-packages/jinja2/__init__.py", line 6, in <module>
    from markupsafe import escape
ModuleNotFoundError: No module named 'markupsafe'
 * ERROR: dev-python/oct2py-4.0.6::gentoo failed (test phase):
 *   (no error message)
 *
 * Call stack:
 *     ebuild.sh, line  125:  Called src_test
 *   environment, line 3804:  Called distutils-r1_src_test
 *   environment, line 1250:  Called _distutils-r1_run_foreach_impl 'python_test'
 *   environment, line  378:  Called python_foreach_impl 'distutils-r1_run_phase' 'python_test'
 *   environment, line 3409:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_test'
 *   environment, line 2718:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_test'
 *   environment, line 2716:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_test'
 *   environment, line  838:  Called distutils-r1_run_phase 'python_test'
 *   environment, line 1187:  Called python_test
 *   environment, line 3764:  Called die
 * The specific snippet of code:
 *       py.test -v -v || die
"""

Probably a missing dependency against python version. I have markupsafe installed but against python3_7, not 3_6:

dev-python/markupsafe-1.1.1::gentoo was built with the following:
USE="-test" PYTHON_TARGETS="python2_7 python3_7 (-pypy3) -python3_6 -python3_8 -python3_9"

[Comment 1 Sergei Trofimovich (RETIRED) 2020-07-13 06:59:47 UTC]

Created attachment 649048 [details]
=dev-python:oct2py-4.0.6.FAIL

[Comment 2 Sergei Trofimovich (RETIRED) 2020-07-13 07:03:21 UTC]

Might not be direct. oct2py's dependency problem? Full backtrace:

"""
 * python3_6: running distutils-r1_run_phase python_test
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.6/py.test", line 11, in <module>
    load_entry_point('pytest==4.6.11', 'console_scripts', 'py.test')()
  File "/usr/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 65, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 214, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/usr/lib64/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib64/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib64/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/lib64/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib64/python3.6/site-packages/_pytest/helpconfig.py", line 94, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/usr/lib64/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib64/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 789, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 997, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib64/python3.6/site-packages/_pytest/config/__init__.py", line 943, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/usr/lib64/python3.6/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/usr/lib64/python3.6/site-packages/importlib_metadata/__init__.py", line 96, in load
    module = import_module(match.group('module'))
  File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/usr/lib64/python3.6/site-packages/_pytest/assertion/rewrite.py", line 304, in load_module
    exec(co, mod.__dict__)
  File "/usr/lib64/python3.6/site-packages/pytest_httpbin/plugin.py", line 3, in <module>
    from httpbin import app as httpbin_app
  File "/usr/lib64/python3.6/site-packages/httpbin/__init__.py", line 3, in <module>
    from .core import *
  File "/usr/lib64/python3.6/site-packages/httpbin/core.py", line 18, in <module>
    from flask import Flask, Response, request, render_template, redirect, jsonify as flask_jsonify, make_response, url_for, abort
  File "/usr/lib64/python3.6/site-packages/flask/__init__.py", line 14, in <module>
    from jinja2 import escape
  File "/usr/lib64/python3.6/site-packages/jinja2/__init__.py", line 6, in <module>
    from markupsafe import escape
ModuleNotFoundError: No module named 'markupsafe'
"""

[Comment 3 Sergei Trofimovich (RETIRED) 2020-07-13 07:26:08 UTC]

Probably jinja should pull in markupsafe dependency:

"""
$ fgrep markupsafe /usr/lib64/python3.6/site-packages/jinja2/__init__.py
from markupsafe import escape
from markupsafe import Markup

$ emerge --info jinja markupsafe

dev-python/jinja-2.11.2::gentoo was built with the following:
USE="-doc -examples -test" PYTHON_TARGETS="python2_7 python3_6 python3_7 (-pypy3) -python3_8 -python3_9"
FEATURES="protect-owned news network-sandbox binpkg-docompress parallel-fetch unmerge-logs config-protect-if-modified xattr sandbox userfetch strict ipc-sandbox unknown-features-warn sfperms ebuild-locks unmerge-orphans qa-unresolved-soname-deps fixlafiles usersandbox binpkg-logs fail-clean binpkg-dostrip merge-sync usersync userpriv assume-digests distlocks preserve-libs multilib-strict pid-sandbox"


dev-python/markupsafe-1.1.1::gentoo was built with the following:
USE="-test" PYTHON_TARGETS="python2_7 python3_7 (-pypy3) -python3_6 -python3_8 -python3_9"
FEATURES="parallel-fetch multilib-strict network-sandbox userfetch fail-clean merge-sync unknown-features-warn binpkg-logs preserve-libs fixlafiles binpkg-docompress sfperms unmerge-orphans assume-digests distlocks news protect-owned ebuild-locks usersandbox unmerge-logs usersync pid-sandbox stricter userpriv xattr sandbox config-protect-if-modified binpkg-dostrip qa-unresolved-soname-deps ipc-sandbox"
"""

Attempt to rebuild package does not seem to pull it in without enabled tests:

With test (ok):
"""
$ PYTHON_TARGETS="python2_7 python3_6 python3_7" emerge -pvN1 dev-python/jinja

[ebuild   R    ] dev-python/markupsafe-1.1.1::gentoo  USE="test*" PYTHON_TARGETS="python2_7 python3_6* python3_7 (-pypy3) -python3_8 -python3_9" 0 KiB
[ebuild   R    ] dev-python/jinja-2.11.2::gentoo  USE="test* -doc -examples" PYTHON_TARGETS="python2_7 python3_6 python3_7 (-pypy3) -python3_8 -python3_9" 0 KiB

Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB

 * IMPORTANT: 34 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.
"""

Without tests (bad):
"""
$ PYTHON_TARGETS="python2_7 python3_6 python3_7" USE=-test FEATURES=-test emerge -pvN1 dev-python/jinja

Total: 0 packages, Size of downloads: 0 KiB
"""

[Comment 4 Sergei Trofimovich (RETIRED) 2020-07-13 07:32:22 UTC]

Oh, it's a side-effect of -N (--newuse). Without it the dependency is pulled in correctly:

"""
$ PYTHON_TARGETS="python2_7 python3_6 python3_7" USE=-test FEATURES=-test emerge -pv1 dev-python/jinja

[ebuild   R    ] dev-python/markupsafe-1.1.1::gentoo  USE="-test" PYTHON_TARGETS="python2_7 python3_6* python3_7 (-pypy3) -python3_8 -python3_9" 0 KiB
[ebuild   R    ] dev-python/jinja-2.11.2::gentoo  USE="-doc -examples -test" PYTHON_TARGETS="python2_7 python3_6 python3_7 (-pypy3) -python3_8 -python3_9" 0 KiB

Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB
"""

[Comment 5 Sergei Trofimovich (RETIRED) 2020-07-13 07:36:21 UTC]

Reassigning to dev-portage@, assuming it's an unexpected behaviour.

[Comment 6 Martin Mokrejš 2022-07-13 10:35:11 UTC]

Same here with QGIS:

Couldn't load plugin 'MetaSearch' due to an error when calling its classFactory() method 

ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/lib/python3.8/site-packages/markupsafe/__init__.py) 
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 423, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/usr/share/qgis/python/plugins/MetaSearch/__init__.py", line 29, in classFactory
    from MetaSearch.plugin import MetaSearchPlugin
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/share/qgis/python/plugins/MetaSearch/plugin.py", line 34, in 
    from MetaSearch.dialogs.maindialog import MetaSearchDialog
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/share/qgis/python/plugins/MetaSearch/dialogs/maindialog.py", line 61, in 
    from MetaSearch.dialogs.manageconnectionsdialog import ManageConnectionsDialog
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/share/qgis/python/plugins/MetaSearch/dialogs/manageconnectionsdialog.py", line 35, in 
    from MetaSearch.util import (get_connections_from_file, get_ui_class,
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/share/qgis/python/plugins/MetaSearch/util.py", line 38, in 
    from jinja2 import Environment, FileSystemLoader
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/lib/python3.8/site-packages/jinja2/__init__.py", line 12, in 
    from .environment import Environment
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/lib/python3.8/site-packages/jinja2/environment.py", line 25, in 
    from .defaults import BLOCK_END_STRING
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/lib/python3.8/site-packages/jinja2/defaults.py", line 3, in 
    from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
  File "/usr/lib/python3.8/site-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/lib/python3.8/site-packages/jinja2/filters.py", line 13, in 
    from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/lib/python3.8/site-packages/markupsafe/__init__.py)


Python version: 3.8.13 (default, Jul  2 2022, 22:51:55) [GCC 12.1.1 20220625] 
QGIS version: 3.22.8-Białowieża Białowieża, exported 

Python Path:
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/geometric_attributes
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/LandSurveyCodesImport
/usr/share/qgis/python
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/usr/share/qgis/python/plugins
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/usr/lib/python3.8/site-packages
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/kmltools/libs
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/LandSurveyCodesImport/gui
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/opeNoise/tools
/home/mmokrejs/.local/share/QGIS/QGIS3/profiles/default/python/plugins/shapetools/ext-libs


# emerge -pv markupsafe

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-python/markupsafe-2.1.1::gentoo  USE="-test" PYTHON_TARGETS="python3_8 python3_9 -pypy3 -python3_10 -python3_11" 0 KiB