Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.7.1: pytest is failing #231

Open
kloczek opened this issue Jun 12, 2021 · 22 comments · May be fixed by #375
Open

4.7.1: pytest is failing #231

kloczek opened this issue Jun 12, 2021 · 22 comments · May be fixed by #375

Comments

@kloczek
Copy link

kloczek commented Jun 12, 2021

Just normal build, install and test cycle used on building package from non-root:

  • "setup.py build":
  • "setupy install --root </install/prefix>"
  • "pytest with PYTHONPATH pointing to setearch and sitelib inside </install/prefix>
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.7.1-3.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.7.1-3.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=3294829005
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.7.1
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, pyfakefs-4.5.0, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, checkdocs-2.7.1, anyio-3.3.0, Faker-8.11.0, asyncio-0.15.1, trio-0.7.0, httpbin-1.0.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0
collected 54 items

jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 15%]
jupyter_core/tests/test_application.py ....                                                                                                                          [ 22%]
. F                                                                                                                                                                  [ 24%]
jupyter_core/tests/test_application.py ....                                                                                                                          [ 32%]
jupyter_core/tests/test_command.py ............                                                                                                                      [ 54%]
jupyter_core/tests/test_paths.py ...s..............F...s.                                                                                                            [100%]

================================================================================= FAILURES =================================================================================
_______________________________________________________________________________ test session _______________________________________________________________________________

cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x7ff977f45430>, when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

/usr/lib/python3.8/site-packages/_pytest/runner.py:311:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>       lambda: ihook(item=item, **kwds), when=when, reraise=reraise
    )

/usr/lib/python3.8/site-packages/_pytest/runner.py:255:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_HookCaller 'pytest_runtest_call'>, args = (), kwargs = {'item': <CheckdocsItem project>}, notincall = set()

    def __call__(self, *args, **kwargs):
        if args:
            raise TypeError("hook calling supports only keyword arguments")
        assert not self.is_historic()
        if self.spec and self.spec.argnames:
            notincall = (
                set(self.spec.argnames) - set(["__multicall__"]) - set(kwargs.keys())
            )
            if notincall:
                warnings.warn(
                    "Argument(s) {} which are declared in the hookspec "
                    "can not be found in this hook call".format(tuple(notincall)),
                    stacklevel=2,
                )
>       return self._hookexec(self, self.get_hookimpls(), kwargs)

/usr/lib/python3.8/site-packages/pluggy/hooks.py:286:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.config.PytestPluginManager object at 0x7ff9a8a3afa0>, hook = <_HookCaller 'pytest_runtest_call'>
methods = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...module 'hypothesis.extra.pytestplugin' from '/usr/lib/python3.8/site-packages/hypothesis/extra/pytestplugin.py'>>, ...]
kwargs = {'item': <CheckdocsItem project>}

    def _hookexec(self, hook, methods, kwargs):
        # called from all hookcaller instances.
        # enable_tracing will set its own wrapping function at self._inner_hookexec
>       return self._inner_hookexec(hook, methods, kwargs)

/usr/lib/python3.8/site-packages/pluggy/manager.py:93:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook = <_HookCaller 'pytest_runtest_call'>
methods = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...module 'hypothesis.extra.pytestplugin' from '/usr/lib/python3.8/site-packages/hypothesis/extra/pytestplugin.py'>>, ...]
kwargs = {'item': <CheckdocsItem project>}

>   self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
        methods,
        kwargs,
        firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
    )

/usr/lib/python3.8/site-packages/pluggy/manager.py:84:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...module 'hypothesis.extra.pytestplugin' from '/usr/lib/python3.8/site-packages/hypothesis/extra/pytestplugin.py'>>, ...]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(hook_impls, caller_kwargs, firstresult=False):
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from _HookCaller.__call__().
        """
        __tracebackhide__ = True
        results = []
        excinfo = None
        try:  # run impl and wrapper setup functions in a loop
            teardowns = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    "hook call must provide argument %r" % (argname,)
                                )

                    if hook_impl.hookwrapper:
                        try:
                            gen = hook_impl.function(*args)
                            next(gen)  # first yield
                            teardowns.append(gen)
                        except StopIteration:
                            _raise_wrapfail(gen, "did not yield")
                    else:
                        res = hook_impl.function(*args)
                        if res is not None:
                            results.append(res)
                            if firstresult:  # halt further impl calls
                                break
            except BaseException:
                excinfo = sys.exc_info()
        finally:
            if firstresult:  # first result hooks return a single value
                outcome = _Result(results[0] if results else None, excinfo)
            else:
                outcome = _Result(results, excinfo)

            # run all wrapper post-yield blocks
            for gen in reversed(teardowns):
                try:
                    gen.send(outcome)
                    _raise_wrapfail(gen, "has second yield")
                except StopIteration:
                    pass

>           return outcome.get_result()

/usr/lib/python3.8/site-packages/pluggy/callers.py:208:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pluggy.callers._Result object at 0x7ff977d7ae50>

    def get_result(self):
        """Get the result(s) for this hook call.

        If the hook was marked as a ``firstresult`` only a single value
        will be returned otherwise a list of results.
        """
        __tracebackhide__ = True
        if self._excinfo is None:
            return self._result
        else:
            ex = self._excinfo
            if _py3:
>               raise ex[1].with_traceback(ex[2])

/usr/lib/python3.8/site-packages/pluggy/callers.py:80:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

hook_impls = [<HookImpl plugin_name='runner', plugin=<module '_pytest.runner' from '/usr/lib/python3.8/site-packages/_pytest/runner...module 'hypothesis.extra.pytestplugin' from '/usr/lib/python3.8/site-packages/hypothesis/extra/pytestplugin.py'>>, ...]
caller_kwargs = {'item': <CheckdocsItem project>}, firstresult = False

    def _multicall(hook_impls, caller_kwargs, firstresult=False):
        """Execute a call into multiple python functions/methods and return the
        result(s).

        ``caller_kwargs`` comes from _HookCaller.__call__().
        """
        __tracebackhide__ = True
        results = []
        excinfo = None
        try:  # run impl and wrapper setup functions in a loop
            teardowns = []
            try:
                for hook_impl in reversed(hook_impls):
                    try:
                        args = [caller_kwargs[argname] for argname in hook_impl.argnames]
                    except KeyError:
                        for argname in hook_impl.argnames:
                            if argname not in caller_kwargs:
                                raise HookCallError(
                                    "hook call must provide argument %r" % (argname,)
                                )

                    if hook_impl.hookwrapper:
                        try:
                            gen = hook_impl.function(*args)
                            next(gen)  # first yield
                            teardowns.append(gen)
                        except StopIteration:
                            _raise_wrapfail(gen, "did not yield")
                    else:
>                       res = hook_impl.function(*args)

/usr/lib/python3.8/site-packages/pluggy/callers.py:187:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = <CheckdocsItem project>

    def pytest_runtest_call(item: Item) -> None:
        _update_current_test_var(item, "call")
        try:
            del sys.last_type
            del sys.last_value
            del sys.last_traceback
        except AttributeError:
            pass
        try:
            item.runtest()
        except Exception as e:
            # Store trace info to allow postmortem debugging
            sys.last_type = type(e)
            sys.last_value = e
            assert e.__traceback__ is not None
            # Skip *this* frame
            sys.last_traceback = e.__traceback__.tb_next
>           raise e

/usr/lib/python3.8/site-packages/_pytest/runner.py:170:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = <CheckdocsItem project>

    def pytest_runtest_call(item: Item) -> None:
        _update_current_test_var(item, "call")
        try:
            del sys.last_type
            del sys.last_value
            del sys.last_traceback
        except AttributeError:
            pass
        try:
>           item.runtest()

/usr/lib/python3.8/site-packages/_pytest/runner.py:162:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <CheckdocsItem project>

    def runtest(self):
        desc = self.get_long_description()
        method_name = f"run_{re.sub('[-/]', '_', desc.content_type)}"
>       getattr(self, method_name)(desc)
E       AttributeError: 'CheckdocsItem' object has no attribute 'run_text_plain'

/usr/lib/python3.8/site-packages/pytest_checkdocs/__init__.py:42: AttributeError
---------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------
ERROR    pep517.envbuild:wrappers.py:371 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ERROR    pep517.envbuild:wrappers.py:371 ipykernel 6.0.3 requires debugpy<2.0,>=1.0.0, which is not installed.
_______________________________________________________________________ test_jupyter_path_prefer_env _______________________________________________________________________

    def test_jupyter_path_prefer_env():
        with patch.dict('os.environ', {'JUPYTER_PREFER_ENV_PATH': 'true'}):
            path = jupyter_path()
>       assert path[0] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /home/tkloczko/.local/share/jupyter

jupyter_core/tests/test_paths.py:186: AssertionError
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:272: only runs on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:259: only run on windows
FAILED ::project - AttributeError: 'CheckdocsItem' object has no attribute 'run_text_plain'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
================================================================= 2 failed, 49 passed, 2 skipped in 9.02s ==================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c/test_safe_get_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_get_no_perms0'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c/test_safe_delete_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_delete_no_perms0'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c/test_safe_set_no_perms0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_safe_set_no_perms0'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c/test_rmtree_errorhandler_rerai0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_rerai0'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c/test_rmtree_errorhandler_reado0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_rmtree_errorhandler_reado0'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-32e0cf8b-1ca4-46ce-b2e7-426818c8a64c'
  warnings.warn(
@kloczek
Copy link
Author

kloczek commented Dec 1, 2021

Just tested 4.9.1 and pytest is still failing but new way:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.9.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.9.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env --deselect ::project
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.9.1
plugins: shutil-1.7.0, virtualenv-1.7.0, mock-3.6.1, cov-2.12.1, anyio-3.3.4, forked-1.3.0, xdist-2.3.0, flaky-3.7.0, tornasync-0.6.0.post2, console-scripts-1.2.0, trio-0.7.0, timeout-2.0.1, asyncio-0.16.0, freezegun-0.4.2, flake8-1.0.7, pyfakefs-4.5.3, hypothesis-6.27.2
collected 56 items / 1 deselected / 55 selected

jupyter_core/tests/test_application.py ........                                                                                                                      [ 14%]
jupyter_core/tests/test_command.py ............                                                                                                                      [ 36%]
jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 50%]
jupyter_core/tests/test_paths.py ..s...s...s..FF........sss.                                                                                                         [100%]

================================================================================= FAILURES =================================================================================
_______________________________________________________________________ test_jupyter_path_user_site ________________________________________________________________________

    def test_jupyter_path_user_site():
        with no_config_env, patch.object(site, 'ENABLE_USER_SITE', True):
            path = jupyter_path()

        # deduplicated expected values
        values = list(dict.fromkeys([
            jupyter_data_dir(),
            os.path.join(site.getuserbase(), 'share', 'jupyter'),
            paths.ENV_JUPYTER_PATH[0]
        ]))
        for p,v in zip(path, values):
>           assert p == v
E           AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /usr/local/share/jupyter
E             ?     ++++++

jupyter_core/tests/test_paths.py:209: AssertionError
______________________________________________________________________ test_jupyter_path_no_user_site ______________________________________________________________________

    def test_jupyter_path_no_user_site():
        with no_config_env, patch.object(site, 'ENABLE_USER_SITE', False):
            path = jupyter_path()
        assert path[0] == jupyter_data_dir()
>       assert path[1] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /usr/local/share/jupyter
E         ?     ++++++

jupyter_core/tests/test_paths.py:215: AssertionError
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:89: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:129: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:169: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:315: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:329: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:363: only runs on windows
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_user_site - AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_no_user_site - AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
========================================================== 2 failed, 47 passed, 6 skipped, 1 deselected in 2.11s ===========================================================

@kloczek
Copy link
Author

kloczek commented Jan 3, 2022

After few upgrades I have now other two units failing

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.9.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.9.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_no_user_site --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_user_site
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.9.1
collected 56 items / 3 deselected / 53 selected

jupyter_core/tests/test_application.py ........                                                                                                                      [ 15%]
jupyter_core/tests/test_command.py ..........FF                                                                                                                      [ 37%]
jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 52%]
jupyter_core/tests/test_paths.py ..s...s...s..........sss.                                                                                                           [100%]

================================================================================= FAILURES =================================================================================
_____________________________________________________________________________ test_not_on_path _____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-484/test_not_on_path0')

    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join('jupyter')
        jupyter.write(
            'from jupyter_core import command; command.main()'
        )
        jupyter.chmod(0o700)
        witness = a.join('jupyter-witness')
        witness_src = '#!%s\n%s\n' % (sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {'PATH': ''}
        if 'SYSTEMROOT' in os.environ:  # Windows http://bugs.python.org/issue20614
            env[str('SYSTEMROOT')] = os.environ['SYSTEMROOT']
        if sys.platform == 'win32':
            env[str('PATHEXT')] = '.EXE'
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), 'witness'], env=env)

jupyter_core/tests/test_command.py:177:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-484/test_not_on_path0/a/jupyter', 'witness'],), kwargs = {'env': {'PATH': ''}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fb4e8e9bd30>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-484/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-484/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
____________________________________________________________________________ test_path_priority ____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-484/test_path_priority0')

    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join('jupyter')
        jupyter.write(
            'from jupyter_core import command; command.main()'
        )
        jupyter.chmod(0o700)
        witness_a = a.join('jupyter-witness')
        witness_a_src = '#!%s\n%s\n' % (sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join('jupyter-witness')
        witness_b_src = '#!%s\n%s\n' % (sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {'PATH':  str(b)}
        if 'SYSTEMROOT' in os.environ:  # Windows http://bugs.python.org/issue20614
            env[str('SYSTEMROOT')] = os.environ['SYSTEMROOT']
        if sys.platform == 'win32':
            env[str('PATHEXT')] = '.EXE'
>       out = check_output([sys.executable, str(jupyter), 'witness'], env=env)

jupyter_core/tests/test_command.py:202:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-484/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-484/test_path_priority0/b'}, 'stdout': -1}, process = <subprocess.Popen object at 0x7fb4e8f21f40>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-484/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-484/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:89: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:129: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:169: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:315: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:329: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:363: only runs on windows
FAILED jupyter_core/tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-484/test_not_o...
FAILED jupyter_core/tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-484/test_pat...
========================================================== 2 failed, 45 passed, 6 skipped, 3 deselected in 1.48s ===========================================================

@kloczek
Copy link
Author

kloczek commented Feb 15, 2022

Trying to upgrade to just released 4.9.2 found that new usnit started failing

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.9.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.9.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect jupyter_core/tests/test_command.py::test_not_on_path --deselect jupyter_core/tests/test_command.py::test_path_priority --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_no_user_site --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env --deselect jupyter_core/tests/test_paths.py::test_jupyter_path_user_site
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.9.2
collected 57 items / 5 deselected / 52 selected

jupyter_core/tests/test_application.py ........                                                                                                                      [ 15%]
jupyter_core/tests/test_command.py ..........F                                                                                                                       [ 36%]
jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 51%]
jupyter_core/tests/test_paths.py ..s...s...s..........sss.                                                                                                           [100%]

================================================================================= FAILURES =================================================================================
________________________________________________________________________________ test_argv0 ________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-39/test_argv00')

    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join('jupyter')
        jupyter.write(
            'from jupyter_core import command; command.main()'
        )
        jupyter.chmod(0o700)
        witness_a = a.join('jupyter-witness')
        witness_a_src = f'''#!{sys.executable}
    import sys
    print(sys.argv[0])
    '''
        write_executable(witness_a, witness_a_src)

        env = {}
        if 'SYSTEMROOT' in os.environ:  # Windows http://bugs.python.org/issue20614
            env[str('SYSTEMROOT')] = os.environ['SYSTEMROOT']
        if sys.platform == 'win32':
            env[str('PATHEXT')] = '.EXE'
>       out = check_output([sys.executable, str(jupyter), 'witness'], env=env)

jupyter_core/tests/test_command.py:224:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-39/test_argv00/a/jupyter', 'witness'],), kwargs = {'env': {}, 'stdout': -1}
process = <subprocess.Popen object at 0x7f1337b52d00>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-39/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-39/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:89: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:129: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:169: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:315: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:329: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:363: only runs on windows
FAILED jupyter_core/tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-39/test_argv00/a/jup...
========================================================== 1 failed, 45 passed, 6 skipped, 5 deselected in 1.35s ===========================================================

@kloczek
Copy link
Author

kloczek commented Apr 18, 2022

Just tested 4.10.0 and looks like now is failing more units

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.10.0, configfile: pyproject.toml, testpaths: jupyter_core/tests/
collected 57 items

jupyter_core/tests/test_application.py ........                                                                                                                      [ 14%]
jupyter_core/tests/test_command.py ..........FFF                                                                                                                     [ 36%]
jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 50%]
jupyter_core/tests/test_paths.py ..s...s...s..FFF........sss.                                                                                                        [100%]

================================================================================= FAILURES =================================================================================
_____________________________________________________________________________ test_not_on_path _____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-31/test_not_on_path0')

    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness = a.join("jupyter-witness")
        witness_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {"PATH": ""}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:183:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_not_on_path0/a/jupyter', 'witness'],), kwargs = {'env': {'PATH': ''}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fb2226d4580>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-31/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
____________________________________________________________________________ test_path_priority ____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-31/test_path_priority0')

    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join("jupyter-witness")
        witness_b_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {"PATH": str(b)}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:206:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-31/test_path_priority0/b'}, 'stdout': -1}, process = <subprocess.Popen object at 0x7fb2222c0d00>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-31/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
________________________________________________________________________________ test_argv0 ________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-31/test_argv00')

    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = f"""#!{sys.executable}
    import sys
    print(sys.argv[0])
    """
        write_executable(witness_a, witness_a_src)

        env = {}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:227:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_argv00/a/jupyter', 'witness'],), kwargs = {'env': {}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fb222588af0>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-31/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
_______________________________________________________________________ test_jupyter_path_user_site ________________________________________________________________________

    def test_jupyter_path_user_site():
        with no_config_env, patch.object(site, "ENABLE_USER_SITE", True):
            path = jupyter_path()

        # deduplicated expected values
        values = list(
            dict.fromkeys(
                [
                    jupyter_data_dir(),
                    os.path.join(site.getuserbase(), "share", "jupyter"),
                    paths.ENV_JUPYTER_PATH[0],
                ]
            )
        )
        for p, v in zip(path, values):
>           assert p == v
E           AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /usr/local/share/jupyter
E             ?     ++++++

jupyter_core/tests/test_paths.py:233: AssertionError
______________________________________________________________________ test_jupyter_path_no_user_site ______________________________________________________________________

    def test_jupyter_path_no_user_site():
        with no_config_env, patch.object(site, "ENABLE_USER_SITE", False):
            path = jupyter_path()
        assert path[0] == jupyter_data_dir()
>       assert path[1] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /usr/local/share/jupyter
E         ?     ++++++

jupyter_core/tests/test_paths.py:240: AssertionError
_______________________________________________________________________ test_jupyter_path_prefer_env _______________________________________________________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>       assert path[0] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /home/tkloczko/.local/share/jupyter

jupyter_core/tests/test_paths.py:246: AssertionError
=========================================================================== slowest 10 durations ===========================================================================
0.20s call     jupyter_core/tests/test_command.py::test_paths_debug
0.11s call     jupyter_core/tests/test_command.py::test_config_dir
0.10s call     jupyter_core/tests/test_command.py::test_runtime_dir
0.10s call     jupyter_core/tests/test_command.py::test_subcommand_not_given
0.10s call     jupyter_core/tests/test_command.py::test_help
0.10s call     jupyter_core/tests/test_command.py::test_subcommand_not_found
0.09s call     jupyter_core/tests/test_command.py::test_paths
0.09s call     jupyter_core/tests/test_command.py::test_paths_json
0.09s call     jupyter_core/tests/test_command.py::test_data_dir
0.05s call     jupyter_core/tests/test_command.py::test_not_on_path
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:106: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:147: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:188: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:358: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:377: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:415: only runs on windows
FAILED jupyter_core/tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_not_on...
FAILED jupyter_core/tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_path...
FAILED jupyter_core/tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-31/test_argv00/a/jup...
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_user_site - AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_no_user_site - AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
================================================================= 6 failed, 45 passed, 6 skipped in 1.93s ==================================================================

@kloczek
Copy link
Author

kloczek commented Apr 18, 2022

I've been tryng to add JUPYTER_CONFIG_PATH env variable pointing to </install/prefix>/etc/jupiter with created that directory before pytest execution but it mage pytest results even vorse.
IMO it should be possible to test wit that env variable

+ export JUPYTER_CONFIG_PATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/etc/jupyter
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.10.0, configfile: pyproject.toml, testpaths: jupyter_core/tests/
collected 57 items

jupyter_core/tests/test_application.py ........                                                                                                                      [ 14%]
jupyter_core/tests/test_command.py .....F....FFF                                                                                                                     [ 36%]
jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 50%]
jupyter_core/tests/test_paths.py ..s...s...s....F...FFF..sss.                                                                                                        [100%]

================================================================================= FAILURES =================================================================================
_____________________________________________________________________________ test_paths_debug _____________________________________________________________________________

    def test_paths_debug():
        vars = [
            "JUPYTER_PREFER_ENV_PATH",
            "JUPYTER_NO_CONFIG",
            "JUPYTER_CONFIG_PATH",
            "JUPYTER_CONFIG_DIR",
            "JUPYTER_PATH",
            "JUPYTER_DATA_DIR",
            "JUPYTER_RUNTIME_DIR",
        ]
        output = get_jupyter_output(["--paths", "--debug"])
        for v in vars:
>           assert f"{v} is not set" in output
E           AssertionError: assert 'JUPYTER_CONFIG_PATH is not set' in 'JUPYTER_PREFER_ENV_PATH is not set, making the user-level path preferred over the environment-level path for data and.../home/tkloczko/.local/share/jupyter\n    /usr/share/jupyter\nruntime:\n    /home/tkloczko/.local/share/jupyter/runtime'

jupyter_core/tests/test_command.py:113: AssertionError
_____________________________________________________________________________ test_not_on_path _____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-35/test_not_on_path0')

    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness = a.join("jupyter-witness")
        witness_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {"PATH": ""}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:183:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_not_on_path0/a/jupyter', 'witness'],), kwargs = {'env': {'PATH': ''}, 'stdout': -1}
process = <subprocess.Popen object at 0x7f12348f1970>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-35/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
____________________________________________________________________________ test_path_priority ____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-35/test_path_priority0')

    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join("jupyter-witness")
        witness_b_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {"PATH": str(b)}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:206:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-35/test_path_priority0/b'}, 'stdout': -1}, process = <subprocess.Popen object at 0x7f1234532400>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-35/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
________________________________________________________________________________ test_argv0 ________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-35/test_argv00')

    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = f"""#!{sys.executable}
    import sys
    print(sys.argv[0])
    """
        write_executable(witness_a, witness_a_src)

        env = {}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:227:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_argv00/a/jupyter', 'witness'],), kwargs = {'env': {}, 'stdout': -1}
process = <subprocess.Popen object at 0x7f12348cbb20>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-35/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
_______________________________________________________________________ test_jupyter_path_prefer_env _______________________________________________________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>       assert path[0] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /home/tkloczko/.local/share/jupyter

jupyter_core/tests/test_paths.py:246: AssertionError
_________________________________________________________________________ test_jupyter_config_path _________________________________________________________________________

    def test_jupyter_config_path():
        with patch.object(site, "ENABLE_USER_SITE", True):
            path = jupyter_config_path()

        # deduplicated expected values
        values = list(
            dict.fromkeys(
                [
                    jupyter_config_dir(),
                    os.path.join(site.getuserbase(), "etc", "jupyter"),
                    paths.ENV_CONFIG_PATH[0],
                ]
            )
        )
        for p, v in zip(path, values):
>           assert p == v
E           AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
E             - /home/tkloczko/.jupyter
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/etc/jupyter

jupyter_core/tests/test_paths.py:290: AssertionError
__________________________________________________________________ test_jupyter_config_path_no_user_site ___________________________________________________________________

    def test_jupyter_config_path_no_user_site():
        with patch.object(site, "ENABLE_USER_SITE", False):
            path = jupyter_config_path()
>       assert path[0] == jupyter_config_dir()
E       AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
E         - /home/tkloczko/.jupyter
E         + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/etc/jupyter

jupyter_core/tests/test_paths.py:296: AssertionError
___________________________________________________________________ test_jupyter_config_path_prefer_env ____________________________________________________________________

    def test_jupyter_config_path_prefer_env():
        with prefer_env, patch.object(site, "ENABLE_USER_SITE", True):
            path = jupyter_config_path()

        # deduplicated expected values
        values = list(
            dict.fromkeys(
                [
                    paths.ENV_CONFIG_PATH[0],
                    jupyter_config_dir(),
                    os.path.join(site.getuserbase(), "etc", "jupyter"),
                ]
            )
        )
        for p, v in zip(path, values):
>           assert p == v
E           AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/usr/etc/jupyter'
E             - /usr/etc/jupyter
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/etc/jupyter

jupyter_core/tests/test_paths.py:315: AssertionError
=========================================================================== slowest 10 durations ===========================================================================
0.11s call     jupyter_core/tests/test_command.py::test_paths
0.11s call     jupyter_core/tests/test_command.py::test_runtime_dir
0.10s call     jupyter_core/tests/test_command.py::test_data_dir
0.10s call     jupyter_core/tests/test_command.py::test_subcommand_not_found
0.10s call     jupyter_core/tests/test_command.py::test_subcommand_not_given
0.10s call     jupyter_core/tests/test_command.py::test_help
0.09s call     jupyter_core/tests/test_command.py::test_paths_debug
0.09s call     jupyter_core/tests/test_command.py::test_config_dir
0.08s call     jupyter_core/tests/test_command.py::test_paths_json
0.05s call     jupyter_core/tests/test_command.py::test_not_on_path
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:106: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:147: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:188: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:358: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:377: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:415: only runs on windows
FAILED jupyter_core/tests/test_command.py::test_paths_debug - AssertionError: assert 'JUPYTER_CONFIG_PATH is not set' in 'JUPYTER_PREFER_ENV_PATH is not set, making the ...
FAILED jupyter_core/tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_not_on...
FAILED jupyter_core/tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_path...
FAILED jupyter_core/tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-35/test_argv00/a/jup...
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_config_path - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_config_path_no_user_site - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_config_path_prefer_env - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/usr/etc/jupyter'
================================================================= 8 failed, 43 passed, 6 skipped in 1.90s ==================================================================

@kloczek
Copy link
Author

kloczek commented Apr 18, 2022

First unit fails shows whole set of env variables which could be used

_____________________________________________________________________________ test_paths_debug _____________________________________________________________________________

    def test_paths_debug():
        vars = [
            "JUPYTER_PREFER_ENV_PATH",
            "JUPYTER_NO_CONFIG",
            "JUPYTER_CONFIG_PATH",
            "JUPYTER_CONFIG_DIR",
            "JUPYTER_PATH",
            "JUPYTER_DATA_DIR",
            "JUPYTER_RUNTIME_DIR",
        ]
        output = get_jupyter_output(["--paths", "--debug"])
        for v in vars:
>           assert f"{v} is not set" in output
E           AssertionError: assert 'JUPYTER_CONFIG_DIR is not set' in 'JUPYTER_PREFER_ENV_PATH is not set, making the user-level path preferred over the environment-level path for data and...er\nruntime:\n    /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/share/jupyter/runtime'

jupyter_core/tests/test_command.py:113: AssertionError

Q: what kind of combination of those env variables should I use for mu build and testing methodology? 🤔

@blink1073
Copy link
Contributor

We're not setting any env variables in CI, we're just running pytest with some coverage-related flags. It seems like the paths themselves are getting thrown off by the PYTHONPATH modifications.

@kloczek
Copy link
Author

kloczek commented Apr 20, 2022

Could you please retest using methodology which I've described on top of this ticket?
That methodology is widely used on packaging software.
Handful of python modules still have some issues with "testing as installed". One of that modules (still) is for example setuptools pypa/setuptools#2318

@blink1073
Copy link
Contributor

Invoking setup.py directly is deprecated by setuptools themselves. I don't think we want to support that going forward.

@kloczek
Copy link
Author

kloczek commented Apr 20, 2022

Invoking setup.py directly is deprecated by setuptools themselves. I don't think we want to support that going forward.

That setuptools ticket is not about setup.py.
Taht ticket is already opened ~9 months. In mean time I've moved in all my rpm packages to pep517 build procedure which consist from:

  • python3 -sBm build -w --no-isolation
  • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
  • install .whl file in </install/prefix>
  • run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

This issue has nothing to do with use setup.py build vs. pep517.

Here is updated pytest output against latest 4.10.0:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-4.10.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-4.10.0, configfile: pyproject.toml, testpaths: jupyter_core/tests/
collected 57 items

jupyter_core/tests/test_application.py ........                                                                                                                      [ 14%]
jupyter_core/tests/test_command.py ..........FFF                                                                                                                     [ 36%]
jupyter_core/tests/test_migrate.py ........                                                                                                                          [ 50%]
jupyter_core/tests/test_paths.py ..s...s...s....F........sss.                                                                                                        [100%]

================================================================================= FAILURES =================================================================================
_____________________________________________________________________________ test_not_on_path _____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-84/test_not_on_path0')

    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness = a.join("jupyter-witness")
        witness_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {"PATH": ""}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:183:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_not_on_path0/a/jupyter', 'witness'],), kwargs = {'env': {'PATH': ''}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fe273134eb0>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-84/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
____________________________________________________________________________ test_path_priority ____________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-84/test_path_priority0')

    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join("jupyter-witness")
        witness_b_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {"PATH": str(b)}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:206:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-84/test_path_priority0/b'}, 'stdout': -1}, process = <subprocess.Popen object at 0x7fe272e55a60>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-84/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
________________________________________________________________________________ test_argv0 ________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-84/test_argv00')

    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = f"""#!{sys.executable}
    import sys
    print(sys.argv[0])
    """
        write_executable(witness_a, witness_a_src)

        env = {}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:227:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_argv00/a/jupyter', 'witness'],), kwargs = {'env': {}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fe27320c9d0>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-84/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
_______________________________________________________________________ test_jupyter_path_prefer_env _______________________________________________________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>       assert path[0] == paths.ENV_JUPYTER_PATH[0]
E       AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E         - /usr/share/jupyter
E         + /home/tkloczko/.local/share/jupyter

jupyter_core/tests/test_paths.py:246: AssertionError
=========================================================================== slowest 10 durations ===========================================================================
0.19s call     jupyter_core/tests/test_command.py::test_paths_debug
0.12s call     jupyter_core/tests/test_command.py::test_subcommand_not_found
0.11s call     jupyter_core/tests/test_command.py::test_paths
0.11s call     jupyter_core/tests/test_command.py::test_paths_json
0.11s call     jupyter_core/tests/test_command.py::test_config_dir
0.11s call     jupyter_core/tests/test_command.py::test_data_dir
0.11s call     jupyter_core/tests/test_command.py::test_runtime_dir
0.10s call     jupyter_core/tests/test_command.py::test_help
0.10s call     jupyter_core/tests/test_command.py::test_subcommand_not_given
0.05s call     jupyter_core/tests/test_command.py::test_not_on_path
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:106: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:147: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:188: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:358: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:377: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:415: only runs on windows
FAILED jupyter_core/tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_not_on...
FAILED jupyter_core/tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_path...
FAILED jupyter_core/tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-84/test_argv00/a/jup...
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
================================================================= 4 failed, 47 passed, 6 skipped in 1.98s ==================================================================

@kloczek
Copy link
Author

kloczek commented Jan 2, 2023

Gentle ping ..
Just tested 5.1.2

+ JUPYTER_CONFIG_DIR=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/etc/jupyter
+ JUPYTER_DATA_DIR=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.16, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-5.1.2, configfile: pyproject.toml, testpaths: jupyter_core/tests/
collected 75 items

jupyter_core/tests/test_application.py .........                                                                                                                                      [ 12%]
jupyter_core/tests/test_async.py ..                                                                                                                                                   [ 14%]
jupyter_core/tests/test_command.py .....F....FFF                                                                                                                                      [ 32%]
jupyter_core/tests/test_migrate.py ........                                                                                                                                           [ 42%]
jupyter_core/tests/test_paths.py .FssF....ssssFF..ssssFF...F.........ssss...                                                                                                          [100%]

========================================================================================= FAILURES ==========================================================================================
_____________________________________________________________________________________ test_paths_debug ______________________________________________________________________________________

    def test_paths_debug():
        vars = [
            "JUPYTER_PREFER_ENV_PATH",
            "JUPYTER_NO_CONFIG",
            "JUPYTER_CONFIG_PATH",
            "JUPYTER_CONFIG_DIR",
            "JUPYTER_PATH",
            "JUPYTER_DATA_DIR",
            "JUPYTER_RUNTIME_DIR",
        ]
        output = get_jupyter_output(["--paths", "--debug"])
        for v in vars:
>           assert f"{v} is not set" in output
E           AssertionError: assert 'JUPYTER_CONFIG_DIR is not set' in 'JUPYTER_PLATFORM_DIRS is set to a false value, or is not set, so we use hardcoded legacy paths for platform-specific ...ter\nruntime:\n    /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter/runtime'

jupyter_core/tests/test_command.py:112: AssertionError
_____________________________________________________________________________________ test_not_on_path ______________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-24/test_not_on_path0')

    @skip_darwin
    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness = a.join("jupyter-witness")
        witness_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {"PATH": ""}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:186:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_not_on_path0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': ''}, 'stdout': -1}, process = <subprocess.Popen object at 0x7ff07d28ba90>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-24/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
____________________________________________________________________________________ test_path_priority _____________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-24/test_path_priority0')

    @skip_darwin
    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join("jupyter-witness")
        witness_b_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {"PATH": str(b)}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:210:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-24/test_path_priority0/b'}, 'stdout': -1}, process = <subprocess.Popen object at 0x7ff07c7584f0>, stdout = b'', stderr = None
retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-24/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
________________________________________________________________________________________ test_argv0 _________________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-24/test_argv00')

    @skip_darwin
    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = f"""#!{sys.executable}
    import sys
    print(sys.argv[0])
    """
        write_executable(witness_a, witness_a_src)

        env = {}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

jupyter_core/tests/test_command.py:232:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_argv00/a/jupyter', 'witness'],)
kwargs = {'env': {}, 'stdout': -1}, process = <subprocess.Popen object at 0x7ff07d02b760>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-24/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
______________________________________________________________________________________ test_config_dir ______________________________________________________________________________________

    def test_config_dir():
        config = jupyter_config_dir()
>       assert config == home_jupyter
E       AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
E         - /home/tkloczko/.jupyter
E         + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/etc/jupyter

jupyter_core/tests/test_paths.py:108: AssertionError
___________________________________________________________________________________ test_config_dir_linux ___________________________________________________________________________________

    @linux
    @use_platformdirs
    def test_config_dir_linux():
        config = jupyter_config_dir()
>       assert config == realpath("~/.config/jupyter")
E       AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tklocz...onfig/jupyter'
E         - /home/tkloczko/.config/jupyter
E         + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/etc/jupyter

jupyter_core/tests/test_paths.py:129: AssertionError
________________________________________________________________________________ test_data_dir_linux_legacy _________________________________________________________________________________

    @linux
    def test_data_dir_linux_legacy():
        with no_xdg:
            data = jupyter_data_dir()
>           assert data == realpath("~/.local/share/jupyter")
E           AssertionError: assert '/home/tklocz...share/jupyter' == '/home/tklocz...share/jupyter'
E             - /home/tkloczko/.local/share/jupyter
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter

jupyter_core/tests/test_paths.py:190: AssertionError
____________________________________________________________________________________ test_data_dir_linux ____________________________________________________________________________________

    @linux
    @use_platformdirs
    def test_data_dir_linux():
        with no_xdg:
            data = jupyter_data_dir()
>           assert data == realpath("~/.local/share/jupyter")
E           AssertionError: assert '/home/tklocz...share/jupyter' == '/home/tklocz...share/jupyter'
E             - /home/tkloczko/.local/share/jupyter
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter

jupyter_core/tests/test_paths.py:202: AssertionError
_______________________________________________________________________________ test_runtime_dir_linux_legacy _______________________________________________________________________________

    @linux
    def test_runtime_dir_linux_legacy():
        with no_xdg:
            runtime = jupyter_runtime_dir()
>           assert runtime == realpath("~/.local/share/jupyter/runtime")
E           AssertionError: assert '/home/tklocz...pyter/runtime' == '/home/tklocz...pyter/runtime'
E             - /home/tkloczko/.local/share/jupyter/runtime
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter/runtime

jupyter_core/tests/test_paths.py:254: AssertionError
__________________________________________________________________________________ test_runtime_dir_linux ___________________________________________________________________________________

    @linux
    @use_platformdirs
    def test_runtime_dir_linux():
        with no_xdg:
            runtime = jupyter_runtime_dir()
>           assert runtime == realpath("~/.local/share/jupyter/runtime")
E           AssertionError: assert '/home/tklocz...pyter/runtime' == '/home/tklocz...pyter/runtime'
E             - /home/tkloczko/.local/share/jupyter/runtime
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter/runtime

jupyter_core/tests/test_paths.py:266: AssertionError
_______________________________________________________________________________ test_jupyter_path_prefer_env ________________________________________________________________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>           assert path[0] == paths.ENV_JUPYTER_PATH[0]
E           AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.1.2-2.fc35.x86_64/usr/share/jupyter

jupyter_core/tests/test_paths.py:309: AssertionError
===================================================================================== warnings summary ======================================================================================
jupyter_core/application.py:21
  /home/tkloczko/rpmbuild/BUILD/jupyter_core-5.1.2/jupyter_core/application.py:21: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from .paths import (

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================================================================== slowest 10 durations ====================================================================================
0.12s call     jupyter_core/tests/test_command.py::test_help
0.11s call     jupyter_core/tests/test_command.py::test_subcommand_not_found
0.11s call     jupyter_core/tests/test_command.py::test_runtime_dir
0.11s call     jupyter_core/tests/test_command.py::test_data_dir
0.11s call     jupyter_core/tests/test_command.py::test_config_dir
0.11s call     jupyter_core/tests/test_command.py::test_paths
0.11s call     jupyter_core/tests/test_command.py::test_paths_debug
0.11s call     jupyter_core/tests/test_command.py::test_subcommand_not_given
0.10s call     jupyter_core/tests/test_command.py::test_paths_json
0.04s call     jupyter_core/tests/test_command.py::test_not_on_path
================================================================================== short test summary info ==================================================================================
SKIPPED [1] jupyter_core/tests/test_paths.py:111: only run on macos
SKIPPED [1] jupyter_core/tests/test_paths.py:118: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:160: only run on macos
SKIPPED [1] jupyter_core/tests/test_paths.py:166: only run on macos
SKIPPED [1] jupyter_core/tests/test_paths.py:173: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:179: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:224: only run on macos
SKIPPED [1] jupyter_core/tests/test_paths.py:230: only run on macos
SKIPPED [1] jupyter_core/tests/test_paths.py:237: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:243: only run on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:459: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:478: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] jupyter_core/tests/test_paths.py:516: only runs on windows
SKIPPED [1] jupyter_core/tests/test_paths.py:521: only runs on windows
FAILED jupyter_core/tests/test_command.py::test_paths_debug - AssertionError: assert 'JUPYTER_CONFIG_DIR is not set' in 'JUPYTER_PLATFORM_DIRS is set to a false value, or is not set, so we use hardcoded legacy paths for platform-specific ...ter\n...
FAILED jupyter_core/tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.
FAILED jupyter_core/tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.
FAILED jupyter_core/tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-24/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.
FAILED jupyter_core/tests/test_paths.py::test_config_dir - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
FAILED jupyter_core/tests/test_paths.py::test_config_dir_linux - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tklocz...onfig/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_data_dir_linux_legacy - AssertionError: assert '/home/tklocz...share/jupyter' == '/home/tklocz...share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_data_dir_linux - AssertionError: assert '/home/tklocz...share/jupyter' == '/home/tklocz...share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_runtime_dir_linux_legacy - AssertionError: assert '/home/tklocz...pyter/runtime' == '/home/tklocz...pyter/runtime'
FAILED jupyter_core/tests/test_paths.py::test_runtime_dir_linux - AssertionError: assert '/home/tklocz...pyter/runtime' == '/home/tklocz...pyter/runtime'
FAILED jupyter_core/tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
=================================================================== 11 failed, 50 passed, 14 skipped, 1 warning in 1.84s ====================================================================

@kloczek
Copy link
Author

kloczek commented Jan 2, 2023

Just FTR there are new units failing

FAILED jupyter_core/tests/test_command.py::test_paths_debug - AssertionError: assert 'JUPYTER_CONFIG_DIR is not set' in 'JUPYTER_PLATFORM_DIRS is set to a false value, or is not set, so we use hardcoded legacy paths for platform-specific ...ter\n...
FAILED jupyter_core/tests/test_paths.py::test_config_dir - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tkloczko/.jupyter'
FAILED jupyter_core/tests/test_paths.py::test_config_dir_linux - AssertionError: assert '/home/tklocz...4/etc/jupyter' == '/home/tklocz...onfig/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_data_dir_linux_legacy - AssertionError: assert '/home/tklocz...share/jupyter' == '/home/tklocz...share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_data_dir_linux - AssertionError: assert '/home/tklocz...share/jupyter' == '/home/tklocz...share/jupyter'
FAILED jupyter_core/tests/test_paths.py::test_runtime_dir_linux_legacy - AssertionError: assert '/home/tklocz...pyter/runtime' == '/home/tklocz...pyter/runtime'
FAILED jupyter_core/tests/test_paths.py::test_runtime_dir_linux - AssertionError: assert '/home/tklocz...pyter/runtime' == '/home/tklocz...pyter/runtime'

However this ould be as well resoult of the fact that now I'm performing my build on the system which is cut off from public network.

@kloczek
Copy link
Author

kloczek commented Jan 2, 2023

If those new units are failing because lack of accss to public network they should be marked by @pytest.mark.network o skip those units automatically on passing `-m "not network"' like it is in many other module test suites.

@jasongrout
Copy link
Member

However this ould be as well resoult of the fact that now I'm performing my build on the system which is cut off from public network.

That should have no effect on these tests. It looks to me like at least some of the tests are failing because JUPYTER_CONFIG_DIR and JUPYTER_DATA_DIR are set when running the tests, which is throwing off the expected default values.

jasongrout added a commit to jasongrout/jupyter_core that referenced this issue Jan 7, 2023
jasongrout added a commit to jasongrout/jupyter_core that referenced this issue Jan 7, 2023
jasongrout added a commit to jasongrout/jupyter_core that referenced this issue Jan 7, 2023
jasongrout added a commit to jasongrout/jupyter_core that referenced this issue Jan 7, 2023
jasongrout added a commit to jasongrout/jupyter_core that referenced this issue Jan 7, 2023
@mtelka
Copy link

mtelka commented Sep 28, 2023

I just tested version 5.3.2 on OpenIndiana and I see these three failures (all other tests either pass or skip):

=================================== FAILURES ===================================
_________________________ test_jupyter_path_user_site __________________________

    def test_jupyter_path_user_site():
        with patch.object(site, "ENABLE_USER_SITE", True):
            path = jupyter_path()
    
            # deduplicated expected values
            values = list(
                dict.fromkeys(
                    [
                        jupyter_data_dir(),
                        os.path.join(site.getuserbase(), "share", "jupyter"),
                        paths.ENV_JUPYTER_PATH[0],
                    ]
                )
            )  
            for p, v in zip(path, values):
>               assert p == v
E               AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
E                 - /usr/share/jupyter
E                 + /usr/local/share/jupyter
E                 ?     ++++++

p          = '/usr/local/share/jupyter'
path       = ['/home/marcel/.local/share/jupyter', '/usr/local/share/jupyter', '/usr/share/jupyter']
v          = '/usr/share/jupyter'
values     = ['/home/marcel/.local/share/jupyter', '/usr/share/jupyter']

tests/test_paths.py:300: AssertionError
________________________ test_jupyter_path_no_user_site ________________________

    def test_jupyter_path_no_user_site():
        with patch.object(site, "ENABLE_USER_SITE", False):
            path = jupyter_path()
            assert path[0] == jupyter_data_dir()
>           assert path[1] == paths.ENV_JUPYTER_PATH[0]
E           AssertionError: assert '/usr/local/share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /usr/local/share/jupyter
E             ?     ++++++

path       = ['/home/marcel/.local/share/jupyter', '/usr/local/share/jupyter', '/usr/share/jupyter']

tests/test_paths.py:307: AssertionError
_________________________ test_jupyter_path_prefer_env _________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>           assert path[0] == paths.ENV_JUPYTER_PATH[0]
E           AssertionError: assert '/home/marcel...share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /home/marcel/.local/share/jupyter

path       = ['/home/marcel/.local/share/jupyter', '/usr/local/share/jupyter', '/usr/share/jupyter']

tests/test_paths.py:313: AssertionError

@mtelka
Copy link

mtelka commented Oct 12, 2023

Testing with 5.4.0 shows the same test failures as with 5.3.2.

@ivanov ivanov linked a pull request Nov 17, 2023 that will close this issue
@ivanov ivanov linked a pull request Nov 17, 2023 that will close this issue
@ivanov
Copy link
Member

ivanov commented Nov 17, 2023

Please try #375 and report back there. Thanks!

@kloczek
Copy link
Author

kloczek commented Nov 17, 2023

It is a bit better however still some units are failing

+ cd jupyter_core-5.5.0
+ export JUPYTER_CONFIG_DIR=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/etc/jupyter JUPYTER_DATA_DIR=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/share/jupyter
+ JUPYTER_CONFIG_DIR=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/etc/jupyter
+ JUPYTER_DATA_DIR=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/share/jupyter
+ ASMFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none'
+ CFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none'
+ CXXFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none'
+ FFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -I/usr/lib64/gfortran/modules'
+ FCFLAGS='-O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -I/usr/lib64/gfortran/modules'
+ LDFLAGS='-Wl,--gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1'
+ RUSTFLAGS='-C codegen-units=1 -C debuginfo=2 -C opt-level=2 -C link-arg=-fdata-sections -C link-arg=-ffunction-sections -C link-arg=-Wl,--as-needed -C link-arg=-Wl,-z,now -C link-arg=-Wl,-z,relro --cap-lints=warn'
+ VALAFLAGS=-g
+ CC=/usr/bin/gcc
+ CXX=/usr/bin/g++
+ FC=/usr/bin/gfortran
+ AR=/usr/bin/gcc-ar
+ NM=/usr/bin/gcc-nm
+ RANLIB=/usr/bin/gcc-ranlib
+ export ASMFLAGS CFLAGS CXXFLAGS FFLAGS FCFLAGS LDFLAGS VALAFLAGS CC CXX FC AR NM RANLIB RUSTFLAGS VALAFLAGS
+ PATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/bin:/usr/bin:/usr/local/sbin:/usr/sbin
+ LD_LIBRARY_PATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/lib64
+ PYTHONDONTWRITEBYTECODE=1
+ PDM_BUILD_SCM_VERSION=5.5.0
+ PBR_VERSION=5.5.0
+ SETUPTOOLS_SCM_PRETEND_VERSION=5.5.0
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-5.5.0
configfile: pyproject.toml
testpaths: tests/
collected 81 items

tests/test_application.py .........                                      [ 11%]
tests/test_command.py ..........FFF                                      [ 27%]
tests/test_migrate.py ...........                                        [ 40%]
tests/test_paths.py ..ss.....ssss....ssss.....F.........ssss...          [ 93%]
tests/test_troubleshoot.py F                                             [ 95%]
tests/test_utils.py ....                                                 [100%]

=================================== FAILURES ===================================
_______________________________ test_not_on_path _______________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0')

    @skip_darwin
    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness = a.join("jupyter-witness")
        witness_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {"PATH": ""}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

a          = local('/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a')
env        = {'PATH': ''}
jupyter    = local('/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter')
tmpdir     = local('/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0')
witness    = local('/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter-witness')
witness_src = '#!/usr/bin/python3\nprint("WITNESS ME")\n'

tests/test_command.py:200:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'PATH': ''}}
        popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter', 'witness'],)
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': ''}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fc0697af8e0>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

capture_output = False
check      = True
input      = None
kwargs     = {'env': {'PATH': ''}, 'stdout': -1}
popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter', 'witness'],)
process    = <subprocess.Popen object at 0x7fc0697af8e0>
retcode    = 1
stderr     = None
stdout     = b''
timeout    = None

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-23/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
______________________________ test_path_priority ______________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0')

    @skip_darwin
    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join("jupyter-witness")
        witness_b_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {"PATH": str(b)}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

a          = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a')
b          = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/b')
env        = {'PATH': '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/b'}
jupyter    = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter')
tmpdir     = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0')
witness_a  = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter-witness')
witness_a_src = '#!/usr/bin/python3\nprint("WITNESS A")\n'
witness_b  = local('/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/b/jupyter-witness')
witness_b_src = '#!/usr/bin/python3\nprint("WITNESS B")\n'

tests/test_command.py:224:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/b'}}
        popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter', 'witness'],)
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/b'}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fc0693c5160>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

capture_output = False
check      = True
input      = None
kwargs     = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/b'}, 'stdout': -1}
popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter', 'witness'],)
process    = <subprocess.Popen object at 0x7fc0693c5160>
retcode    = 1
stderr     = None
stdout     = b''
timeout    = None

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-23/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
__________________________________ test_argv0 __________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-23/test_argv00')

    @skip_darwin
    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = f"""#!{sys.executable}
    import sys
    print(sys.argv[0])
    """
        write_executable(witness_a, witness_a_src)

        env = {}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

a          = local('/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a')
env        = {}
jupyter    = local('/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter')
tmpdir     = local('/tmp/pytest-of-tkloczko/pytest-23/test_argv00')
witness_a  = local('/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter-witness')
witness_a_src = '#!/usr/bin/python3\nimport sys\nprint(sys.argv[0])\n'

tests/test_command.py:246:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {}}
        popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter', 'witness'],)
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True
popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter', 'witness'],)
kwargs = {'env': {}, 'stdout': -1}
process = <subprocess.Popen object at 0x7fc0695bbfd0>, stdout = b''
stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

capture_output = False
check      = True
input      = None
kwargs     = {'env': {}, 'stdout': -1}
popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter', 'witness'],)
process    = <subprocess.Popen object at 0x7fc0695bbfd0>
retcode    = 1
stderr     = None
stdout     = b''
timeout    = None

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-23/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
_________________________ test_jupyter_path_prefer_env _________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>           assert path[0] == paths.ENV_JUPYTER_PATH[0]
E           AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /home/tkloczko/.local/share/jupyter

path       = ['/home/tkloczko/.local/share/jupyter', '/usr/share/jupyter']

tests/test_paths.py:313: AssertionError
______________________________ test_troubleshoot _______________________________

capsys = <_pytest.capture.CaptureFixture object at 0x7fc0787096a0>

    def test_troubleshoot(capsys):
        """Smoke test the troubleshoot function"""
        main()
        out = capsys.readouterr().out
>       assert "pip list" in out
E       AssertionError: assert 'pip list' in '$PATH:\n\t/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/bin\n\t/usr/bin\n\t/usr/local...13.2.1-3)]\n\nplatform.platform():\n\tLinux-6.6.0-0.rc5.20231013git10a6e5feccb8.44.fc40.x86_64-x86_64-with-glibc2.34\n'

capsys     = <_pytest.capture.CaptureFixture object at 0x7fc0787096a0>
out        = '$PATH:\n\t/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.0-2.fc35.x86_64/usr/bin\n\t/usr/bin\n\t/usr/local...13.2.1-3)]\n\nplatform.platform():\n\tLinux-6.6.0-0.rc5.20231013git10a6e5feccb8.44.fc40.x86_64-x86_64-with-glibc2.34\n'

tests/test_troubleshoot.py:8: AssertionError
----------------------------- Captured stderr call -----------------------------
/usr/bin/python3: No module named pip
=============================== warnings summary ===============================
jupyter_core/application.py:22
  /home/tkloczko/rpmbuild/BUILD/jupyter_core-5.5.0/jupyter_core/application.py:22: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from .paths import (

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================= slowest 10 durations =============================
0.19s call     tests/test_command.py::test_paths_debug
0.12s call     tests/test_command.py::test_data_dir
0.11s call     tests/test_command.py::test_config_dir
0.11s call     tests/test_command.py::test_subcommand_not_found
0.10s call     tests/test_command.py::test_subcommand_not_given
0.10s call     tests/test_command.py::test_runtime_dir
0.10s call     tests/test_command.py::test_help
0.09s call     tests/test_command.py::test_paths_json
0.09s call     tests/test_command.py::test_paths
0.05s call     tests/test_troubleshoot.py::test_troubleshoot
=========================== short test summary info ============================
SKIPPED [1] tests/test_paths.py:111: only run on macos
SKIPPED [1] tests/test_paths.py:119: only run on windows
SKIPPED [1] tests/test_paths.py:161: only run on macos
SKIPPED [1] tests/test_paths.py:167: only run on macos
SKIPPED [1] tests/test_paths.py:174: only run on windows
SKIPPED [1] tests/test_paths.py:180: only run on windows
SKIPPED [1] tests/test_paths.py:225: only run on macos
SKIPPED [1] tests/test_paths.py:231: only run on macos
SKIPPED [1] tests/test_paths.py:241: only run on windows
SKIPPED [1] tests/test_paths.py:247: only run on windows
SKIPPED [1] tests/test_paths.py:463: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] tests/test_paths.py:482: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] tests/test_paths.py:520: only runs on windows
SKIPPED [1] tests/test_paths.py:525: only runs on windows
FAILED tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-o...
FAILED tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-o...
FAILED tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-o...
FAILED tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
FAILED tests/test_troubleshoot.py::test_troubleshoot - AssertionError: assert 'pip list' in '$PATH:\n\t/home/tkloczko/rpmbuild/BUI...
============= 5 failed, 62 passed, 14 skipped, 1 warning in 2.03s ==============

@ivanov
Copy link
Member

ivanov commented Nov 17, 2023

@kloczek thanks for your report - can you suggest a path forward here? Do you think there a common cause to these errors or does it make more sense to open separate issues for the problems you're running into here, and tackle them one at a time?

@kloczek
Copy link
Author

kloczek commented Dec 18, 2023

Just tested 5.5.1 and this time I've looked a git closer. Looks like jupyter_core and test suite are trying to use pip.

jupyter_core/troubleshoot.py:    env["pip"] = subs([sys.executable, "-m", "pip", "list"])
jupyter_core/troubleshoot.py:    if environment_data["pip"]:
jupyter_core/troubleshoot.py:        print("\npip list:")
jupyter_core/troubleshoot.py:        for package in environment_data["pip"].split("\n"):
tests/test_troubleshoot.py:    assert "pip list" in out

Is it really necessary to use pip list? 🤔

@ivanov
Copy link
Member

ivanov commented Dec 18, 2023

Hmmm, not sure I follow. The code you're pointing to only runs if the system has a working pip, and the test_troubleshoot.py is not among the failures your reported.

@kloczek
Copy link
Author

kloczek commented Dec 18, 2023

Here is updated pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_core-5.5.1
configfile: pyproject.toml
testpaths: tests/
collected 81 items

tests/test_application.py .........                                                                                                                                                   [ 11%]
tests/test_command.py ..........FFF                                                                                                                                                   [ 27%]
tests/test_migrate.py ...........                                                                                                                                                     [ 40%]
tests/test_paths.py ..ss.....ssss....ssss.....F.........ssss...                                                                                                                       [ 93%]
tests/test_troubleshoot.py F                                                                                                                                                          [ 95%]
tests/test_utils.py ....                                                                                                                                                              [100%]

========================================================================================= FAILURES ==========================================================================================
_____________________________________________________________________________________ test_not_on_path ______________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0')

    @skip_darwin
    def test_not_on_path(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness = a.join("jupyter-witness")
        witness_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS ME")')
        write_executable(witness, witness_src)

        env = {"PATH": ""}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
        # This won't work on windows unless
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

a          = local('/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a')
env        = {'PATH': ''}
jupyter    = local('/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter')
tmpdir     = local('/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0')
witness    = local('/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter-witness')
witness_src = '#!/usr/bin/python3\nprint("WITNESS ME")\n'

tests/test_command.py:201:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'PATH': ''}}
        popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter', 'witness'],)
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': ''}, 'stdout': -1}, process = <subprocess.Popen object at 0x7fd0927a6940>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.

capture_output = False
check      = True
input      = None
kwargs     = {'env': {'PATH': ''}, 'stdout': -1}
popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter', 'witness'],)
process    = <subprocess.Popen object at 0x7fd0927a6940>
retcode    = 1
stderr     = None
stdout     = b''
timeout    = None

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
____________________________________________________________________________________ test_path_priority _____________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0')

    @skip_darwin
    def test_path_priority(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS A")')
        write_executable(witness_a, witness_a_src)

        b = tmpdir.mkdir("b")
        witness_b = b.join("jupyter-witness")
        witness_b_src = "#!{}\n{}\n".format(sys.executable, 'print("WITNESS B")')
        write_executable(witness_b, witness_b_src)

        env = {"PATH": str(b)}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

a          = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a')
b          = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/b')
env        = {'PATH': '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/b'}
jupyter    = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter')
tmpdir     = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0')
witness_a  = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter-witness')
witness_a_src = '#!/usr/bin/python3\nprint("WITNESS A")\n'
witness_b  = local('/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/b/jupyter-witness')
witness_b_src = '#!/usr/bin/python3\nprint("WITNESS B")\n'

tests/test_command.py:225:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/b'}}
        popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter', 'witness'],)
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter', 'witness'],)
kwargs = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/b'}, 'stdout': -1}, process = <subprocess.Popen object at 0x7fd092647340>, stdout = b'', stderr = None
retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.

capture_output = False
check      = True
input      = None
kwargs     = {'env': {'PATH': '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/b'}, 'stdout': -1}
popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter', 'witness'],)
process    = <subprocess.Popen object at 0x7fd092647340>
retcode    = 1
stderr     = None
stdout     = b''
timeout    = None

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
________________________________________________________________________________________ test_argv0 _________________________________________________________________________________________

tmpdir = local('/tmp/pytest-of-tkloczko/pytest-74/test_argv00')

    @skip_darwin
    def test_argv0(tmpdir):
        a = tmpdir.mkdir("a")
        jupyter = a.join("jupyter")
        jupyter.write("from jupyter_core import command; command.main()")
        jupyter.chmod(0o700)
        witness_a = a.join("jupyter-witness")
        witness_a_src = f"""#!{sys.executable}
    import sys
    print(sys.argv[0])
    """
        write_executable(witness_a, witness_a_src)

        env = {}
        if "SYSTEMROOT" in os.environ:  # Windows http://bugs.python.org/issue20614
            env["SYSTEMROOT"] = os.environ["SYSTEMROOT"]
        if sys.platform == "win32":
            env["PATHEXT"] = ".EXE"
>       out = check_output([sys.executable, str(jupyter), "witness"], env=env)

a          = local('/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a')
env        = {}
jupyter    = local('/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter')
tmpdir     = local('/tmp/pytest-of-tkloczko/pytest-74/test_argv00')
witness_a  = local('/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter-witness')
witness_a_src = '#!/usr/bin/python3\nimport sys\nprint(sys.argv[0])\n'

tests/test_command.py:247:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        kwargs     = {'env': {}}
        popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter', 'witness'],)
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter', 'witness'],)
kwargs = {'env': {}, 'stdout': -1}, process = <subprocess.Popen object at 0x7fd092851ee0>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.

capture_output = False
check      = True
input      = None
kwargs     = {'env': {}, 'stdout': -1}
popenargs  = (['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter', 'witness'],)
process    = <subprocess.Popen object at 0x7fd092851ee0>
retcode    = 1
stderr     = None
stdout     = b''
timeout    = None

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter", line 1, in <module>
    from jupyter_core import command; command.main()
ModuleNotFoundError: No module named 'jupyter_core'
_______________________________________________________________________________ test_jupyter_path_prefer_env ________________________________________________________________________________

    def test_jupyter_path_prefer_env():
        with prefer_env:
            path = jupyter_path()
>           assert path[0] == paths.ENV_JUPYTER_PATH[0]
E           AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
E             - /usr/share/jupyter
E             + /home/tkloczko/.local/share/jupyter

path       = ['/home/tkloczko/.local/share/jupyter', '/usr/share/jupyter']

tests/test_paths.py:314: AssertionError
_____________________________________________________________________________________ test_troubleshoot _____________________________________________________________________________________

capsys = <_pytest.capture.CaptureFixture object at 0x7fd092647df0>

    def test_troubleshoot(capsys):
        """Smoke test the troubleshoot function"""
        main()
        out = capsys.readouterr().out
>       assert "pip list" in out
E       AssertionError: assert 'pip list' in '$PATH:\n\t/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.1-2.fc35.x86_64/usr/bin\n\t/usr/bin\n\t/usr/sbin\...13.2.1-1)]\n\nplatform.platform():\n\tLinux-6.6.0-0.rc5.20231013git10a6e5feccb8.44.fc40.x86_64-x86_64-with-glibc2.34\n'

capsys     = <_pytest.capture.CaptureFixture object at 0x7fd092647df0>
out        = '$PATH:\n\t/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.1-2.fc35.x86_64/usr/bin\n\t/usr/bin\n\t/usr/sbin\...13.2.1-1)]\n\nplatform.platform():\n\tLinux-6.6.0-0.rc5.20231013git10a6e5feccb8.44.fc40.x86_64-x86_64-with-glibc2.34\n'

tests/test_troubleshoot.py:10: AssertionError
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
/usr/bin/python3: No module named pip
===================================================================================== warnings summary ======================================================================================
jupyter_core/application.py:23
  /home/tkloczko/rpmbuild/BUILD/jupyter_core-5.5.1/jupyter_core/application.py:23: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from .paths import (

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================================================================== slowest 10 durations ====================================================================================
0.19s call     tests/test_command.py::test_paths_debug
0.12s call     tests/test_command.py::test_config_dir
0.11s call     tests/test_command.py::test_runtime_dir
0.10s call     tests/test_command.py::test_paths_json
0.10s call     tests/test_command.py::test_subcommand_not_given
0.10s call     tests/test_command.py::test_subcommand_not_found
0.10s call     tests/test_command.py::test_help
0.10s call     tests/test_command.py::test_data_dir
0.09s call     tests/test_command.py::test_paths
0.04s call     tests/test_troubleshoot.py::test_troubleshoot
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_paths.py:112: only run on macos
SKIPPED [1] tests/test_paths.py:120: only run on windows
SKIPPED [1] tests/test_paths.py:162: only run on macos
SKIPPED [1] tests/test_paths.py:168: only run on macos
SKIPPED [1] tests/test_paths.py:175: only run on windows
SKIPPED [1] tests/test_paths.py:181: only run on windows
SKIPPED [1] tests/test_paths.py:226: only run on macos
SKIPPED [1] tests/test_paths.py:232: only run on macos
SKIPPED [1] tests/test_paths.py:242: only run on windows
SKIPPED [1] tests/test_paths.py:248: only run on windows
SKIPPED [1] tests/test_paths.py:464: only run on windows/cpython or pypy >= 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] tests/test_paths.py:481: only run on windows/pypy < 7.3.6: https://foss.heptapod.net/pypy/pypy/-/issues/3469
SKIPPED [1] tests/test_paths.py:519: only runs on windows
SKIPPED [1] tests/test_paths.py:524: only runs on windows
FAILED tests/test_command.py::test_not_on_path - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_not_on_path0/a/jupyter', 'witness']' returned non-zero exit status 1.
FAILED tests/test_command.py::test_path_priority - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_path_priority0/a/jupyter', 'witness']' returned non-zero exit status 1.
FAILED tests/test_command.py::test_argv0 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pytest-of-tkloczko/pytest-74/test_argv00/a/jupyter', 'witness']' returned non-zero exit status 1.
FAILED tests/test_paths.py::test_jupyter_path_prefer_env - AssertionError: assert '/home/tklocz...share/jupyter' == '/usr/share/jupyter'
FAILED tests/test_troubleshoot.py::test_troubleshoot - AssertionError: assert 'pip list' in '$PATH:\n\t/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-core-5.5.1-2.fc35.x86_64/usr/bin\n\t/usr/bin\n\t/usr/sbin\...13.2.1-1)]\n\nplatform.pla...
==================================================================== 5 failed, 62 passed, 14 skipped, 1 warning in 2.03s ====================================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants