Docker stacks foundation image build fails with current mamba #2146

vizeit · 2024-09-25T22:55:39Z

What docker image(s) are you using?

base-notebook, docker-stacks-foundation, minimal-notebook, pytorch-notebook, scipy-notebook

Host OS system

Ubuntu 22.04

Host architecture

x86_64

What Docker command are you running?

Docker build

How to Reproduce the problem?

Build docker stacks foundation image, the build throws an error

Command output

No response

Expected behavior

No response

Actual behavior

build results into the following error,

#15 ERROR: process "/bin/bash -o pipefail -c set -x &&     arch=$(uname -m) &&     if [ \"${arch}\" = \"x86_64\" ]; then         arch=\"64\";     fi &&     wget --progress=dot:giga -O -         \"[https://micro.mamba.pm/api/micromamba/linux-${arch}/latest\](https://micro.mamba.pm/api/micromamba/linux-$%7Barch%7D/latest/)" | tar -xvj bin/micromamba &&     PYTHON_SPECIFIER=\"python=${PYTHON_VERSION}\" &&     if [[ \"${PYTHON_VERSION}\" == \"default\" ]]; then PYTHON_SPECIFIER=\"python\"; fi &&     ./bin/micromamba install         --root-prefix=\"${CONDA_DIR}\"         --prefix=\"${CONDA_DIR}\"         --yes         \"${PYTHON_SPECIFIER}\"         'mamba'         'jupyter_core' &&     rm -rf /tmp/bin/ &&     mamba list --full-name 'python' | tail -1 | tr -s ' ' | cut -d ' ' -f 1,2 | sed 's/\\.[^.]*$/.*/' >> \"${CONDA_DIR}/conda-meta/pinned\" &&     mamba clean --all -f -y &&     fix-permissions \"${CONDA_DIR}\" &&     fix-permissions \"/home/${NB_USER}\"" did not complete successfully: exit code: 109
------
 > [10/14] RUN set -x &&     arch=$(uname -m) &&     if [ "${arch}" = "x86_64" ]; then         arch="64";     fi &&     wget --progress=dot:giga -O -         "[https://micro.mamba.pm/api/micromamba/linux-${arch}/latest](https://micro.mamba.pm/api/micromamba/linux-$%7Barch%7D/latest)" | tar -xvj bin/micromamba &&     PYTHON_SPECIFIER="python=3.11" &&     if [[ "3.11" == "default" ]]; then PYTHON_SPECIFIER="python"; fi &&     ./bin/micromamba install         --root-prefix="/opt/conda"         --prefix="/opt/conda"         --yes         "${PYTHON_SPECIFIER}"         'mamba'         'jupyter_core' &&     rm -rf /tmp/bin/ &&     mamba list --full-name 'python' | tail -1 | tr -s ' ' | cut -d ' ' -f 1,2 | sed 's/\.[^.]*$/.*/' >> "/opt/conda/conda-meta/pinned" &&     mamba clean --all -f -y &&     fix-permissions "/opt/conda" &&     fix-permissions "/home/jovyan":
16.42     micromamba run -n base mycommand
16.42 
16.43 + rm -rf /tmp/bin/
16.44 + mamba list --full-name python
16.44 + tail -1
16.44 + tr -s ' '
16.44 + cut -d ' ' -f 1,2
16.44 + sed 's/\.[^.]*$/.*/'
16.45 The following argument was not expected: --full-name
16.45 Run with --help for more information.
------
Dockerfile:104
--------------------
 103 |     WORKDIR /tmp
 104 | >>> RUN set -x && \
 105 | >>>     arch=$(uname -m) && \
 106 | >>>     if [ "${arch}" = "x86_64" ]; then \
 107 | >>>         # Should be simpler, see <https://github.com/mamba-org/mamba/issues/1437>
 108 | >>>         arch="64"; \
 109 | >>>     fi && \
 110 | >>>     # https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html#linux-and-macos
 111 | >>>     wget --progress=dot:giga -O - \
 112 | >>>         "[https://micro.mamba.pm/api/micromamba/linux-${arch}/latest](https://micro.mamba.pm/api/micromamba/linux-$%7Barch%7D/latest)" | tar -xvj bin/micromamba && \
 113 | >>>     PYTHON_SPECIFIER="python=${PYTHON_VERSION}" && \
 114 | >>>     if [[ "${PYTHON_VERSION}" == "default" ]]; then PYTHON_SPECIFIER="python"; fi && \
 115 | >>>     # Install the packages
 116 | >>>     ./bin/micromamba install \
 117 | >>>         --root-prefix="${CONDA_DIR}" \
 118 | >>>         --prefix="${CONDA_DIR}" \
 119 | >>>         --yes \
 120 | >>>         "${PYTHON_SPECIFIER}" \
 121 | >>>         'mamba' \
 122 | >>>         'jupyter_core' && \
 123 | >>>     rm -rf /tmp/bin/ && \
 124 | >>>     # Pin major.minor version of python
 125 | >>>     # https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#preventing-packages-from-updating-pinning
 126 | >>>     mamba list --full-name 'python' | tail -1 | tr -s ' ' | cut -d ' ' -f 1,2 | sed 's/\.[^.]*$/.*/' >> "${CONDA_DIR}/conda-meta/pinned" && \
 127 | >>>     mamba clean --all -f -y && \
 128 | >>>     fix-permissions "${CONDA_DIR}" && \
 129 | >>>     fix-permissions "/home/${NB_USER}"
 130 |     
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c set -x &&     arch=$(uname -m) &&     if [ \"${arch}\" = \"x86_64\" ]; then         arch=\"64\";     fi &&     wget --progress=dot:giga -O -         \"[https://micro.mamba.pm/api/micromamba/linux-${arch}/latest\](https://micro.mamba.pm/api/micromamba/linux-$%7Barch%7D/latest/)" | tar -xvj bin/micromamba &&     PYTHON_SPECIFIER=\"python=${PYTHON_VERSION}\" &&     if [[ \"${PYTHON_VERSION}\" == \"default\" ]]; then PYTHON_SPECIFIER=\"python\"; fi &&     ./bin/micromamba install         --root-prefix=\"${CONDA_DIR}\"         --prefix=\"${CONDA_DIR}\"         --yes         \"${PYTHON_SPECIFIER}\"         'mamba'         'jupyter_core' &&     rm -rf /tmp/bin/ &&     mamba list --full-name 'python' | tail -1 | tr -s ' ' | cut -d ' ' -f 1,2 | sed 's/\\.[^.]*$/.*/' >> \"${CONDA_DIR}/conda-meta/pinned\" &&     mamba clean --all -f -y &&     fix-permissions \"${CONDA_DIR}\" &&     fix-permissions \"/home/${NB_USER}\"" did not complete successfully: exit code: 109

Anything else?

No response

Latest Docker version

I've updated my Docker version to the latest available, and the issue persists

The text was updated successfully, but these errors were encountered:

jabbera · 2024-09-26T01:47:04Z

+1 here.

jabbera · 2024-09-26T02:01:54Z

I added the following to my build script to work around it. I'm hoping I don't also have to pin it.

sed -i "s/latest/1.5.10/g" ./docker-stacks/docker-stacks-foundation/Dockerfile
sed -i "s/'mamba' /'mamba==1.5.10' /g" ./docker-stacks/docker-stacks-foundation/Dockerfile

manics · 2024-09-26T10:43:14Z

Mamba 2.0.0 has just been released. @jjerphan is the failure above

mamba list --full-name python

The following argument was not expected: --full-name

expected?

JohanMabille · 2024-09-26T12:25:12Z

It's a regression, we are working on a fix.

ktaletsk · 2024-09-26T14:28:13Z

It would be a good practice to pin micromamba version, regardless of the fix

mathbunnyru · 2024-09-30T07:25:40Z

I pushed a fix preventing mamba 2.0.0 from installing.
Hope next release will solve the problem and I will revert my change.

It would be a good practice to pin micromamba version, regardless of the fix

Note: the error on our side occurs with mamba, not micromamba during mamba list --full-name 'python'

Pinning micromamba version would not have helped here, as far as I understand: the error is caused by mamba, and the latest mamba would have still been installed, regardless of micromamba version.

Pinning mamba version is also not something I would like to do - I prefer this repo to have unpinned versions for almost everything, and it works well - people eventually get the latest versions of software they use, and maintenance is much easier as well.
And if something breaks, it breaks fast, and we should let developers of upstream packages know about failures faster - it makes these bugs easier to track and fix.

jabbera · 2024-09-30T18:05:37Z

Pinning micromamba version would not have helped here, as far as I understand: the error is caused by mamba, and the latest mamba would have still been installed, regardless of micromamba version.

You are correct.

JohanMabille · 2024-10-03T13:09:57Z

We have released micromamba 2.0.2 which should fix this issue, can you confirm it does?

mathbunnyru · 2024-10-03T13:36:50Z

We have released micromamba 2.0.2 which should fix this issue, can you confirm it does?

The --full-name issue has been fixed, I confirm.

There are still remaining issues with mamba v2, which prevent us from updating, please, take a look here: #2147
The main one is not being able to use conda when mamba v2 is installed.

vizeit added the type:Bug A problem with the definition of one of the docker images maintained here label Sep 25, 2024

vizeit changed the title ~~Docker stacks foundation image build fails with current micromamba~~ Docker stacks foundation image build fails with current mamba Sep 25, 2024

JohanMabille mentioned this issue Sep 26, 2024

Micromamba 2.0: the command "list" dropped the --full-name option mamba-org/mamba#3473

Closed

jrbourbeau mentioned this issue Sep 28, 2024

Notebook image build failure dask/dask-docker#330

Open

mathbunnyru linked a pull request Oct 1, 2024 that will close this issue

Install mamba 2.0 #2147

Open

4 tasks

cganley1 mentioned this issue Oct 1, 2024

modified for mamba version and package version updates spacetelescope/science-platform-images#64

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker stacks foundation image build fails with current mamba #2146

Docker stacks foundation image build fails with current mamba #2146

vizeit commented Sep 25, 2024 •

edited

Loading

jabbera commented Sep 26, 2024

jabbera commented Sep 26, 2024

manics commented Sep 26, 2024 •

edited

Loading

JohanMabille commented Sep 26, 2024

ktaletsk commented Sep 26, 2024

mathbunnyru commented Sep 30, 2024 •

edited

Loading

jabbera commented Sep 30, 2024

JohanMabille commented Oct 3, 2024

mathbunnyru commented Oct 3, 2024 •

edited

Loading

Docker stacks foundation image build fails with current mamba #2146

Docker stacks foundation image build fails with current mamba #2146

Comments

vizeit commented Sep 25, 2024 • edited Loading

What docker image(s) are you using?

Host OS system

Host architecture

What Docker command are you running?

How to Reproduce the problem?

Command output

Expected behavior

Actual behavior

Anything else?

Latest Docker version

jabbera commented Sep 26, 2024

jabbera commented Sep 26, 2024

manics commented Sep 26, 2024 • edited Loading

JohanMabille commented Sep 26, 2024

ktaletsk commented Sep 26, 2024

mathbunnyru commented Sep 30, 2024 • edited Loading

jabbera commented Sep 30, 2024

JohanMabille commented Oct 3, 2024

mathbunnyru commented Oct 3, 2024 • edited Loading

vizeit commented Sep 25, 2024 •

edited

Loading

manics commented Sep 26, 2024 •

edited

Loading

mathbunnyru commented Sep 30, 2024 •

edited

Loading

mathbunnyru commented Oct 3, 2024 •

edited

Loading