Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: containerize filed #4117

Open
FahriBilici opened this issue Aug 11, 2023 · 29 comments
Open

bug: containerize filed #4117

FahriBilici opened this issue Aug 11, 2023 · 29 comments
Labels
bug Something isn't working

Comments

@FahriBilici
Copy link

Describe the bug

I am trying to containerize a classification model from huggingface. I followed guide from docs and it was working until the containerize step. when I was trying to containerize it in >>> RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh part it throw this error ERROR: failed to solve: process "/bin/sh -c bash -euxo pipefail /home/bentoml/bento/env/python/install.sh" did not complete successfully: exit code: 2
here is the explanation log:
Encountered exception while trying to building image: Command '['/usr/local/bin/docker', 'build', '--tag', 'test:at4eqcbyj2weflg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpb83n2mw1fsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpb83n2mw1fsTempFS/']' returned non-zero exit status 1. Traceback (most recent call last): File "/Users/-/opt/anaconda3/envs/solity/lib/python3.10/site-packages/bentoml/_internal/container/__init__.py", line 234, in build return builder.build(**kwargs) File "/Users/-/opt/anaconda3/envs/solity/lib/python3.10/site-packages/bentoml/_internal/container/base.py", line 190, in build raise BentoMLException(str(e)) from None bentoml.exceptions.BentoMLException: Command '['/usr/local/bin/docker', 'build', '--tag', 'test:at4eqcbyj2weflg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpb83n2mw1fsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpb83n2mw1fsTempFS/']' returned non-zero exit status 1.

To reproduce

No response

Expected behavior

No response

Environment

bentoml:1.1.1
python:3.10.12
@FahriBilici FahriBilici added the bug Something isn't working label Aug 11, 2023
@aarnphm
Copy link
Contributor

aarnphm commented Aug 11, 2023

Can you send the whole stack trace here?

@FahriBilici
Copy link
Author

Can you send the whole stack trace here?

57.89 /home/bentoml/bento/env/python/install.sh: line 25:     8 Killed                  pip3 install -r "$REQUIREMENTS_TXT" "${PIP_ARGS[@]}"
------
Dockerfile:43
--------------------
  41 |     COPY --chown=bentoml:bentoml ./env/python ./env/python/
  42 |     # install python packages with install.sh
  43 | >>> RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh
  44 |     COPY --chown=bentoml:bentoml . ./
  45 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c bash -euxo pipefail /home/bentoml/bento/env/python/install.sh" did not complete successfully: exit code: 137

Encountered exception while trying to building image: Command '['/usr/local/bin/docker', 'build', '--tag', 'test:at4eqcbyj2weflg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmppizt58bgfsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmppizt58bgfsTempFS/']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/fahribilici/opt/anaconda3/envs/solity/lib/python3.10/site-packages/bentoml/_internal/container/__init__.py", line 234, in build
    return builder.build(**kwargs)
  File "/Users/fahribilici/opt/anaconda3/envs/solity/lib/python3.10/site-packages/bentoml/_internal/container/base.py", line 190, in build
    raise BentoMLException(str(e)) from None
bentoml.exceptions.BentoMLException: Command '['/usr/local/bin/docker', 'build', '--tag', 'test:at4eqcbyj2weflg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmppizt58bgfsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmppizt58bgfsTempFS/']' returned non-zero exit status 1.

@aarnphm
Copy link
Contributor

aarnphm commented Aug 14, 2023

Can you try

docker builder prune --filter type=exec.cachemount

@FahriBilici
Copy link
Author

Can you try

docker builder prune --filter type=exec.cachemount

I tried this and also prune --all, but nothing changed. The same error happens in the same step

@aarnphm
Copy link
Contributor

aarnphm commented Aug 14, 2023

can you send your bentofile.yaml here?

@FahriBilici
Copy link
Author

FahriBilici commented Aug 15, 2023 •
edited
Loading

can you send your bentofile.yaml here?

service: 'service:svc'
include:
  - '*.py'
exclude:
  - "__pycache__"
  - "test.py"
python:
  requirements_txt: "requirments.txt"
models:
  - sentiment-analysis:latest

@FahriBilici
Copy link
Author

I also tried with the example from website and I am getting the same error

@aarnphm
Copy link
Contributor

aarnphm commented Aug 19, 2023

Can you send bentoml env -o bash?

@FahriBilici
Copy link
Author

FahriBilici commented Aug 19, 2023 •
edited
Loading

Can you send bentoml env -o bash?

py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
BENTOML_DEBUG=''
BENTOML_QUIET=''
BENTOML_BUNDLE_LOCAL_BUILD=''
BENTOML_DO_NOT_TRACK=''
BENTOML_CONFIG=''
BENTOML_CONFIG_OPTIONS=''
BENTOML_PORT=''
BENTOML_HOST=''
BENTOML_API_WORKERS=''
bentoml="1.1.1"
python="3.10.12"
platform="macOS-10.16-x86_64-i386-64bit"
uid_gid="501:20"
conda="4.8.3"
in_conda_env="True"
conda_packages=( "name: solity" "channels:" "  - conda-forge" "  - defaults" "dependencies:" "  - appnope=0.1.3=pyhd8ed1ab_0" "  - asttokens=2.2.1=pyhd8ed1ab_0" "  - backcall=0.2.0=pyh9f0ad1d_0" "  - backports=1.0=pyhd8ed1ab_3" "  - backports.functools_lru_cache=1.6.5=pyhd8ed1ab_0" "  - bzip2=1.0.8=h1de35cc_0" "  - ca-certificates=2023.7.22=h8857fd0_0" "  - comm=0.1.4=pyhd8ed1ab_0" "  - decorator=5.1.1=pyhd8ed1ab_0" "  - executing=1.2.0=pyhd8ed1ab_0" "  - importlib_metadata=6.8.0=hd8ed1ab_0" "  - ipykernel=6.25.1=pyh5fb750a_0" "  - ipython=8.14.0=pyhd1c38e8_0" "  - jedi=0.19.0=pyhd8ed1ab_0" "  - jupyter_client=8.3.0=pyhd8ed1ab_0" "  - jupyter_core=5.3.1=py310h2ec42d9_0" "  - libcxx=16.0.6=hd57cbcb_0" "  - libffi=3.4.4=hecd8cb5_0" "  - libsodium=1.0.18=hbcb3906_1" "  - matplotlib-inline=0.1.6=pyhd8ed1ab_0" "  - ncurses=6.4=hcec6c5f_0" "  - nest-asyncio=1.5.6=pyhd8ed1ab_0" "  - openssl=3.1.2=h8a1eda9_0" "  - parso=0.8.3=pyhd8ed1ab_0" "  - pexpect=4.8.0=pyh1a96a4e_2" "  - pickleshare=0.7.5=py_1003" "  - prompt-toolkit=3.0.39=pyha770c72_0" "  - prompt_toolkit=3.0.39=hd8ed1ab_0" "  - ptyprocess=0.7.0=pyhd3deb0d_0" "  - pure_eval=0.2.2=pyhd8ed1ab_0" "  - python=3.10.12=h5ee71fb_0" "  - python-dateutil=2.8.2=pyhd8ed1ab_0" "  - python_abi=3.10=2_cp310" "  - readline=8.2=hca72f7f_0" "  - six=1.16.0=pyh6c4a22f_0" "  - sqlite=3.41.2=h6c40b1e_0" "  - stack_data=0.6.2=pyhd8ed1ab_0" "  - tk=8.6.12=h5d9f67b_0" "  - traitlets=5.9.0=pyhd8ed1ab_0" "  - typing_extensions=4.7.1=pyha770c72_0" "  - wcwidth=0.2.6=pyhd8ed1ab_0" "  - xz=5.4.2=h6c40b1e_0" "  - zeromq=4.3.4=he49afe7_1" "  - zlib=1.2.13=h4dc903c_0" "  - pip:" "    - absl-py==1.4.0" "    - accelerate==0.21.0" "    - adal==1.2.7" "    - aiofiles==23.1.0" "    - aiohttp==3.8.4" "    - aiokafka==0.8.1" "    - aiosignal==1.3.1" "    - alembic==1.11.1" "    - anyio==3.7.1" "    - appdirs==1.4.4" "    - argcomplete==2.1.2" "    - asgiref==3.7.2" "    - astunparse==1.6.3" "    - async-timeout==4.0.2" "    - attrs==23.1.0" "    - azure-common==1.1.28" "    - azure-core==1.28.0" "    - azure-graphrbac==0.61.1" "    - azure-mgmt-authorization==3.0.0" "    - azure-mgmt-containerregistry==10.1.0" "    - azure-mgmt-core==1.4.0" "    - azure-mgmt-keyvault==10.2.2" "    - azure-mgmt-resource==22.0.0" "    - azure-mgmt-storage==21.0.0" "    - azureml-core==1.52.0" "    - backports-tempfile==1.0" "    - backports-weakref==1.0.post1" "    - bcrypt==4.0.1" "    - bentoml==1.1.1" "    - blinker==1.6.2" "    - boto3==1.28.3" "    - botocore==1.31.3" "    - brotli==1.0.9" "    - build==0.10.0" "    - cachetools==5.3.1" "    - cattrs==23.1.2" "    - certifi==2023.5.7" "    - cffi==1.15.1" "    - charset-normalizer==3.2.0" "    - circus==0.18.0" "    - click==8.1.5" "    - click-option-group==0.5.6" "    - cloudpickle==2.2.1" "    - contextlib2==21.6.0" "    - contourpy==1.1.0" "    - cryptography==41.0.2" "    - cycler==0.11.0" "    - databricks-cli==0.17.7" "    - debugpy==1.6.8" "    - deepmerge==1.1.0" "    - deprecated==1.2.14" "    - distlib==0.3.6" "    - docker==6.1.3" "    - entrypoints==0.4" "    - exceptiongroup==1.1.2" "    - fastapi==0.89.1" "    - filelock==3.12.2" "    - flask==2.3.2" "    - flatbuffers==23.5.26" "    - fonttools==4.41.0" "    - frozenlist==1.4.0" "    - fs==2.4.16" "    - fsspec==2023.6.0" "    - gast==0.4.0" "    - gevent==23.7.0" "    - geventhttpclient==2.0.2" "    - gitdb==4.0.10" "    - gitpython==3.1.32" "    - google-api-core==2.11.1" "    - google-auth==2.22.0" "    - google-auth-oauthlib==1.0.0" "    - google-cloud-core==2.3.3" "    - google-cloud-storage==2.10.0" "    - google-crc32c==1.5.0" "    - google-pasta==0.2.0" "    - google-resumable-media==2.5.0" "    - googleapis-common-protos==1.59.1" "    - greenlet==2.0.2" "    - grpcio==1.56.0" "    - gunicorn==20.1.0" "    - h11==0.14.0" "    - h5py==3.9.0" "    - huggingface-hub==0.16.4" "    - humanfriendly==10.0" "    - idna==3.4" "    - importlib-metadata==6.0.1" "    - importlib-resources==6.0.0" "    - isodate==0.6.1" "    - itsdangerous==2.1.2" "    - jeepney==0.8.0" "    - jinja2==3.1.2" "    - jmespath==1.0.1" "    - joblib==1.3.1" "    - jsonpickle==3.0.1" "    - jupyter-core==5.3.1" "    - kafka-python==2.0.2" "    - keras==2.13.1" "    - kiwisolver==1.4.4" "    - knack==0.10.1" "    - kubernetes==26.1.0" "    - libclang==16.0.0" "    - mako==1.2.4" "    - markdown==3.4.3" "    - markdown-it-py==3.0.0" "    - markupsafe==2.1.3" "    - matplotlib==3.7.2" "    - mdurl==0.1.2" "    - mlflow==2.5.0" "    - mlserver==1.3.5" "    - mlserver-mlflow==1.3.5" "    - mpmath==1.3.0" "    - msal==1.22.0" "    - msal-extensions==1.0.0" "    - msrest==0.7.1" "    - msrestazure==0.6.4" "    - multidict==6.0.4" "    - ndg-httpsclient==0.5.1" "    - networkx==3.1" "    - nltk==3.8.1" "    - numpy==1.24.3" "    - oauthlib==3.2.2" "    - opentelemetry-api==1.18.0" "    - opentelemetry-instrumentation==0.39b0" "    - opentelemetry-instrumentation-aiohttp-client==0.39b0" "    - opentelemetry-instrumentation-asgi==0.39b0" "    - opentelemetry-sdk==1.18.0" "    - opentelemetry-semantic-conventions==0.39b0" "    - opentelemetry-util-http==0.39b0" "    - opt-einsum==3.3.0" "    - orjson==3.9.2" "    - packaging==23.0" "    - pandas==2.0.3" "    - paramiko==3.2.0" "    - pathspec==0.11.1" "    - pillow==10.0.0" "    - pip==23.1.2" "    - pip-requirements-parser==32.0.1" "    - pip-tools==7.3.0" "    - pkginfo==1.9.6" "    - platformdirs==3.8.1" "    - portalocker==2.7.0" "    - prometheus-client==0.17.1" "    - prometheus-flask-exporter==0.22.4" "    - protobuf==3.20.0" "    - psutil==5.9.5" "    - py-grpc-prometheus==0.7.0" "    - pyarrow==12.0.1" "    - pyasn1==0.5.0" "    - pyasn1-modules==0.3.0" "    - pycparser==2.21" "    - pydantic==1.10.11" "    - pygments==2.15.1" "    - pyjwt==2.7.0" "    - pynacl==1.5.0" "    - pynvml==11.5.0" "    - pyopenssl==23.2.0" "    - pyparsing==3.0.9" "    - pyproject-hooks==1.0.0" "    - pysftp==0.2.9" "    - pysocks==1.7.1" "    - python-dotenv==1.0.0" "    - python-json-logger==2.0.7" "    - python-multipart==0.0.6" "    - python-rapidjson==1.10" "    - pytz==2023.3" "    - pyyaml==6.0" "    - pyzmq==25.1.1" "    - querystring-parser==1.2.4" "    - regex==2023.6.3" "    - requests==2.31.0" "    - requests-auth-aws-sigv4==0.7" "    - requests-oauthlib==1.3.1" "    - rich==13.5.2" "    - rsa==4.9" "    - s3transfer==0.6.1" "    - safetensors==0.3.1" "    - schema==0.7.5" "    - scikit-learn==1.3.0" "    - scipy==1.11.1" "    - secretstorage==3.3.3" "    - sentencepiece==0.1.99" "    - setuptools==67.8.0" "    - simple-di==0.1.5" "    - smmap==5.0.0" "    - sniffio==1.3.0" "    - sqlalchemy==2.0.18" "    - sqlparse==0.4.4" "    - starlette==0.22.0" "    - starlette-exporter==0.16.0" "    - sympy==1.12" "    - tabulate==0.9.0" "    - tensorboard==2.13.0" "    - tensorboard-data-server==0.7.1" "    - tensorflow==2.13.0" "    - tensorflow-estimator==2.13.0" "    - tensorflow-io-gcs-filesystem==0.32.0" "    - termcolor==2.3.0" "    - threadpoolctl==3.2.0" "    - tokenizers==0.13.3" "    - tomli==2.0.1" "    - torch==2.0.1" "    - torchvision==0.15.2" "    - tornado==6.3.2" "    - tqdm==4.65.0" "    - transformers==4.30.2" "    - tritonclient==2.35.0" "    - typing-extensions==4.5.0" "    - tzdata==2023.3" "    - urllib3==1.26.16" "    - uvicorn==0.22.0" "    - uvloop==0.17.0" "    - virtualenv==20.23.1" "    - watchfiles==0.19.0" "    - websocket-client==1.6.1" "    - werkzeug==2.3.6" "    - wheel==0.38.4" "    - wrapt==1.15.0" "    - yarl==1.9.2" "    - zipp==3.16.1" "    - zope-event==5.0" "    - zope-interface==6.0" "prefix: /Users/fahribilici/opt/anaconda3/envs/solity" "" )
pip_packages=( "absl-py==1.4.0" "accelerate==0.21.0" "adal==1.2.7" "aiofiles==23.1.0" "aiohttp==3.8.4" "aiokafka==0.8.1" "aiosignal==1.3.1" "alembic==1.11.1" "anyio==3.7.1" "appdirs==1.4.4" "appnope @ file:///home/conda/feedstock_root/build_artifacts/appnope_1649077682618/work" "argcomplete==2.1.2" "asgiref==3.7.2" "asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1670263926556/work" "astunparse==1.6.3" "async-timeout==4.0.2" "attrs==23.1.0" "azure-common==1.1.28" "azure-core==1.28.0" "azure-graphrbac==0.61.1" "azure-mgmt-authorization==3.0.0" "azure-mgmt-containerregistry==10.1.0" "azure-mgmt-core==1.4.0" "azure-mgmt-keyvault==10.2.2" "azure-mgmt-resource==22.0.0" "azure-mgmt-storage==21.0.0" "azureml-core==1.52.0" "backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work" "backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1687772187254/work" "backports.tempfile==1.0" "backports.weakref==1.0.post1" "bcrypt==4.0.1" "bentoml==1.1.1" "blinker==1.6.2" "boto3==1.28.3" "botocore==1.31.3" "Brotli==1.0.9" "build==0.10.0" "cachetools==5.3.1" "cattrs==23.1.2" "certifi==2023.5.7" "cffi==1.15.1" "charset-normalizer==3.2.0" "circus==0.18.0" "click==8.1.5" "click-option-group==0.5.6" "cloudpickle==2.2.1" "comm @ file:///home/conda/feedstock_root/build_artifacts/comm_1691044910542/work" "contextlib2==21.6.0" "contourpy==1.1.0" "cryptography==41.0.2" "cycler==0.11.0" "databricks-cli==0.17.7" "debugpy @ file:///Users/runner/miniforge3/conda-bld/debugpy_1691021314990/work" "decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work" "deepmerge==1.1.0" "Deprecated==1.2.14" "distlib==0.3.6" "docker==6.1.3" "entrypoints==0.4" "exceptiongroup==1.1.2" "executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1667317341051/work" "fastapi==0.89.1" "filelock==3.12.2" "Flask==2.3.2" "flatbuffers==23.5.26" "fonttools==4.41.0" "frozenlist==1.4.0" "fs==2.4.16" "fsspec==2023.6.0" "gast==0.4.0" "gevent==23.7.0" "geventhttpclient==2.0.2" "gitdb==4.0.10" "GitPython==3.1.32" "google-api-core==2.11.1" "google-auth==2.22.0" "google-auth-oauthlib==1.0.0" "google-cloud-core==2.3.3" "google-cloud-storage==2.10.0" "google-crc32c==1.5.0" "google-pasta==0.2.0" "google-resumable-media==2.5.0" "googleapis-common-protos==1.59.1" "greenlet==2.0.2" "grpcio==1.56.0" "gunicorn==20.1.0" "h11==0.14.0" "h5py==3.9.0" "huggingface-hub==0.16.4" "humanfriendly==10.0" "idna==3.4" "importlib-metadata==6.0.1" "importlib-resources==6.0.0" "ipykernel @ file:///Users/runner/miniforge3/conda-bld/ipykernel_1691424478312/work" "ipython @ file:///Users/runner/miniforge3/conda-bld/ipython_1685727999785/work" "isodate==0.6.1" "itsdangerous==2.1.2" "jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1690896916983/work" "jeepney==0.8.0" "Jinja2==3.1.2" "jmespath==1.0.1" "joblib==1.3.1" "jsonpickle==3.0.1" "jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1687700988094/work" "jupyter_core @ file:///Users/runner/miniforge3/conda-bld/jupyter_core_1686775719487/work" "kafka-python==2.0.2" "keras==2.13.1" "kiwisolver==1.4.4" "knack==0.10.1" "kubernetes==26.1.0" "libclang==16.0.0" "Mako==1.2.4" "Markdown==3.4.3" "markdown-it-py==3.0.0" "MarkupSafe==2.1.3" "matplotlib==3.7.2" "matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work" "mdurl==0.1.2" "mlflow==2.5.0" "mlserver==1.3.5" "mlserver-mlflow==1.3.5" "mpmath==1.3.0" "msal==1.22.0" "msal-extensions==1.0.0" "msrest==0.7.1" "msrestazure==0.6.4" "multidict==6.0.4" "ndg-httpsclient==0.5.1" "nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1664684991461/work" "networkx==3.1" "nltk==3.8.1" "numpy==1.24.3" "oauthlib==3.2.2" "opentelemetry-api==1.18.0" "opentelemetry-instrumentation==0.39b0" "opentelemetry-instrumentation-aiohttp-client==0.39b0" "opentelemetry-instrumentation-asgi==0.39b0" "opentelemetry-sdk==1.18.0" "opentelemetry-semantic-conventions==0.39b0" "opentelemetry-util-http==0.39b0" "opt-einsum==3.3.0" "orjson==3.9.2" "packaging==23.0" "pandas==2.0.3" "paramiko==3.2.0" "parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work" "pathspec==0.11.1" "pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1667297516076/work" "pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work" "Pillow==10.0.0" "pip-requirements-parser==32.0.1" "pip-tools==7.3.0" "pkginfo==1.9.6" "platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1690813113769/work" "portalocker==2.7.0" "prometheus-client==0.17.1" "prometheus-flask-exporter==0.22.4" "prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1688565951714/work" "protobuf==3.20.0" "psutil @ file:///Users/runner/miniforge3/conda-bld/psutil_1681775164927/work" "ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl" "pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work" "py-grpc-prometheus==0.7.0" "pyarrow==12.0.1" "pyasn1==0.5.0" "pyasn1-modules==0.3.0" "pycparser==2.21" "pydantic==1.10.11" "Pygments==2.15.1" "PyJWT==2.7.0" "PyNaCl==1.5.0" "pynvml==11.5.0" "pyOpenSSL==23.2.0" "pyparsing==3.0.9" "pyproject_hooks==1.0.0" "pysftp==0.2.9" "PySocks==1.7.1" "python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work" "python-dotenv==1.0.0" "python-json-logger==2.0.7" "python-multipart==0.0.6" "python-rapidjson==1.10" "pytz==2023.3" "PyYAML==6.0" "pyzmq @ file:///Users/runner/miniforge3/conda-bld/pyzmq_1691667536098/work" "querystring-parser==1.2.4" "regex==2023.6.3" "requests==2.31.0" "requests-auth-aws-sigv4==0.7" "requests-oauthlib==1.3.1" "rich==13.5.2" "rsa==4.9" "s3transfer==0.6.1" "safetensors==0.3.1" "schema==0.7.5" "scikit-learn==1.3.0" "scipy==1.11.1" "SecretStorage==3.3.3" "sentencepiece==0.1.99" "simple-di==0.1.5" "six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work" "smmap==5.0.0" "sniffio==1.3.0" "SQLAlchemy==2.0.18" "sqlparse==0.4.4" "stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1669632077133/work" "starlette==0.22.0" "starlette-exporter==0.16.0" "sympy==1.12" "tabulate==0.9.0" "tensorboard==2.13.0" "tensorboard-data-server==0.7.1" "tensorflow==2.13.0" "tensorflow-estimator==2.13.0" "tensorflow-io-gcs-filesystem==0.32.0" "termcolor==2.3.0" "threadpoolctl==3.2.0" "tokenizers==0.13.3" "tomli==2.0.1" "torch==2.0.1" "torchvision==0.15.2" "tornado @ file:///Users/runner/miniforge3/conda-bld/tornado_1684150243998/work" "tqdm==4.65.0" "traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1675110562325/work" "transformers==4.30.2" "tritonclient==2.35.0" "typing_extensions==4.5.0" "tzdata==2023.3" "urllib3==1.26.16" "uvicorn==0.22.0" "uvloop==0.17.0" "virtualenv==20.23.1" "watchfiles==0.19.0" "wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1673864653149/work" "websocket-client==1.6.1" "Werkzeug==2.3.6" "wrapt==1.15.0" "yarl==1.9.2" "zipp==3.16.1" "zope.event==5.0" "zope.interface==6.0" )

@aarnphm
Copy link
Contributor

aarnphm commented Aug 20, 2023

What is that warning on L1?

@FahriBilici
Copy link
Author

What is that warning on L1?

I didn't understand what you mean

@aarnphm
Copy link
Contributor

aarnphm commented Aug 22, 2023 

py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

@FahriBilici
Copy link
Author 

py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

this not happened again but here full tracelog again:

Building OCI-compliant image for test:at4eqcbyj2weflg6 with docker

[+] Building 105.1s (12/15)                                docker:desktop-linux
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 1.75kB                                     0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim        2.1s
 => [base-container  1/11] FROM docker.io/library/python:3.10-slim@sha25  25.6s
 => => resolve docker.io/library/python:3.10-slim@sha256:4d440b214e447ded  0.0s
 => => sha256:52d2b7f179e32b4cbd579ee3c4958027988f9a827 29.12MB / 29.12MB  2.5s
 => => sha256:2b8a9a2240c1224b34f6aafbc3310f9a3fe65bd6893 3.50MB / 3.50MB  0.6s
 => => sha256:618a49bbc6c68a58b082ceb072a9464370d0203a5 17.43MB / 17.43MB  2.3s
 => => sha256:4d440b214e447deddc0a94de23a3d97d28dfafdf125 1.65kB / 1.65kB  0.0s
 => => sha256:13cc673c11ee90d6ba92d95f35f4d8e59148937f1e3 1.37kB / 1.37kB  0.0s
 => => sha256:f31204aad67273a64cc5b0e64e2a613ded5d817d909 6.92kB / 6.92kB  0.0s
 => => sha256:708d13076071a7b9fb3eefb4a360636911b7d83b3bb1939 243B / 243B  1.2s
 => => sha256:90b76fc2ebde9ad1526763819788ed8ea1817179c05 3.37MB / 3.37MB  2.2s
 => => extracting sha256:52d2b7f179e32b4cbd579ee3c4958027988f9a8274850ab  11.8s
 => => extracting sha256:2b8a9a2240c1224b34f6aafbc3310f9a3fe65bd689305090  1.2s
 => => extracting sha256:618a49bbc6c68a58b082ceb072a9464370d0203a55c70e9c  6.6s
 => => extracting sha256:708d13076071a7b9fb3eefb4a360636911b7d83b3bb19391  0.1s
 => => extracting sha256:90b76fc2ebde9ad1526763819788ed8ea1817179c0520cd9  1.5s
 => [internal] load build context                                         46.9s
 => => transferring context: 1.13GB                                       46.5s
 => [base-container  2/11] RUN rm -f /etc/apt/apt.conf.d/docker-clean; ec  1.1s
 => [base-container  3/11] RUN --mount=type=cache,target=/var/lib/apt --  35.2s
 => [base-container  4/11] RUN groupadd -g 1034 -o bentoml && useradd -m   0.5s
 => [base-container  5/11] RUN mkdir /home/bentoml/bento && chown bentoml  0.2s
 => [base-container  6/11] WORKDIR /home/bentoml/bento                     0.0s
 => [base-container  7/11] COPY --chown=bentoml:bentoml ./env/python ./en  0.0s
 => ERROR [base-container  8/11] RUN --mount=type=cache,target=/root/.ca  40.0s
------
 > [base-container  8/11] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh:
0.202 + set -exuo pipefail
0.204 +++ dirname -- /home/bentoml/bento/env/python/install.sh
0.208 ++ cd -- /home/bentoml/bento/env/python
0.208 ++ pwd
0.210 + BASEDIR=/home/bentoml/bento/env/python
0.212 + PIP_ARGS=(--no-warn-script-location)
0.214 + REQUIREMENTS_TXT=/home/bentoml/bento/env/python/requirements.txt
0.216 + REQUIREMENTS_LOCK=/home/bentoml/bento/env/python/requirements.lock.txt
0.220 + WHEELS_DIR=/home/bentoml/bento/env/python/wheels
0.220 + BENTOML_VERSION=1.1.1
0.220 + '[' -f /home/bentoml/bento/env/python/requirements.lock.txt ']'
0.220 + '[' -f /home/bentoml/bento/env/python/requirements.txt ']'
0.220 + echo 'Installing pip packages from '\''requirements.txt'\''..'
0.220 + pip3 install -r /home/bentoml/bento/env/python/requirements.txt --no-warn-script-location
0.220 Installing pip packages from 'requirements.txt'..
2.518 Collecting certifi==2023.7.22
2.642   Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
2.688      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 3.8 MB/s eta 0:00:00
2.877 Collecting charset-normalizer==3.2.0
2.901   Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
2.918      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.8/201.8 kB 20.5 MB/s eta 0:00:00
2.975 Collecting filelock==3.12.2
2.998   Downloading filelock-3.12.2-py3-none-any.whl (10 kB)
3.069 Collecting fsspec==2023.6.0
3.097   Downloading fsspec-2023.6.0-py3-none-any.whl (163 kB)
3.110      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.8/163.8 kB 26.8 MB/s eta 0:00:00
3.189 Collecting huggingface-hub==0.16.4
3.219   Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
3.232      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.8/268.8 kB 35.2 MB/s eta 0:00:00
3.279 Collecting idna==3.4
3.301   Downloading idna-3.4-py3-none-any.whl (61 kB)
3.310      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 14.1 MB/s eta 0:00:00
3.367 Collecting Jinja2==3.1.2
3.389   Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
3.399      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 26.1 MB/s eta 0:00:00
3.535 Collecting MarkupSafe==2.1.3
3.556   Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
3.628 Collecting mpmath==1.3.0
3.657   Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
3.677      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 36.0 MB/s eta 0:00:00
3.773 Collecting networkx==3.1
3.827   Downloading networkx-3.1-py3-none-any.whl (2.1 MB)
3.917      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 25.9 MB/s eta 0:00:00
4.387 Collecting numpy==1.25.2
4.424   Downloading numpy-1.25.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
5.245      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 20.5 MB/s eta 0:00:00
5.385 Collecting packaging==23.1
5.408   Downloading packaging-23.1-py3-none-any.whl (48 kB)
5.416      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 13.5 MB/s eta 0:00:00
5.830 Collecting protobuf==4.24.0
5.854   Downloading protobuf-4.24.0-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
5.868      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.6/311.6 kB 35.3 MB/s eta 0:00:00
5.969 Collecting PyYAML==6.0.1
5.997   Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
6.023      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 33.8 MB/s eta 0:00:00
6.963 Collecting regex==2023.8.8
6.995   Downloading regex-2023.8.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (771 kB)
7.022      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 771.9/771.9 kB 34.4 MB/s eta 0:00:00
7.109 Collecting requests==2.31.0
7.133   Downloading requests-2.31.0-py3-none-any.whl (62 kB)
7.142      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 15.3 MB/s eta 0:00:00
7.403 Collecting safetensors==0.3.1
7.436   Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
7.482      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 31.6 MB/s eta 0:00:00
7.631 Collecting sentencepiece==0.1.99
7.668   Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
7.711      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 35.4 MB/s eta 0:00:00
7.850 Collecting sympy==1.12
7.918   Downloading sympy-1.12-py3-none-any.whl (5.7 MB)
8.164      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 23.7 MB/s eta 0:00:00
8.630 Collecting tokenizers==0.13.3
8.657   Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
9.136      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 16.4 MB/s eta 0:00:00
9.262 Collecting torch==2.0.1
9.287   Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
39.79 /home/bentoml/bento/env/python/install.sh: line 25:     8 Killed                  pip3 install -r "$REQUIREMENTS_TXT" "${PIP_ARGS[@]}"
------
Dockerfile:43
--------------------
  41 |     COPY --chown=bentoml:bentoml ./env/python ./env/python/
  42 |     # install python packages with install.sh
  43 | >>> RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh
  44 |     COPY --chown=bentoml:bentoml . ./
  45 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c bash -euxo pipefail /home/bentoml/bento/env/python/install.sh" did not complete successfully: exit code: 137

Encountered exception while trying to building image: Command '['/usr/local/bin/docker', 'build', '--tag', 'test:at4eqcbyj2weflg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpuy0o5plffsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpuy0o5plffsTempFS/']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/fahribilici/opt/anaconda3/envs/solity/lib/python3.10/site-packages/bentoml/_internal/container/__init__.py", line 234, in build
    return builder.build(**kwargs)
  File "/Users/fahribilici/opt/anaconda3/envs/solity/lib/python3.10/site-packages/bentoml/_internal/container/base.py", line 190, in build
    raise BentoMLException(str(e)) from None
bentoml.exceptions.BentoMLException: Command '['/usr/local/bin/docker', 'build', '--tag', 'test:at4eqcbyj2weflg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpuy0o5plffsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpuy0o5plffsTempFS/']' returned non-zero exit status 1.

@aarnphm
Copy link
Contributor

aarnphm commented Aug 22, 2023

I assume you are using conda?

@FahriBilici
Copy link
Author

I assume you are using conda?

yes

@FahriBilici
Copy link
Author

I created new enviroment and followed this guide and i am getting same error

@FahriBilici
Copy link
Author

@aarnphm can you reproduce it with this?

I created new enviroment and followed this guide and i am getting same error

@aarnphm
Copy link
Contributor

aarnphm commented Aug 25, 2023

Can you try this in a virtualenv or non-conda environment?

@FahriBilici
Copy link
Author

Can you try this in a virtualenv or non-conda environment?

tried with virtualenv. same error happens in same step

WARNING: --strip-extras is becoming the default in version 8.0.0. To silence this warning, either use --strip-extras to opt into the new default or use --no-strip-extras to retain the existing behavior.

██████╗ ███████╗███╗   ██╗████████╗ ██████╗ ███╗   ███╗██╗
██╔══██╗██╔════╝████╗  ██║╚══██╔══╝██╔═══██╗████╗ ████║██║
██████╔╝█████╗  ██╔██╗ ██║   ██║   ██║   ██║██╔████╔██║██║
██╔══██╗██╔══╝  ██║╚██╗██║   ██║   ██║   ██║██║╚██╔╝██║██║
██████╔╝███████╗██║ ╚████║   ██║   ╚██████╔╝██║ ╚═╝ ██║███████╗
╚═════╝ ╚══════╝╚═╝  ╚═══╝   ╚═╝    ╚═════╝ ╚═╝     ╚═╝╚══════╝

Successfully built Bento(tag="summarization:rl2wefcdjgv2vlg6").

Possible next steps:

 * Containerize your Bento with `bentoml containerize`:
    $ bentoml containerize summarization:rl2wefcdjgv2vlg6  [or bentoml build --containerize]

 * Push to BentoCloud with `bentoml push`:
    $ bentoml push summarization:rl2wefcdjgv2vlg6 [or bentoml build --push]
(bento_docker_env) (bento) fahribilici@Mehmet-MacBook-Pro bentoml_example % bentoml containerize summarization:latest
Building OCI-compliant image for summarization:rl2wefcdjgv2vlg6 with docker

[+] Building 157.5s (12/15)                                                                                                                                  docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                         0.0s
 => => transferring dockerfile: 1.71kB                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                            0.0s
 => => transferring context: 2B                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim                                                                                                          4.3s
 => [base-container  1/11] FROM docker.io/library/python:3.10-slim@sha256:4d440b214e447deddc0a94de23a3d97d28dfafdf125a8b4bb8073381510c9ee2                                   0.0s
 => [internal] load build context                                                                                                                                           37.1s
 => => transferring context: 1.23GB                                                                                                                                         37.0s
 => CACHED [base-container  2/11] RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache     0.0s
 => [base-container  3/11] RUN --mount=type=cache,target=/var/lib/apt --mount=type=cache,target=/var/cache/apt set -eux &&     apt-get update -y &&     apt-get install -q  42.3s
 => [base-container  4/11] RUN groupadd -g 1034 -o bentoml && useradd -m -u 1034 -g 1034 -o -r bentoml                                                                       0.4s 
 => [base-container  5/11] RUN mkdir /home/bentoml/bento && chown bentoml:bentoml /home/bentoml/bento -R                                                                     0.1s 
 => [base-container  6/11] WORKDIR /home/bentoml/bento                                                                                                                       0.0s 
 => [base-container  7/11] COPY --chown=bentoml:bentoml ./env/python ./env/python/                                                                                           0.0s 
 => ERROR [base-container  8/11] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh                              110.1s 
------                                                                                                                                                                            
 > [base-container  8/11] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh:                                           
0.122 + set -exuo pipefail                                                                                                                                                        
0.124 +++ dirname -- /home/bentoml/bento/env/python/install.sh                                                                                                                    
0.126 ++ cd -- /home/bentoml/bento/env/python                                                                                                                                     
0.127 ++ pwd                                                                                                                                                                      
0.128 + BASEDIR=/home/bentoml/bento/env/python
0.128 + PIP_ARGS=(--no-warn-script-location)
0.129 + REQUIREMENTS_TXT=/home/bentoml/bento/env/python/requirements.txt
0.129 + REQUIREMENTS_LOCK=/home/bentoml/bento/env/python/requirements.lock.txt
0.129 + WHEELS_DIR=/home/bentoml/bento/env/python/wheels
0.129 + BENTOML_VERSION=1.1.3
0.130 + '[' -f /home/bentoml/bento/env/python/requirements.lock.txt ']'
0.130 + echo 'Installing pip packages from '\''requirements.lock.txt'\''..'
0.131 Installing pip packages from 'requirements.lock.txt'..
0.131 + pip3 install -r /home/bentoml/bento/env/python/requirements.lock.txt --no-warn-script-location
2.174 Collecting certifi==2023.7.22
2.178   Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
2.342 Collecting charset-normalizer==3.2.0
2.348   Using cached charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
2.415 Collecting filelock==3.12.2
2.419   Using cached filelock-3.12.2-py3-none-any.whl (10 kB)
2.482 Collecting fsspec==2023.6.0
2.486   Using cached fsspec-2023.6.0-py3-none-any.whl (163 kB)
2.570 Collecting huggingface-hub==0.16.4
2.575   Using cached huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
2.625 Collecting idna==3.4
2.630   Using cached idna-3.4-py3-none-any.whl (61 kB)
2.685 Collecting jinja2==3.1.2
2.690   Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
2.808 Collecting markupsafe==2.1.3
2.814   Using cached MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
2.861 Collecting mpmath==1.3.0
2.867   Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
2.938 Collecting networkx==3.1
2.961   Using cached networkx-3.1-py3-none-any.whl (2.1 MB)
3.466 Collecting numpy==1.25.2
3.543   Using cached numpy-1.25.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
3.697 Collecting packaging==23.1
3.701   Using cached packaging-23.1-py3-none-any.whl (48 kB)
3.801 Collecting pyyaml==6.0.1
3.807   Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
4.682 Collecting regex==2023.8.8
4.687   Using cached regex-2023.8.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (771 kB)
4.787 Collecting requests==2.31.0
4.791   Using cached requests-2.31.0-py3-none-any.whl (62 kB)
4.954 Collecting safetensors==0.3.3
5.159   Downloading safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
5.385      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 5.8 MB/s eta 0:00:00
5.446 Collecting sympy==1.12
5.464   Using cached sympy-1.12-py3-none-any.whl (5.7 MB)
5.799 Collecting tokenizers==0.13.3
5.817   Using cached tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
5.970 Collecting torch==2.0.1
6.018   Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
110.0 /home/bentoml/bento/env/python/install.sh: line 25:     8 Killed                  pip3 install -r "$REQUIREMENTS_LOCK" "${PIP_ARGS[@]}"
------
Dockerfile:43
--------------------
  41 |     COPY --chown=bentoml:bentoml ./env/python ./env/python/
  42 |     # install python packages with install.sh
  43 | >>> RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh
  44 |     COPY --chown=bentoml:bentoml . ./
  45 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c bash -euxo pipefail /home/bentoml/bento/env/python/install.sh" did not complete successfully: exit code: 137

Encountered exception while trying to building image: Command '['/usr/local/bin/docker', 'build', '--tag', 'summarization:rl2wefcdjgv2vlg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpvqcuvf4ifsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpvqcuvf4ifsTempFS/']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/fahribilici/bento_docker_env/lib/python3.10/site-packages/bentoml/_internal/container/__init__.py", line 234, in build
    return builder.build(**kwargs)
  File "/Users/fahribilici/bento_docker_env/lib/python3.10/site-packages/bentoml/_internal/container/base.py", line 190, in build
    raise BentoMLException(str(e)) from None
bentoml.exceptions.BentoMLException: Command '['/usr/local/bin/docker', 'build', '--tag', 'summarization:rl2wefcdjgv2vlg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpvqcuvf4ifsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmpvqcuvf4ifsTempFS/']' returned non-zero exit status 1.

@aarnphm
Copy link
Contributor

aarnphm commented Aug 25, 2023 •
edited
Loading

How about your requirements.txt? Can you send the package you include in here?

@FahriBilici
Copy link
Author

How about your requirements.txt? Can you send the package you include in here?

I did it with deploy-a-transformer-model-with-bentoml tutorial from docks. So there is no requirments txt file but here is bentofile.yaml:

service: 'service:svc'
include:
  - '*.py'
python:
  packages:
    - torch
    - transformers
docker:
  system_packages:
    - git
models:
  - summarization:latest

@aarnphm
Copy link
Contributor

aarnphm commented Aug 25, 2023 •
edited
Loading

Can you try containerize with --opt platform=linux/amd64?

@FahriBilici
Copy link
Author

Can you try containerize with --opt platform=linux/amd64?

I tried and added --debug option. I guess same error?


Memoized: {'platform': ('linux/amd64',)}
Building OCI-compliant image for summarization:rl2wefcdjgv2vlg6 with docker

Building OCI-compliant image for summarization:rl2wefcdjgv2vlg6 with docker

Using base Dockerfile template: "python_debian.j2" (path: "/Users/fahribilici/opt/anaconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/container/frontend/dockerfile/templates/python_debian.j2")
docker build cmd: '/usr/local/bin/docker build --platform linux/amd64 --progress plain --tag summarization:rl2wefcdjgv2vlg6 --file /private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmplcunxd1zfsTempFS/env/docker/Dockerfile /var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmplcunxd1zfsTempFS/'
#0 building with "desktop-linux" instance using docker driver

#1 [internal] load .dockerignore
#1 transferring context: 2B 0.0s done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.71kB 0.0s done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/python:3.10-slim
#3 DONE 3.2s

#4 [base-container  1/11] FROM docker.io/library/python:3.10-slim@sha256:4d440b214e447deddc0a94de23a3d97d28dfafdf125a8b4bb8073381510c9ee2
#4 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 89.32MB 1.8s
#5 transferring context: 339.24MB 6.9s
#5 transferring context: 585.75MB 12.0s
#5 transferring context: 829.67MB 17.1s
#5 transferring context: 1.06GB 22.1s
#5 transferring context: 1.23GB 26.1s done
#5 DONE 26.1s

#6 [base-container  3/11] RUN --mount=type=cache,target=/var/lib/apt --mount=type=cache,target=/var/cache/apt set -eux &&     apt-get update -y &&     apt-get install -q -y --no-install-recommends --allow-remove-essential         ca-certificates gnupg2 bash build-essential git
#6 CACHED

#7 [base-container  4/11] RUN groupadd -g 1034 -o bentoml && useradd -m -u 1034 -g 1034 -o -r bentoml
#7 CACHED

#8 [base-container  5/11] RUN mkdir /home/bentoml/bento && chown bentoml:bentoml /home/bentoml/bento -R
#8 CACHED

#9 [base-container  6/11] WORKDIR /home/bentoml/bento
#9 CACHED

#10 [base-container  2/11] RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
#10 CACHED

#11 [base-container  7/11] COPY --chown=bentoml:bentoml ./env/python ./env/python/
#11 CACHED

#12 [base-container  8/11] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh
#12 0.370 + set -exuo pipefail
#12 0.372 +++ dirname -- /home/bentoml/bento/env/python/install.sh
#12 0.374 ++ cd -- /home/bentoml/bento/env/python
#12 0.375 ++ pwd
#12 0.376 + BASEDIR=/home/bentoml/bento/env/python
#12 0.376 + PIP_ARGS=(--no-warn-script-location)
#12 0.376 + REQUIREMENTS_TXT=/home/bentoml/bento/env/python/requirements.txt
#12 0.377 + REQUIREMENTS_LOCK=/home/bentoml/bento/env/python/requirements.lock.txt
#12 0.377 + WHEELS_DIR=/home/bentoml/bento/env/python/wheels
#12 0.377 + BENTOML_VERSION=1.1.3
#12 0.378 + '[' -f /home/bentoml/bento/env/python/requirements.lock.txt ']'
#12 0.378 + echo 'Installing pip packages from '\''requirements.lock.txt'\''..'
#12 0.378 Installing pip packages from 'requirements.lock.txt'..
#12 0.379 + pip3 install -r /home/bentoml/bento/env/python/requirements.lock.txt --no-warn-script-location
#12 2.487 Collecting certifi==2023.7.22
#12 2.493   Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
#12 2.654 Collecting charset-normalizer==3.2.0
#12 2.659   Using cached charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
#12 2.725 Collecting filelock==3.12.2
#12 2.731   Using cached filelock-3.12.2-py3-none-any.whl (10 kB)
#12 2.803 Collecting fsspec==2023.6.0
#12 2.807   Using cached fsspec-2023.6.0-py3-none-any.whl (163 kB)
#12 2.883 Collecting huggingface-hub==0.16.4
#12 2.889   Using cached huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
#12 2.946 Collecting idna==3.4
#12 2.951   Using cached idna-3.4-py3-none-any.whl (61 kB)
#12 3.011 Collecting jinja2==3.1.2
#12 3.015   Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
#12 3.135 Collecting markupsafe==2.1.3
#12 3.140   Using cached MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
#12 3.190 Collecting mpmath==1.3.0
#12 3.196   Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
#12 3.268 Collecting networkx==3.1
#12 3.284   Using cached networkx-3.1-py3-none-any.whl (2.1 MB)
#12 3.707 Collecting numpy==1.25.2
#12 3.800   Using cached numpy-1.25.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
#12 4.327 Collecting packaging==23.1
#12 4.332   Using cached packaging-23.1-py3-none-any.whl (48 kB)
#12 4.432 Collecting pyyaml==6.0.1
#12 4.437   Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
#12 5.291 Collecting regex==2023.8.8
#12 5.297   Using cached regex-2023.8.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (771 kB)
#12 5.388 Collecting requests==2.31.0
#12 5.392   Using cached requests-2.31.0-py3-none-any.whl (62 kB)
#12 5.558 Collecting safetensors==0.3.3
#12 5.566   Using cached safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
#12 5.634 Collecting sympy==1.12
#12 5.653   Using cached sympy-1.12-py3-none-any.whl (5.7 MB)
#12 5.982 Collecting tokenizers==0.13.3
#12 6.003   Using cached tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
#12 6.147 Collecting torch==2.0.1
#12 6.336   Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
#12 160.7 /home/bentoml/bento/env/python/install.sh: line 25:     9 Killed                  pip3 install -r "$REQUIREMENTS_LOCK" "${PIP_ARGS[@]}"
#12 ERROR: process "/bin/sh -c bash -euxo pipefail /home/bentoml/bento/env/python/install.sh" did not complete successfully: exit code: 137
------
 > [base-container  8/11] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh:
5.392   Using cached requests-2.31.0-py3-none-any.whl (62 kB)
5.558 Collecting safetensors==0.3.3
5.566   Using cached safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
5.634 Collecting sympy==1.12
5.653   Using cached sympy-1.12-py3-none-any.whl (5.7 MB)
5.982 Collecting tokenizers==0.13.3
6.003   Using cached tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
6.147 Collecting torch==2.0.1
6.336   Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
160.7 /home/bentoml/bento/env/python/install.sh: line 25:     9 Killed                  pip3 install -r "$REQUIREMENTS_LOCK" "${PIP_ARGS[@]}"
------
Dockerfile:43
--------------------
  41 |     COPY --chown=bentoml:bentoml ./env/python ./env/python/
  42 |     # install python packages with install.sh
  43 | >>> RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bento/env/python/install.sh
  44 |     COPY --chown=bentoml:bentoml . ./
  45 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c bash -euxo pipefail /home/bentoml/bento/env/python/install.sh" did not complete successfully: exit code: 137

Encountered exception while trying to building image: Command '['/usr/local/bin/docker', 'build', '--platform', 'linux/amd64', '--progress', 'plain', '--tag', 'summarization:rl2wefcdjgv2vlg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmplcunxd1zfsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmplcunxd1zfsTempFS/']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/fahribilici/opt/anaconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/container/__init__.py", line 234, in build
    return builder.build(**kwargs)
  File "/Users/fahribilici/opt/anaconda3/envs/bento/lib/python3.10/site-packages/bentoml/_internal/container/base.py", line 190, in build
    raise BentoMLException(str(e)) from None
bentoml.exceptions.BentoMLException: Command '['/usr/local/bin/docker', 'build', '--platform', 'linux/amd64', '--progress', 'plain', '--tag', 'summarization:rl2wefcdjgv2vlg6', '--file', '/private/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmplcunxd1zfsTempFS/env/docker/Dockerfile', '/var/folders/yc/wf15jqy94h32mzth0h_hdw_r0000gn/T/tmplcunxd1zfsTempFS/']' returned non-zero exit status 1.
Tracking Error: Not found default cloud context, please login!

@aarnphm
Copy link
Contributor

aarnphm commented Aug 25, 2023

Sorry but I tried on both M1 and EC2 with x64 and arm, cannot reproduce this bug with virtualenv

@aarnphm
Copy link
Contributor

aarnphm commented Aug 25, 2023

What version of docker is this?

@FahriBilici
Copy link
Author

What version of docker is this?

I am using a Macbook with an intel chip. I followed the quickstart example project, thinking my model had problems. Here is my docker version:

Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.21.1 (114176)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:52:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@FahriBilici
Copy link
Author

@aarnphm I tried with my friend m1 and it worked. The problem may be related to the Intel chip. Is there any idea about solving this problem?

@Mayurji
Copy link

Mayurji commented Sep 21, 2023

@FahriBilici, is this issue resolved?

@nengelmann nengelmann mentioned this issue Jun 18, 2024
Open
@nengelmann
Copy link

For me it was resolved by removing my snap installation of docker and reinstall docker according to the docker documentation. I used the convenience-script for Ubuntu. You might also need the post-installation steps for your docker installation. In the original post the docker installation was not done via snap, however as I face the same trace back except the docker path, your docker installation is likely to cause this issue.

  1. Reinstall docker according to the docker documentation
  2. Execute post-installation steps (Maybe not needed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Mayurji @FahriBilici @aarnphm @nengelmann