当pip要求中包含雪花依赖性时,docker build失败

问题描述 投票:1回答:3

我试图从我的python应用程序中构建docker镜像。但是当我在pip requirements.txt文件中添加“snowflake-connector-python”时,docker构建失败了。

requirements.txt: -

boto3
pycryptodome
snowflake-connector-python

Dockerfile: -

FROM python:alpine3.6
RUN python --version

COPY . /sample-app
WORKDIR /sample-app

RUN python -m pip install --upgrade pip
RUN pip install --upgrade setuptools
RUN pip --version
RUN pip install -r requirements.txt

CMD [ "python", "runner.py" ]

CLI docker build: -

docker build -t sample-app:1.0 .

错误: -

Step 8/9 : RUN pip install -r requirements.txt
 ---> Running in 13ab5873e0b5
Collecting boto3 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/92/37/bcd816851f8a04de7d2ba9df5893acc4c6680fb3ab4442895490724143fb/boto3-1.7.67-py2.py3-none-any.whl (128kB)
Collecting pycryptodome (from -r requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/96/15/a2a2aa89f57a17075579fcd296a0fcff98640776666a4371d33f7f613bb9/pycryptodome-3.6.4.tar.gz (7.1MB)
Collecting snowflake-connector-python (from -r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/66/df/e44ec4d55fce341c9756f5f6012049b85f6a679fc3a423eefd7f66527a46/snowflake_connector_python-1.6.5-py2.py3-none-any.whl (142kB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl
Collecting botocore<1.11.0,>=1.10.67 (from boto3->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/d6/81/59f34f9965277be8e480efea09b949e406b1afbfcf29a87c6ae6c2b52121/botocore-1.10.67-py2.py3-none-any.whl (4.4MB)
Collecting s3transfer<0.2.0,>=0.1.10 (from boto3->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)
Collecting pyjwt (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/93/d1/3378cc8184a6524dc92993090ee8b4c03847c567e298305d6cf86987e005/PyJWT-1.6.4-py2.py3-none-any.whl
Collecting pyasn1<0.5.0,>=0.2.1 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl (72kB)
Collecting six (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting pyOpenSSL<18.0.0,>=16.2.0 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/79/db/7c0cfe4aa8341a5fab4638952520d8db6ab85ff84505e12c00ea311c3516/pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
Collecting cryptography<2.3,>=1.8.2 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/ec/b2/faa78c1ab928d2b2c634c8b41ff1181f0abdd9adf9193211bd606ffa57e2/cryptography-2.2.2.tar.gz (443kB)
Collecting azure-storage<=0.36.0,>0.34.2 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/76/26/482c033f8f3a48d16cf75aad91c3e1256856719f4117fabb4696d33aa5da/azure_storage-0.36.0-py2.py3-none-any.whl (190kB)
Collecting azure-common<2 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/91/66/6636001c0b6be5423f4bbaffe5febbf9eda39bd1941eb39a73504d8f6f71/azure_common-1.1.14-py2.py3-none-any.whl
Collecting ijson (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/7f/e9/8508c5f4987ba238a2b169e582c1f70a47272b22a2f1fb06b9318201bb9e/ijson-2.3-py2.py3-none-any.whl
Collecting cffi>=1.9 (from snowflake-connector-python->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/e7/a7/4cd50e57cc6f436f1cc3a7e8fa700ff9b8b4d471620629074913e3735fb2/cffi-1.11.5.tar.gz (438kB)
    Complete output from command python setup.py egg_info:

        No working compiler found, or bogus compiler options passed to
        the compiler from Python's standard "distutils" module.  See
        the error messages above.  Likely, the problem is not related
        to CFFI but generic to the setup.py of any Python package that
        tries to compile C code.  (Hints: on OS/X 10.8, for errors about
        -mno-fused-madd see http://stackoverflow.com/questions/22313407/
        Otherwise, see https://wiki.python.org/moin/CompLangPython or
        the IRC channel #python on irc.freenode.net.)

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-7dudc372/cffi/
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

可能是什么问题 ?我在这里错过了什么吗?

https://docs.snowflake.net/manuals/user-guide/python-connector-install.html

python-3.x docker dockerfile snowflake-datawarehouse
3个回答
2
投票

alpine linux非常小,他们不得不删除所有东西,所以你必须自己安装它。有些pip包需要在你的机器上编译/构建,但是alpine缺少这些依赖性,所以安装它们,然后进行pip安装,然后删除它们,因为它们不再需要,你将减小图像大小。

雪花docs还说你必须安装libffi-devopenssl-dev

FROM python:alpine3.6

COPY . /sample-app
WORKDIR /sample-app

RUN apk add --update --no-cache --virtual build-deps gcc python3-dev musl-dev libc-dev linux-headers libxslt-dev libxml2-dev \
&& apk add libffi-dev openssl-dev \
&& pip install --upgrade pip setuptools \
&& pip install -r requirements.txt \
&& apk del build-deps

CMD [ "python", "runner.py" ]

1
投票

将编译库/包添加到您的alpine图像,在RUN上面/之前的Dockerfile中添加pip install语句 -

WORKDIR /sample-app
RUN apk add --no-cache build-base libffi-dev openssl-dev ncurses-dev
RUN python -m pip install --upgrade pip

这基本上安装了gcc,g ++和make来编译包,在你的情况下也需要其他包。我猜他们被删除或没有原装qazxsw poi图像。


0
投票

我发现在linux上使用python:alpine3.6有助于为AWS lambda构建python部署包,但不确定这是否有用。

我需要像Mazel Tov建议的virtual env,以获得pip不包含的剩余依赖项。

© www.soinside.com 2019 - 2024. All rights reserved.