在 Dockerfile 中安装仅限 CPU 的 PyTorch

Question

我对 Docker 和容器化相当陌生。我想减小生产中

my_proj

docker 容器的大小。

我更喜欢通过

Poetry

安装软件包和管理依赖项。

如何在 PyTorch 中指定使用

CPU-only

Dockerfile

？

通过以下方式执行此操作。

bash

终端，它将是：

poetry add pytorch-cpu torchvision-cpu -c pytorch

（或

conda install

...）

---

我现有的

Dockerfile

:

FROM python:3.7-slim as base
RUN apt-get update -y \
    && apt-get -y --no-install-recommends install curl wget\
    && rm -rf /var/lib/apt/lists/* 
ENV ROOT /home/worker/python/my_proj
WORKDIR $ROOT

ARG ATLASSIAN_TOKEN
ARG POETRY_HTTP_BASIC_AZURE_PASSWORD
ARG ACCESS_KEY
ENV AWS_ACCESS_KEY_ID=$ACCESS_KEY
ARG SECRET_KEY
ENV AWS_SECRET_ACCESS_KEY=$SECRET_KEY
ARG REPO
ENV REPO_URL=$REPO
ENV PYPIRC_PATH=$ROOT/.pypirc

ENV \
    PYTHONFAULTHANDLER=1 \
    POETRY_VERSION=1.1.4 \
    POETRY_HOME=/etc/poetry \
    XDG_CACHE_HOME=/home/worker/.cache \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    MPLCONFIGDIR=/home/worker/matplotlib \
    PATH=/home/worker/python/my_proj/.venv/bin:/usr/local/bin:/etc/poetry/bin:$PATH

ADD https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py ./
RUN python get-poetry.py && chmod +x /etc/poetry/bin/poetry
RUN --mount=type=cache,target=/root/.cache pip install twine keyring artifacts-keyring
RUN --mount=type=cache,target=/root/.cache apt update && apt install gcc -y

FROM base as ws
ARG WS_APIKEY
ARG WS_PROJECTVERSION=
ARG WS_PROJECTNAME=workers-python-my_proj
ARG WS_PRODUCTNAME=HALO
COPY --chown=worker:worker . .
RUN --mount=type=cache,uid=1000,target=/home/worker/.cache poetry install --no-dev
COPY --from=openjdk:15-slim-buster /usr/local/openjdk-15 /usr/local/openjdk-15
ENV JAVA_HOME /usr/local/openjdk-15
ENV PATH $JAVA_HOME/bin:$PATH
RUN --mount=type=cache,uid=1000,target=/home/worker/.cache ./wss_agent.sh

FROM base as test
COPY . .
RUN poetry config experimental.new-installer false
RUN poetry install
RUN cd my_proj && poetry run invoke deployconfluence_server_pass=$ATLASSIAN_TOKEN

FROM base as package
COPY . .
RUN poetry build
RUN python -m pip install --upgrade pip && \
pip install twine keyring artifacts-keyring && \
twine upload -r $REPO_URL --config-file $PYPIRC_PATH dist/* --skip-existing

FROM base as build
COPY . .
RUN poetry config experimental.new-installer false
RUN poetry install --no-dev
RUN pip3 --no-cache-dir install --upgrade awscli
RUN aws s3 cp s3://....tar.gz $ROOT/my_proj # censored url
RUN mkdir $ROOT/my_proj/bert-base-cased && cd $ROOT/my_proj/bert-base-cased && \
wget https://huggingface.co/bert-base-cased/resolve/main/config.json && \
wget https://huggingface.co/bert-base-cased/resolve/main/tokenizer.json && \
wget https://huggingface.co/bert-base-cased/resolve/main/tokenizer_config.json 

FROM python:3.7-slim as production
ENV  ROOT=/home/worker/python/my_proj \
     VIRTUAL_ENV=/home/worker/python/my_proj/.venv\
     PATH=/home/worker/python/my_proj/.venv/bin:/home/worker/python/my_proj:$PATH
COPY --from=build /home/worker/python/my_proj/pyproject.toml /home/worker/python/
COPY --from=build /home/worker/python/my_proj/.venv /home/worker/python/my_proj/.venv
COPY --from=build /home/worker/python/my_proj/my_proj /home/worker/python/my_proj
WORKDIR $ROOT
ENV PYTHONPATH=$ROOT:/home/worker/python/
ENTRYPOINT [ "primary_worker", "--mongo" ]

Answer 1

通过 pip 安装它应该可以：

RUN pip3 install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

Answer 2

如果您遇到

pip

，可通过

ERROR: Could not find a version that satisfies the requirement / ERROR: No matching distribution found

安装的替代方法：

RUN pip install torch==2.0.0 --index-url https://download.pytorch.org/whl/cpu

Answer 3

如果您需要将其添加到

requirements.txt

文件中，这就是完成的方法。

将此行添加到文件顶部：

--find-links https://download.pytorch.org/whl/torch_stable.html

像这样 -

requirements.txt

（加上将

+cpu

添加到版本中）：

--find-links https://download.pytorch.org/whl/torch_stable.html

annotated-types==0.6.0
anyio==4.2.0
async-timeout==4.0.3
asyncio==3.4.3
asyncpg==0.29.0
threadpoolctl==3.2.0
tokenizers==0.15.0
torch==2.1.2+cpu
torchvision==0.16.2+cpu

希望这可以帮助其他可能想知道这是如何完成的人。