我应该在序言中说,我确实按照
SpaCy
文档安装了 SpaCy
库和感兴趣的模型。
pip install -U pip setuptools wheel
pip install -U 'spacy[apple]'
python -m spacy download zh_core_web_sm
python -m spacy download en_core_web_sm
python -m spacy download fr_core_news_sm
python -m spacy download de_core_news_sm
python -m spacy download ja_core_news_sm
python -m spacy download es_core_news_sm
目前一直停留在在 Docker 环境中使用
ja_core_news_sm
的 Python 镜像在 Docker 中安装 python:3.11-slim
。我正在我的 SpaCy
Docker 镜像上串联安装一些其他 arm64
管道模块,这就是冲突的根源。其他管道模型是:
我收到以下错误:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.7/71.7 MB 4.4 MB/s eta 0:00:00
Building wheels for collected packages: sudachipy
Building wheel for sudachipy (pyproject.toml): started
Building wheel for sudachipy (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for sudachipy (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [39 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-311
creating build/lib.linux-aarch64-cpython-311/sudachipy
copying py_src/sudachipy/config.py -> build/lib.linux-aarch64-cpython-311/sudachipy
copying py_src/sudachipy/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy
copying py_src/sudachipy/command_line.py -> build/lib.linux-aarch64-cpython-311/sudachipy
copying py_src/sudachipy/errors.py -> build/lib.linux-aarch64-cpython-311/sudachipy
creating build/lib.linux-aarch64-cpython-311/sudachipy/dictionary
copying py_src/sudachipy/dictionary/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/dictionary
creating build/lib.linux-aarch64-cpython-311/sudachipy/tokenizer
copying py_src/sudachipy/tokenizer/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/tokenizer
creating build/lib.linux-aarch64-cpython-311/sudachipy/morphemelist
copying py_src/sudachipy/morphemelist/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/morphemelist
creating build/lib.linux-aarch64-cpython-311/sudachipy/morpheme
copying py_src/sudachipy/morpheme/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/morpheme
copying py_src/sudachipy/sudachipy.pyi -> build/lib.linux-aarch64-cpython-311/sudachipy
creating build/lib.linux-aarch64-cpython-311/sudachipy/resources
copying py_src/sudachipy/resources/sudachi.json -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
copying py_src/sudachipy/resources/rewrite.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
copying py_src/sudachipy/resources/unk.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
copying py_src/sudachipy/resources/char.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
running build_rust
error: can't find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for sudachipy
Failed to build sudachipy
ERROR: Could not build wheels for sudachipy, which is required to install pyproject.toml-based projects
这是我的 Dockerfile:
# Use the official Python image, with Python 3.11
FROM python:3.11-slim
# Set environment variables to reduce Python bytecode generation and buffering
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# Set working directory
WORKDIR /app
# Install essential dependencies including Python development headers and GCC
RUN apt-get update && \
apt-get install -y --no-install-recommends \
libpq-dev \
gcc \
ffmpeg \
python3-dev \
libc-dev \
&& apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Update pip and install Python packages
COPY ./docker-requirements.txt /app/
RUN pip install --upgrade pip && \
pip install --no-cache-dir -r docker-requirements.txt
# Copy application code to container
COPY . /app
# Expose the port the app runs on
EXPOSE 5000
# Make the entrypoint script executable
RUN chmod +x /app/shell_scripts/entrypoint.sh /app/shell_scripts/wait-for-it.sh /app/shell_scripts/docker-ngrok-tunnel.sh
# Define entrypoint
ENTRYPOINT ["/app/shell_scripts/entrypoint.sh"]
进一步阅读,这是一个存档模块。
这是来自 PyPi 的注释,它解释了这个问题,因为我的 Mac 具有 M2 芯片:
Binary wheels
We provide binary builds for macOS (10.14+), Windows and Linux only for x86_64 architecture. x86 32-bit architecture is not supported and is not tested. MacOS source builds seem to work on ARM-based (Aarch64) Macs, but this architecture also is not tested and require installing Rust toolchain and Cargo.
我似乎无法安装所有型号
感谢 @aab 在 Rust 编译器的方向上的推动。
银弹除了安装 Rust 编译器之外还升级了我的
es
SpaCy
管道,因为 sudachiPy
依赖于 Rust
编译器。
# Use the official Python image, with Python 3.11
FROM python:3.11-slim
# Set environment variables to reduce Python bytecode generation and buffering
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# Set working directory
WORKDIR /app
# Install essential dependencies including Python development headers and GCC
RUN apt-get update && \
apt-get install -y --no-install-recommends \
python3-dev \
build-essential \
git \
libpq-dev \
gcc \
ffmpeg \
libc-dev \
curl \
&& apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Install Rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Update pip and install Python packages
COPY ./docker-requirements.txt /app/
RUN pip install --upgrade pip && \
pip install --no-cache-dir -r docker-requirements.txt
# Install Cython, SpaCy and language models
RUN pip install -U pip setuptools wheel && \
pip install -U spacy && \
pip install --upgrade 'sudachipy>=0.6.8' && \
python -m spacy download zh_core_web_sm && \
python -m spacy download en_core_web_sm && \
python -m spacy download fr_core_news_md && \
python -m spacy download de_core_news_sm && \
python -m spacy download es_core_news_md && \
python -m spacy download ja_core_news_sm
# Copy application code to container
COPY . /app
# Expose the port the app runs on
EXPOSE 5000
# Make the entrypoint script executable
RUN chmod +x /app/shell_scripts/entrypoint.sh /app/shell_scripts/wait-for-it.sh /app/shell_scripts/docker-ngrok-tunnel.sh
# Define entrypoint
ENTRYPOINT ["/app/shell_scripts/entrypoint.sh"]