为 AWS Lambda 函数构建 Docker 映像时出错:由于缺少 HDF5 库,无法安装 h5py

问题描述 投票:0回答:1

我正在尝试为需要 h5py 库的 AWS Lambda 函数构建 Docker 映像。但是,我在构建过程中遇到了与缺少 HDF5 依赖项相关的错误。

 => ERROR [4/6] RUN pip3 install h5py                                                                                                                                                                                               5.3s 
------                                                                                                                                                                                                                                   
 > [4/6] RUN pip3 install h5py:                                                                                                                                                                                                          
0.346 Collecting h5py                                                                                                                                                                                                                    
0.410   Downloading h5py-3.11.0.tar.gz (406 kB)                                                                                                                                                                                          
0.580   Installing build dependencies: started                                                                                                                                                                                           
4.652   Installing build dependencies: finished with status 'done'                                                                                                                                                                       
4.652   Getting requirements to build wheel: started
4.774   Getting requirements to build wheel: finished with status 'done'
4.775   Preparing metadata (pyproject.toml): started
4.895   Preparing metadata (pyproject.toml): finished with status 'done'
5.014 Collecting numpy>=1.17.3 (from h5py)
5.014   Using cached numpy-2.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (62 kB)
5.021 Using cached numpy-2.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
5.037 Building wheels for collected packages: h5py
5.038   Building wheel for h5py (pyproject.toml): started
5.199   Building wheel for h5py (pyproject.toml): finished with status 'error'
5.202   error: subprocess-exited-with-error
5.202   
5.202   × Building wheel for h5py (pyproject.toml) did not run successfully.
5.202   │ exit code: 1
5.202   ╰─> [75 lines of output]
5.202       running bdist_wheel
5.202       running build
5.202       running build_py
5.202       creating build
5.202       creating build/lib.linux-aarch64-cpython-311
5.202       creating build/lib.linux-aarch64-cpython-311/h5py
5.202       copying h5py/version.py -> build/lib.linux-aarch64-cpython-311/h5py
5.202       copying h5py/__init__.py -> build/lib.linux-aarch64-cpython-311/h5py
5.202       copying h5py/h5py_warnings.py -> build/lib.linux-aarch64-cpython-311/h5py
5.202       copying h5py/ipy_completer.py -> build/lib.linux-aarch64-cpython-311/h5py
5.202       creating build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/group.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/selections.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/dims.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/vds.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/__init__.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/filters.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/files.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/compat.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/datatype.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/dataset.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/base.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/selections2.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       copying h5py/_hl/attrs.py -> build/lib.linux-aarch64-cpython-311/h5py/_hl
5.202       creating build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_completions.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_big_endian_file.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_file2.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5pl.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5t.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5d_direct_chunk.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_dimension_scales.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_file_alignment.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/common.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_dims_dimensionproxy.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_attrs_data.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_file_image.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/conftest.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_datatype.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/__init__.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_base.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5z.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_filters.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_dataset_getitem.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_errors.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_file.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5p.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_attribute_create.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_ros3.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_selections.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5o.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_attrs.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_slicing.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_dataset.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_h5f.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_dataset_swmr.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_dtype.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_objects.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       copying h5py/tests/test_group.py -> build/lib.linux-aarch64-cpython-311/h5py/tests
5.202       creating build/lib.linux-aarch64-cpython-311/h5py/tests/data_files
5.202       copying h5py/tests/data_files/__init__.py -> build/lib.linux-aarch64-cpython-311/h5py/tests/data_files
5.202       creating build/lib.linux-aarch64-cpython-311/h5py/tests/test_vds
5.202       copying h5py/tests/test_vds/test_virtual_source.py -> build/lib.linux-aarch64-cpython-311/h5py/tests/test_vds
5.202       copying h5py/tests/test_vds/test_highlevel_vds.py -> build/lib.linux-aarch64-cpython-311/h5py/tests/test_vds
5.202       copying h5py/tests/test_vds/__init__.py -> build/lib.linux-aarch64-cpython-311/h5py/tests/test_vds
5.202       copying h5py/tests/test_vds/test_lowlevel_vds.py -> build/lib.linux-aarch64-cpython-311/h5py/tests/test_vds
5.202       copying h5py/tests/data_files/vlen_string_dset_utc.h5 -> build/lib.linux-aarch64-cpython-311/h5py/tests/data_files
5.202       copying h5py/tests/data_files/vlen_string_dset.h5 -> build/lib.linux-aarch64-cpython-311/h5py/tests/data_files
5.202       copying h5py/tests/data_files/vlen_string_s390x.h5 -> build/lib.linux-aarch64-cpython-311/h5py/tests/data_files
5.202       running build_ext
5.202       Loading library to get build settings and version: libhdf5.so
5.202       error: Unable to load dependency HDF5, make sure HDF5 is installed properly
5.202       Library dirs checked: []
5.202       error: libhdf5.so: cannot open shared object file: No such file or directory
5.202       [end of output]
5.202   
5.202   note: This error originates from a subprocess, and is likely not a problem with pip.
5.203   ERROR: Failed building wheel for h5py
5.203 Failed to build h5py
5.208 ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (h5py)
------
Dockerfile:17
--------------------
  15 |     
  16 |     # Install h5py with HDF5 support
  17 | >>> RUN pip3 install h5py
  18 |     
  19 |     # Copy function code
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install h5py" did not complete successfully: exit code: 1

我的 Dockerfile

FROM public.ecr.aws/lambda/python:3.11

# Install system dependencies
RUN yum update -y && yum install -y \
    python3-pip \
    pkgconfig \
    gcc \
    gcc-c++ \
    make \
    hdf5-devel \
    hdf5

# Update pip to the latest version
RUN pip3 install --upgrade pip

# Install h5py with HDF5 support
RUN pip3 install h5py

# Copy function code
COPY . ${LAMBDA_TASK_ROOT}

# Install additional dependencies
RUN pip3 install -r requirements.txt

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "lambda_function.lambda_handler" ]

需求.txt

pymongo
python-dotenv
Pillow
boto3
Flask
flask-cors
numpy
deepface

docker 构建命令 -

docker build --no-cache --platform linux/arm64 -t picalive-backend-dev .

我正在使用 Apple M2 来构建此图像。

尽管付出了很多努力,我仍然没有成功为我的 Lambda 函数构建 Docker 镜像。我可以采取哪些额外步骤来解决此问题并确保 h5py 正确安装并具有所需的 HDF5 依赖项?

任何帮助或指导将不胜感激。谢谢!

docker aws-lambda h5py amazon-ecr deepface
1个回答
0
投票

验证图书馆位置:

  • Locate libhdf5.so:在 Docker 映像中,使用命令locate libhdf5.so 查找库文件的实际位置。
  • 检查LD_LIBRARY_PATH:如果locate 显示该文件,请确保LD_LIBRARY_PATH 环境变量包含该文件所在的目录。 您可以使用 echo $LD_LIBRARY_PATH 命令来验证这一点。

如果未自动找到该库,您可能需要在 pip3 install h5py 步骤中提供显式链接:

     RUN pip3 install --no-cache-dir --global-option=build_ext --global-option="-L/usr/lib64" --global-option="-I/usr/include/hdf5/serial" h5py
 

--no-cache-dir 可防止缓存构建出现问题。 - -L/usr/lib64 告诉编译器到哪里寻找库。如果 libhdf5.so 位于其他位置,您可能需要调整路径。 - -I /usr/include/hdf5/serial 告诉编译器在哪里可以找到必要的头文件。

© www.soinside.com 2019 - 2024. All rights reserved.