使用 gcc9 和 glibc2.31 交叉编译 aarch64-linux 的 qt5.15.14 时测试 libdl 失败

Question

1。我现在正在尝试交叉编译 Qt5.15.14 以在arm64板上运行。

• 主机：x86_64 linux Ubuntu 22.04
• 目标：aarch64-linux-gnu
• 工具链：aarch64-none-linux-gnu-gcc 9.2，从ARM网站下载
• Sysroot：ubuntu 20.04 基本映像，安装了 libc6-dev
• 工具链的Glibc：2.30
• Sysroot 的 Glibc：2.31

我的目的是启用“elgfs”，但是“configure”脚本给出了错误信息：

ERROR: Feature 'dlopen' was enabled, but the pre-condition 'config.unix && libs.libdl' failed.

ERROR: Feature 'egl' was enabled, but the pre-condition '(features.opengl || features.openvg) && (features.angle || libs.egl) && (features.dlopen || !config.unix || config.integrity)' failed.

ERROR: Feature 'eglfs' was enabled, but the pre-condition '!config.android && !config.darwin && !config.win32 && !config.wasm && features.egl' failed.

我检查了前置条件，我相信如果 libs.libdl 测试通过，其他测试也会通过。

---

2。我从config.tests/libdl/main.cpp复制测试源文件，内容是

/* Generated by configure */
#include <dlfcn.h>
int main(int argc, char **argv)
{
    (void)argc; (void)argv;
    /* BEGIN TEST: */
    dlclose(dlopen(0, 0));
    dlsym(RTLD_DEFAULT, 0);
    dlerror();
    /* END TEST */
    return 0;
}

并做一些测试：

测试1在没有sysroot的主机上交叉编译

$ARMCC -o a.out  main.cpp -L${TOOLCHAIN_DIR}/aarch64-none-linux-gnu/libc/usr/lib64/ -ldl -fPIC

a.out 已生成。和ldd

    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x000000550183e000)
    libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000005501852000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000005501a37000)
    libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000005501ae2000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000005501b06000)
    /lib/ld-linux-aarch64.so.1 (0x0000005500000000)

Test2 使用 qemu 在目标上编译

g++ -o a.out  main.cpp -ldl -fPIC

a.out 已生成。 ldd 给出

    libdl.so.2 (0x000000550184a000)
    libc.so.6 (0x000000550185e000)
    /lib/ld-linux-aarch64.so.1 (0x0000005500000000)

Test3 在主机上使用 sysroot 进行交叉编译

$ARMCC -o a.o  main.cpp -ldl -fPIC -Wl,-rpath-link,${SYSROOT}/lib/aarch64-linux-gnu -I=/usr/include/aarch64-linux-gnu -L=/lib/aarch64-linux-gnu --sysroot=${SYSROOT} --verbose

日志：

Using built-in specs.
COLLECT_GCC=/home/dq/toolchain/gcc9/bin/aarch64-none-linux-gnu-g++
COLLECT_LTO_WRAPPER=/home/dq/toolchain/gcc9/bin/../libexec/gcc/aarch64-none-linux-gnu/9.2.1/lto-wrapper
Target: aarch64-none-linux-gnu
Configured with: /tmp/dgboter/bbs/rhev-vm2--rhe6x86_64/buildbot/rhe6x86_64--aarch64-none-linux-gnu/build/src/gcc/configure --target=aarch64-none-linux-gnu --prefix= --with-sysroot=/aarch64-none-linux-gnu/libc --with-build-sysroot=/tmp/dgboter/bbs/rhev-vm2--rhe6x86_64/buildbot/rhe6x86_64--aarch64-none-linux-gnu/build/build-aarch64-none-linux-gnu/install//aarch64-none-linux-gnu/libc --with-bugurl=https://bugs.linaro.org/ --enable-gnu-indirect-function --enable-shared --disable-libssp --disable-libmudflap --enable-checking=release --enable-languages=c,c++,fortran --with-gmp=/tmp/dgboter/bbs/rhev-vm2--rhe6x86_64/buildbot/rhe6x86_64--aarch64-none-linux-gnu/build/build-aarch64-none-linux-gnu/host-tools --with-mpfr=/tmp/dgboter/bbs/rhev-vm2--rhe6x86_64/buildbot/rhe6x86_64--aarch64-none-linux-gnu/build/build-aarch64-none-linux-gnu/host-tools --with-mpc=/tmp/dgboter/bbs/rhev-vm2--rhe6x86_64/buildbot/rhe6x86_64--aarch64-none-linux-gnu/build/build-aarch64-none-linux-gnu/host-tools --with-isl=/tmp/dgboter/bbs/rhev-vm2--rhe6x86_64/buildbot/rhe6x86_64--aarch64-none-linux-gnu/build/build-aarch64-none-linux-gnu/host-tools --enable-fix-cortex-a53-843419 --with-pkgversion='GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)'
Thread model: posix
gcc version 9.2.1 20191025 (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 
COLLECT_GCC_OPTIONS='-o' 'a.o' '-fPIC' '-I' '=/usr/include/aarch64-linux-gnu' '-L=/lib/aarch64-linux-gnu' '-v' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'
 /home/dq/toolchain/gcc9/bin/../libexec/gcc/aarch64-none-linux-gnu/9.2.1/cc1plus -quiet -v -I =/usr/include/aarch64-linux-gnu -iprefix /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/ -isysroot /home/dq/sysroot/ubuntu/ -D_GNU_SOURCE main.cpp -quiet -dumpbase main.cpp -mlittle-endian -mabi=lp64 -auxbase main -version -fPIC -o /tmp/ccm4e77A.s
GNU C++14 (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) version 9.2.1 20191025 (aarch64-none-linux-gnu)
        compiled by GNU C version 4.8.1, GMP version 4.3.2, MPFR version 3.1.6, MPC version 1.0.3, isl version isl-0.15-1-g835ea3a-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring duplicate directory "/home/dq/toolchain/gcc9/bin/../lib/gcc/../../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include/c++/9.2.1"
ignoring duplicate directory "/home/dq/toolchain/gcc9/bin/../lib/gcc/../../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include/c++/9.2.1/aarch64-none-linux-gnu"
ignoring duplicate directory "/home/dq/toolchain/gcc9/bin/../lib/gcc/../../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include/c++/9.2.1/backward"
ignoring duplicate directory "/home/dq/toolchain/gcc9/bin/../lib/gcc/../../lib/gcc/aarch64-none-linux-gnu/9.2.1/include"
ignoring duplicate directory "/home/dq/toolchain/gcc9/bin/../lib/gcc/../../lib/gcc/aarch64-none-linux-gnu/9.2.1/include-fixed"
ignoring duplicate directory "/home/dq/toolchain/gcc9/bin/../lib/gcc/../../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/dq/sysroot/ubuntu//usr/include/aarch64-linux-gnu
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include/c++/9.2.1
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include/c++/9.2.1/aarch64-none-linux-gnu
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include/c++/9.2.1/backward
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/include
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/include-fixed
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/include
 /home/dq/sysroot/ubuntu/usr/local/include
 /home/dq/sysroot/ubuntu/usr/include
End of search list.
GNU C++14 (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) version 9.2.1 20191025 (aarch64-none-linux-gnu)
        compiled by GNU C version 4.8.1, GMP version 4.3.2, MPFR version 3.1.6, MPC version 1.0.3, isl version isl-0.15-1-g835ea3a-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 2c1b06ae38fc097703f75fa12d3b7029
COLLECT_GCC_OPTIONS='-o' 'a.o' '-fPIC' '-I' '=/usr/include/aarch64-linux-gnu' '-L=/lib/aarch64-linux-gnu' '-v' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'
 /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/as -v -I =/usr/include/aarch64-linux-gnu -EL -mabi=lp64 -o /tmp/ccVgrV4P.o /tmp/ccm4e77A.s
GNU assembler version 2.33.1 (aarch64-none-linux-gnu) using BFD version (GNU Toolchain for the A-profile Architecture 9.2-2019.12 (arm-9.10)) 2.33.1.20191209
COMPILER_PATH=/home/dq/toolchain/gcc9/bin/../libexec/gcc/aarch64-none-linux-gnu/9.2.1/:/home/dq/toolchain/gcc9/bin/../libexec/gcc/:/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/
LIBRARY_PATH=/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/:/home/dq/toolchain/gcc9/bin/../lib/gcc/:/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/lib/../lib64/:/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/lib/:/home/dq/sysroot/ubuntu/lib/:/home/dq/sysroot/ubuntu/usr/lib/
COLLECT_GCC_OPTIONS='-o' 'a.o' '-fPIC' '-I' '=/usr/include/aarch64-linux-gnu' '-L=/lib/aarch64-linux-gnu' '-v' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'
 /home/dq/toolchain/gcc9/bin/../libexec/gcc/aarch64-none-linux-gnu/9.2.1/collect2 -plugin /home/dq/toolchain/gcc9/bin/../libexec/gcc/aarch64-none-linux-gnu/9.2.1/liblto_plugin.so -plugin-opt=/home/dq/toolchain/gcc9/bin/../libexec/gcc/aarch64-none-linux-gnu/9.2.1/lto-wrapper -plugin-opt=-fresolution=/tmp/cc1mxz8m.res -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc --sysroot=/home/dq/sysroot/ubuntu/ --eh-frame-hdr -dynamic-linker /lib/ld-linux-aarch64.so.1 -X -EL -maarch64linux --fix-cortex-a53-843419 -o a.o /home/dq/sysroot/ubuntu/lib/crt1.o /home/dq/sysroot/ubuntu/lib/crti.o /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/crtbegin.o -L=/lib/aarch64-linux-gnu -L/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1 -L/home/dq/toolchain/gcc9/bin/../lib/gcc -L/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/lib/../lib64 -L/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/lib -L/home/dq/sysroot/ubuntu/lib -L/home/dq/sysroot/ubuntu/usr/lib /tmp/ccVgrV4P.o -ldl -rpath-link /home/dq/sysroot/ubuntu//lib/aarch64-linux-gnu -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/crtend.o /home/dq/sysroot/ubuntu/lib/crtn.o
/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/ld: /tmp/ccVgrV4P.o: in function `main':
main.cpp:(.text+0x18): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/ld: /home/dq/sysroot/ubuntu//lib/aarch64-linux-gnu/libdl.a(dlopen.o): in function `dlopen':
(.text+0x14): undefined reference to `__dlopen'
/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/ld: /home/dq/sysroot/ubuntu//lib/aarch64-linux-gnu/libdl.a(dlclose.o): in function `dlclose':
(.text+0x0): undefined reference to `__dlclose'
/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/ld: /home/dq/sysroot/ubuntu//lib/aarch64-linux-gnu/libdl.a(dlsym.o): in function `dlsym':
(.text+0x14): undefined reference to `__dlsym'
/home/dq/toolchain/gcc9/bin/../lib/gcc/aarch64-none-linux-gnu/9.2.1/../../../../aarch64-none-linux-gnu/bin/ld: /home/dq/sysroot/ubuntu//lib/aarch64-linux-gnu/libdl.a(dlerror.o): in function `dlerror':
(.text+0x0): undefined reference to `__dlerror'
collect2: error: ld returned 1 exit status

3。错误消息显示“__dlopen”未定义，由 libdl.a 引用，因此我转储了 glibc 库以查找哪个库包含此函数并找出答案：

# readelf -s libc.a | grep dlopen
    30: 0000000000000000     0 NOTYPE  GLOBAL HIDDEN   UND __libc_dlopen_mode
    11: 0000000000000000     0 NOTYPE  WEAK   HIDDEN   UND __dlopen
File: libc.a(sdlopen.o)
     5: 0000000000000000   160 FUNC    LOCAL  DEFAULT    1 dlopen_doit
    18: 00000000000000a0   140 FUNC    GLOBAL HIDDEN     1 __dlopen
    44: 0000000000000000     0 NOTYPE  GLOBAL HIDDEN   UND __dlopen
    17: 0000000000000000     0 NOTYPE  GLOBAL HIDDEN   UND __libc_dlopen_mode
    60: 0000000000000000     0 NOTYPE  GLOBAL HIDDEN   UND __libc_dlopen_mode
     3: 0000000000000000    80 FUNC    LOCAL  DEFAULT    1 do_dlopen
    48: 00000000000004f8   144 FUNC    GLOBAL HIDDEN     1 __libc_dlopen_mode

libc.a 位于 ${SYSROOT}/usr/lib/aarch64-linux-gnu，使用 -rpath-link 包含路径，链接器自动包含选项“-lc”

4 当转储测试1和测试2中生成的a.out的符号时，仅包含dlopen@GLIBC_2.17:

    9: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND dlopen@GLIBC_2.17 (2)
   103: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND dlopen@@GLIBC_2.17

测试4：在主机上使用sysroot进行交叉编译

$ARMCC -o a.o  main.cpp -lc -ldl-2.31 -fPIC -Wl,-rpath-link,${SYSROOT}/lib/aarch64-linux-gnu -I=/usr/include/aarch64-linux-gnu -L=/lib/aarch64-linux-gnu --sysroot=${SYSROOT}

sysroot 上的 libdl.so 是指向“/lib/aarch64-linux-gnu/libdl-2.31.so”的符号链接，因此我将“-ldl”替换为“-ldl-2.31”，并且成功了。

所以。我认为是链接器找不到真正的库导致了问题

问题：

1. 链接器如何处理sysroot中的符号链接？ --sysroot 会影响链接器的行为吗？
2. 如何正确配置？

这就是我现在所找到的一切，我现在不能再进一步了。有人可以帮忙吗？谢谢！！

Answer 1

现代 cmake 交叉编译到 aarch64-with-sysroots

按照这个链接，我选择第四种方法来解决这个问题，从而将libdl.so的符号链接更改为相对路径。