Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spu bazel编译问题 #430

Closed
walkovernamtso opened this issue Dec 8, 2023 · 5 comments
Closed

spu bazel编译问题 #430

walkovernamtso opened this issue Dec 8, 2023 · 5 comments
Assignees
@anakinxc

Comments

@walkovernamtso
Copy link

walkovernamtso commented Dec 8, 2023 •
edited
Loading

Issue Type

Build/Install

Modules Involved

PSI

Have you reproduced the bug with SPU HEAD?

Yes

Have you searched existing issues?

Yes

SPU Version

spu0.6

OS Platform and Distribution

centos7

Python Version

Python 3.8.16

Compiler Version

gcc11

Current Behavior?

bazel build //examples/cpp:simple_psi

报错（以前编译过，是可以成功的）

Standalone code to reproduce the issue

(secretflow) [lishihao@compute spu-0.6.0b0]$ bazel build //examples/cpp:simple_psi
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/xla/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'tf_runtime' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/xla/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'llvm-raw' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'pybind11_bazel' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_github_grpc_grpc' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'zlib' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'rules_python' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'pybind11' because it already exists.
DEBUG: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'upb' because it already exists.
INFO: Analyzed target //examples/cpp:simple_psi (150 packages loaded, 11697 targets configured).
INFO: Found 1 target...
ERROR: /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/external/zlib/BUILD.bazel:24:20: Foreign Cc - CMake: Building zlib [for tool] failed: (Exit 1): bash failed: error executing command (from target @zlib//:zlib) /bin/bash -c bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/wrapper_build_script.sh

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
rules_foreign_cc: Build failed!
rules_foreign_cc: Keeping temp build directory and dependencies directory for debug.
rules_foreign_cc: Please note that the directories inside a sandbox are still cleaned unless you specify --sandbox_debug Bazel command line flag.
rules_foreign_cc: Printing build logs:
_____ BEGIN BUILD LOGS _____

Bazel external C/C++ Rules. Building library zlib

Environment:______________
TMPDIR=/tmp
BUILD_WRAPPER_SCRIPT=bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/wrapper_build_script.sh
EXT_BUILD_ROOT=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib
INSTALLDIR=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib
PATH=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib:/home/lishihao/anaconda3/envs/secretflow/bin:/home/lishihao/anaconda3/bin:/usr/local/java/jdk1.8.0_271/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/local/go/bin:/usr/local/arm/gcc-linaro-13.0.0-2022.11-x86_64_arm-linux-gnueabihf/bin:/usr/local/maven/apache-maven-3.9.5/bin::/opt/rh/devtoolset-11/root/usr/bin:/sbin/mysqld:/usr/local/go/bin:/home/lishihao/cmake/bin::/opt/rh/devtoolset-11/root/usr/bin:/home/lishihao/.local/bin:/home/lishihao/bin
BUILD_TMPDIR=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib.build_tmpdir
PWD=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib
EXT_BUILD_DEPS=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib.ext_build_deps
SHLVL=3
BUILD_LOG=bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/CMake.log
BUILD_SCRIPT=bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/build_script.sh
_=/usr/bin/env

  • cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_TOOLCHAIN_FILE=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib.build_tmpdir/crosstool_bazel.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib -DCMAKE_PREFIX_PATH=/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib.ext_build_deps -DCMAKE_RANLIB= -DCMAKE_MAKE_PROGRAM=ninja -G Ninja /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/external/zlib
    CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
    Compatibility with CMake < 3.5 will be removed from a future version of
    CMake.

    Update the VERSION argument value or use a ... suffix to tell
    CMake that the project does not need compatibility with older versions.

-- The C compiler identification is GNU 11.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc - broken
CMake Error at /home/lishihao/anaconda3/envs/secretflow/lib/python3.8/site-packages/cmake/data/share/cmake-3.27/Modules/CMakeTestCCompiler.cmake:67 (message):
The C compiler

"/usr/bin/gcc"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: '/home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib.build_tmpdir/CMakeFiles/CMakeScratch/TryCompile-DVacC5'

Run Build Command(s): ninja -v cmTC_aeecb
[1/2] /usr/bin/gcc   -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -D_FORTIFY_SOURCE=1 -DNDEBUG -ffunction-sections -fdata-sections -fno-canonical-system-headers -Wno-builtin-macro-redefined -D__DATE__=\"redacted\" -D__TIMESTAMP__=\"redacted\" -D__TIME__=\"redacted\"  -fPIE -o CMakeFiles/cmTC_aeecb.dir/testCCompiler.c.o -c /home/lishihao/.cache/bazel/_bazel_lishihao/ddec707999975aa4e40d2a2fe63d9276/sandbox/processwrapper-sandbox/1/execroot/spulib/bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib.build_tmpdir/CMakeFiles/CMakeScratch/TryCompile-DVacC5/testCCompiler.c
[2/2] : && /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -D_FORTIFY_SOURCE=1 -DNDEBUG -ffunction-sections -fdata-sections -fno-canonical-system-headers -Wno-builtin-macro-redefined -D__DATE__=\"redacted\" -D__TIMESTAMP__=\"redacted\" -D__TIME__=\"redacted\" -fuse-ld=gold -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/usr/bin -pass-exit-codes -Wl,--gc-sections -lstdc++ -lm   -rdynamic CMakeFiles/cmTC_aeecb.dir/testCCompiler.c.o -o cmTC_aeecb   && :
FAILED: cmTC_aeecb 
: && /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -D_FORTIFY_SOURCE=1 -DNDEBUG -ffunction-sections -fdata-sections -fno-canonical-system-headers -Wno-builtin-macro-redefined -D__DATE__=\"redacted\" -D__TIMESTAMP__=\"redacted\" -D__TIME__=\"redacted\" -fuse-ld=gold -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/usr/bin -pass-exit-codes -Wl,--gc-sections -lstdc++ -lm   -rdynamic CMakeFiles/cmTC_aeecb.dir/testCCompiler.c.o -o cmTC_aeecb   && :
/usr/bin/ld.gold: --push-state: unknown option
/usr/bin/ld.gold: use the --help option for usage information
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:4 (project)

-- Configuring incomplete, errors occurred!
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-opt-exec-50AE0418/bin/external/zlib/zlib_foreign_cc/CMake.log

Target //examples/cpp:simple_psi failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 5.734s, Critical Path: 0.92s
INFO: 9 processes: 9 internal.
FAILED: Build did NOT complete successfully
@anakinxc
Copy link
Collaborator

anakinxc commented Dec 8, 2023

Hi @daydayuphere

麻烦安装一下 lld 然后再尝试一下，我们不推荐用 gold linker

@walkovernamtso
Copy link
Author

不懂这个lld，怎么安装呢

@anakinxc
Copy link
Collaborator

anakinxc commented Dec 8, 2023

centos7 可以参考这里

@walkovernamtso
Copy link
Author

walkovernamtso commented Dec 8, 2023 •
edited
Loading

我直接用脚本这个打包，他编译正常 python3 setup.py bdist_wheel
但是直接用命令bazel build //examples/cpp:simple_psi，出错

@walkovernamtso
Copy link
Author

安装lld没有成功。
尝试了下升级ld.gold版本，升级到1.16解决了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anakinxc anakinxc

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@anakinxc @walkovernamtso