llvm-11 and friends fails with lto

[hedmo]

llvm-11 has been released and it fails with. affected packages :
` / # cat /etc/portage/package.cflags/ltolocalworkarounds.conf

BEGIN: LTO workarounds

sys-devel/llvm FLAGS-=-flto
sys-devel/clang FLAGS-=-flto
sys-libs/libomp FLAGS-=-flto

END: LTO not recommended

/ #
`

[perfect7gentleman]

Build LLVM/Clang with LLVM/Clang.

[hedmo]

Build LLVM/Clang with LLVM/Clang.

what do youn mean?

[elsandosgrande]

Build Clang 11 with Clang 10, same with LLVM.

[perfect7gentleman]

++ all LLVM tools and libs

[AnonymousRetard]

I have successfully built both clang:11 and libomp-11.0.0 with -flto. It's only llvm:11 that fails.
It seems to build a broken llvm-tablegen binary which then crashes during the build when it tries to use it to generate files needed to continue building.

I tried switching compiler to clang like this:
/etc/portage/env/clang:
CC="clang"
CXX="clang++"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"

/etc/portage/package.env:
sys-devel/llvm clang

Then it builds but clang doesn't recognize any of these flags:
-fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fno-semantic-interposition -flto=4 -fuse-linker-plugin

So then it builds without lto.
As a final test I tried using this instead:
/etc/portage/env/clang-lto:
CC="clang"
CXX="clang++"
CFLAGS="${CFLAGS} -flto"
CXXFLAGS="${CXXFLAGS} -flto"
LDFLAGS="-Wl,-O3 -Wl,--as-needed"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"

Which then tries to build llvm:11 with clang and -flto but then the build fails in the same way (it builds a broken llvm-tablegen which crashes when launched).

[khuei]

I got them to successfully build with LTO by disabling --as-needed for llvm and clang

[Ninpo]

I got them to successfully build with LTO by disabling --as-needed for llvm and clang

Confirm this worked for me and building libomp/llvm/clang-11 with clang-10 allowed libomp to build with LTO.

To whom should a bug report be sent for the no-as-needed switch with llvm/clang?

[AnonymousRetard]

I can also confirm that building llvm:11 with all flags from this overlay:
CFLAGS="-march=znver1 -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -flto=4 -fuse-linker-plugin -pipe -fomit-frame-pointer -fno-finite-math-only -Wl,-O1"
works as long as I remove --as-needed from LDFLAGS. But not if I try to build llvm:11 with clang:11, it only works with GCC 10.2.0.
I could also get it to build with clang:11 if I disable LTO but removing --as-needed only works for GCC. With clang:11 it gets me a lot further but it still fails in the final linking step. I have not tried clang:10.

Additionally I never had any problems with building clang, only llvm. As for libomp I have the "cuda" USE-flag enabled for that package which forces me to use <= GCC 9.

[InBetweenNames]

I went ahead and disabled LTO on these packages as I'm strapped for time at the moment to deep dive and figure out where they are breaking. Normally LLVM builds fine with LTO, so this is surprising.

[leandrolnh]

Maybe you can try -flto=thin instead, my setup works fine with this:

/etc/portage/env/compiler-clang.conf:
CC="clang"
CXX="clang++"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"
CFLAGS="-march=native -O3 -pipe -flto=thin"
CXXFLAGS="${CFLAGS} -stdlib=libc++"
LDFLAGS="-Wl,-O2 -Wl,--as-needed -fuse-ld=lld -stdlib=libc++ -rtlib=compiler-rt -unwindlib=libunwind -flto=thin -Wl,--thinlto-jobs=8"

/etc/portage/package.env:
sys-devel/llvm-common compiler-clang.conf
sys-devel/llvm compiler-clang.conf
sys-devel/llvmgold compiler-clang.conf
sys-devel/lld compiler-clang.conf
sys-devel/clang compiler-clang.conf
sys-devel/clang-common compiler-clang.conf
sys-libs/compiler-rt-sanitizers compiler-clang.conf
sys-libs/compiler-rt compiler-clang.conf
sys-devel/clang-runtime compiler-clang.conf
sys-libs/libcxx compiler-clang.conf
sys-libs/libcxxabi compiler-clang.conf
sys-libs/libomp compiler-clang.conf
sys-libs/llvm-libunwind compiler-clang.conf
dev-lang/rust compiler-clang.conf
media-libs/mesa compiler-clang.conf

[hedmo]

in my opinion.i dont think it is a good idea to mixing compilers.well on local basics it is okay but not global like here
in lto-overlay.I went ahead and disabled LTO on these packages as i have more package in my system that has a mix off clang and gcc when it is compiling.BTW i am keeping :
sys-devel/llvm *FLAGS-="${IPAPTA}"
to just in case.

[barolo]

they compile fine with clang --flto=full and -fwhole-program-vtables

[perfect7gentleman]

@barolo , how long will it take to build llvm and clang with -flto=full and -fwhole-program-vtables against -flto=thin ?

[barolo]

@perfect7gentleman genlop says that it took 53 min for llvm and 54 for clang [ multilib ], on my 4 core APU

[killbox998]

I'm still getting build failures even with lto disabled

FAILED: include/llvm/IR/IntrinsicsPowerPC.h cd /var/tmp/portage/sys-devel/llvm-11.0.0/work/llvm-11.0.0_build-abi_x86_32.x86 && /var/tmp/portage/sys-devel/llvm-11.0.0/work/llvm-11.0.0_build-abi_x86_32.x86/bin/llvm-tblgen -gen-intrinsic-enums -intrinsic-prefix=ppc -I /var/tmp/portage/sys-devel/llvm-11.0.0/work/llvm/include/llvm/IR -I /usr/include/libxml2 -I /var/tmp/portage/sys-devel/llvm-11.0.0/work/llvm-11.0.0_build-abi_x86_32.x86/include -I /var/tmp/portage/sys-devel/llvm-11.0.0/work/llvm/include /var/tmp/portage/sys-devel/llvm-11.0.0/work/llvm/include/llvm/IR/Intrinsics.td --write-if-changed -o include/llvm/IR/IntrinsicsPowerPC.h -d include/llvm/IR/IntrinsicsPowerPC.h.d terminate called after throwing an instance of 'std::system_error' what(): Unknown error -1

https://dpaste.com/FFGJR9WXS

[perfect7gentleman]

so, you wanna say that -flto=1 means it's disabled?

[khuei]

/etc/portage/package.cflags/ltoworkarounds.conf -> sys-devel/llvm: FLAGS-=-flto* # Issue #619 temporarily disabled for now due to build errors

I think it's because it's missing the prefix *, so it should be *FLAGS-=-flto* rather than FLAGS-=-flto*

[killbox998]

Yeah, That was the issue

[Hello71]

llvm requires -Wl,--no-as-needed due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55394#c7, fixed in gcc 11. it causes terminate called after throwing an instance of 'std::system_error' what(): Unknown error -1.

[alanswanson]

Testing -Wl,--no-as-needed it's only required for tablegen with sys-devel/llvm and sys-devel/clang and not sys-libs/libomp (when using LTO). Only build difference was libLLVM-11.so being linked to librt.so.

[hedmo]

llvm requires -Wl,--no-as-needed due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55394#c7, fixed in gcc 11. it causes terminate called after throwing an instance of 'std::system_error' what(): Unknown error -1.

okej . I Will remove my overides and check .

[hedmo]

i can confirm that llvm-11 and friends compiles with full LTO on gcc-11

[laomaiweng]

FWIW, this issue is tracked in https://bugs.gentoo.org/749162, and there are patches there available for both LLVM and Clang that have been reported to fix LTO builds with GCC<11.

[Tomte-Fan]

With llvm 16.0.4 I have to turn off IPAPTA or it will fail to build with a stack trace.