Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Enable cross-language ThinLTO #4953

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

str4d
Copy link
Contributor

@str4d str4d commented Jan 20, 2021

Causes warnings at link time due to C++ and Rust using different target
strings. These warnings do not break CI, because while we require -Werror
to build on Linux, that only affects compilation, not linking.

See rust-lang/rust#33147 for more details.

Closes #6719.

@str4d str4d added the A-build Area: Build system label Jan 20, 2021
@str4d
Copy link
Contributor Author

str4d commented Jan 20, 2021

@zkbot try

@zkbot
Copy link
Contributor

zkbot commented Jan 20, 2021

⌛ Trying commit 12ff63e with merge c529408...

zkbot added a commit that referenced this pull request Jan 20, 2021
build: Enable cross-language ThinLTO

Causes warnings at link time due to C++ and Rust using different target
strings. These warnings do not break CI, because while we require `-Werror`
to build on Linux, that only affects compilation, not linking.

See rust-lang/rust#33147 for more details.
@zkbot
Copy link
Contributor

zkbot commented Jan 20, 2021

☀️ Test successful - pr-try
State: approved= try=True

daira
daira previously approved these changes Jan 21, 2021
Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK. The target triple warning is not a blocker to merging.

Copy link
Contributor

@rex4539 rex4539 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NACK 12ff63e

  CXXLD    zcashd
  CXXLD    zcash-cli
  CXXLD    zcash-tx
  CXXLD    bench/bench_bitcoin
  CXXLD    test/test_bitcoin
  CXXLD    zcash-gtest
error: can't create module summary index for buffer: Invalid summary version 9. Version should be in the range [1-8].
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
make[2]: *** [zcash-cli] Error 254
make[2]: *** Waiting for unfinished jobs....
error: can't create module summary index for buffer: Invalid summary version 9. Version should be in the range [1-8].
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
make[2]: *** [zcash-tx] Error 254
error: can't create module summary index for buffer: Invalid summary version 9. Version should be in the range [1-8].
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
make[2]: *** [bench/bench_bitcoin] Error 254
error: can't create module summary index for buffer: Invalid summary version 9. Version should be in the range [1-8].
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
make[2]: *** [zcashd] Error 254
error: can't create module summary index for buffer: Invalid summary version 9. Version should be in the range [1-8].
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
make[2]: *** [test/test_bitcoin] Error 254
error: can't create module summary index for buffer: Invalid summary version 9. Version should be in the range [1-8].
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
make[2]: *** [zcash-gtest] Error 254
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

@mdr0id mdr0id added the safe-to-build Used to send PR to prod CI environment label Feb 1, 2021
@nuttycom nuttycom marked this pull request as draft February 2, 2021 14:05
@str4d
Copy link
Contributor Author

str4d commented Feb 2, 2021

In addition to the above issue, this PR appears to slow down the test suites. We need to spend more time figuring out what ThinLTO is actually doing to the code.

@str4d str4d force-pushed the linker-plugin-lto branch from 12ff63e to e3c0417 Compare April 1, 2021 23:47
@str4d
Copy link
Contributor Author

str4d commented Apr 3, 2021

@zkbot try

@zkbot
Copy link
Contributor

zkbot commented Apr 3, 2021

⌛ Trying commit e3c0417 with merge 9910e3c...

zkbot added a commit that referenced this pull request Apr 3, 2021
build: Enable cross-language ThinLTO

Causes warnings at link time due to C++ and Rust using different target
strings. These warnings do not break CI, because while we require `-Werror`
to build on Linux, that only affects compilation, not linking.

See rust-lang/rust#33147 for more details.
@zkbot
Copy link
Contributor

zkbot commented Apr 3, 2021

💥 Test timed out

@nuttycom nuttycom removed the safe-to-build Used to send PR to prod CI environment label Jul 18, 2022
@str4d str4d force-pushed the linker-plugin-lto branch from e3c0417 to 35ba628 Compare February 22, 2023 18:41
@str4d
Copy link
Contributor Author

str4d commented Feb 22, 2023

Rebased on current master, which required making a few changes to the PR:

  • We now set -flto=thin in CFLAGS and LDFLAGS instead of appending to CC, as the latter breaks parts of CMake.
  • We pass -Clinker=$(CLANG) -Clink-arg=-fuse-ld=lld in RUSTFLAGS so that the Rust binaries we build also use cross-language ThinLTO.

@str4d str4d added the safe-to-build Used to send PR to prod CI environment label Feb 22, 2023
@ECC-CI ECC-CI removed safe-to-build Used to send PR to prod CI environment labels Feb 22, 2023
@str4d str4d force-pushed the linker-plugin-lto branch from 35ba628 to 42a7db2 Compare June 14, 2023 21:10
@str4d
Copy link
Contributor Author

str4d commented Jun 14, 2023

Rebased on current master to fix merge conflicts.

Causes warnings at link time due to C++ and Rust using different target
strings. See rust-lang/rust#33147 for more
details.
@daira daira added I-performance Problems and improvements with respect to performance L-rust Involves Rust code. A-rust-ffi Area: The Rust FFI in the librustzcash library. labels Jun 16, 2023
@str4d str4d force-pushed the linker-plugin-lto branch from 42a7db2 to 0a14b78 Compare June 16, 2023 02:55
@str4d
Copy link
Contributor Author

str4d commented Jun 16, 2023

Rebased on current master to get fix for ARM64 CI builder.

@@ -99,6 +99,7 @@ ZC_REQUIRE_TOOL(AR, ar)
ZC_REQUIRE_TOOL(RANLIB, ranlib)
ZC_REQUIRE_TOOL(STRIP, strip)
ZC_REQUIRE_PROG([GIT], [git])
ZC_REQUIRE_PROG(CLANG, clang)
Copy link
Contributor

@daira daira Jun 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using the installed clang, rather than the pinned clang built under depends?

(I know why we aren't on macOS, but that should be the only exception.)

@daira
Copy link
Contributor

daira commented Jun 16, 2023

We should only do this if we have strong evidence of a performance improvement. Otherwise, it's just adding complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-build Area: Build system A-rust-ffi Area: The Rust FFI in the librustzcash library. I-performance Problems and improvements with respect to performance L-rust Involves Rust code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable cross-language LTO
7 participants