fix(IDX): make replica build more deterministic #2441
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This includes a couple of fixes to make the replica build more deterministic.
This in theory allows anyone building
//rs/replica
with Bazel in the Docker image to get a bit-for-bit reproducible replica executable, and this should also improve Bazel cache hit rates.This fixes for the following issues:
Non-reproducible
jemalloc
build: thetikv-jemalloc-sys
crate vendorsjemalloc
and builds it as part ofbuild.rs
. Unfortunately that build in not deterministic. To make it more deterministic we buildjemalloc
separately, which also speeds up rebuilds as the C code does not need to be rebuilt when rust versions change. We only enable this in Linux; this also includes a patch to support this inrules_rust
: Fix annotations' crate.select for lists bazelbuild/rules_rust#2981Non-reproducible
rules_rust
build log: this includes a backport of a fix that disables build logs inrules_rust
by default (backported because our build is not compatible with the latestrules_rust
) build.rs stdout creates non-deterministic issues bazelbuild/rules_rust#2974Non-reproducible make & pkgconfig toolchains: some toolchains packaged by
rules_foreign_cc
cause build determinism issues so instead we use the ones installed in the container and remove build logs: Make and pkg-config toolchains are not reproducible bazel-contrib/rules_foreign_cc#1313Non-reproducible obj file generation in cc-rs: the
cc-rs
crate used in many C builds, including the (ASM) build ofring
's crypto bits, generates object files that include the Bazel sandbox full path: Object file hash prefix creates determinism issues rust-lang/cc-rs#1271Non-reproducible codegen:
cranelift-isle
andcranelift-codegen-meta
include references to source files as absolute paths that include the Bazel sandbox path: Avoid printing full file paths in cranelift generated code bytecodealliance/wasmtime#9553