tools/docker/syzbot: replace clang, LLVM with stable kernel archive verson #5385
Labels
Comments
novitoll
added a commit
to novitoll/syzkaller
that referenced
this issue
Oct 27, 2024
Replace clang, llvm apt packages in syzbot Dockerfile
with the stable kernel version. Reduce Dockerfile code (less RUN layers).
Use stable LLVM version that Linux kernel doc suggests [1].
These 2 binaries are missing in the archive: "clang-format, clang-tidy",
hence install them as the current apt package.
This patch increases syzbot docker image size by 230MB, see the
comparison between "syz_master" (current master branch built image),
and with the applied patch "syz_master_diff".
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
syz_master_diff latest 40b209ef582e 6 hours ago 5.54GB
syz_master latest 292faad84de9 13 hours ago 5.31GB
The archive is 123M size and I've built the image 3 times with removing
the images as the fresh install, and curl downloading for my 500Mbit/s
Ethernet downlink took:
1st build: 0 hour 0 min 50 sec
2nd build: 1 hour 3 min 5 sec
3rd build: 0 hour 1 min 5 sec
Hereby, I'm concerned if the archive can be mirrored from
"mirrors.edge.kernel.org" to more stable/reliable storage with more
distributed CDN like dl.google.com, or it's something local with my downlink.
Per the 3rd build, the patch also speeds up (~1 min faster) the docker image
build time:
...
Step 7/27 : RUN curl -L <redacted>
---> Running in ae2853dfc0fc
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 123M 100 123M 0 0 1916k 0 0:01:05 0:01:05 --:--:-- 2513k
...
Successfully built 387ae7e06dfa
Successfully tagged syz_master_diff:latest
real 6m23.013s
user 0m0.145s
sys 0m0.094s
and the original "syz_master" fresh build:
...
Successfully built 843592c9ce1b
Successfully tagged syz_master:latest
real 7m18.333s
user 0m0.152s
sys 0m0.090s
Verified the binaries' versions in the container as well:
$ docker run --rm --name syz_master_diff -it syz_master_diff bash
/🤖 for e in {clang,clang++,clang-tidy,clang-format,ld.lld,llvm-nm,\
> llvm-ar,llvm-objcopy,llvm-objdump,llvm-addr2line,llvm-readelf,\
> llvm-strip}; do echo; $e --version; done
Debian clang version 15.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Debian clang version 15.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Debian LLVM version 15.0.6
Optimized build.
Default target: x86_64-pc-linux-gnu
Host CPU: skylake
Debian clang-format version 15.0.6
ClangBuiltLinux LLD 15.0.7 (compatible with GNU linkers)
llvm-nm, compatible with GNU nm
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
llvm-objcopy, compatible with GNU objcopy
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
Registered Targets:
aarch64 - AArch64 (little endian)
aarch64_32 - AArch64 (little endian ILP32)
aarch64_be - AArch64 (big endian)
arm - ARM
arm64 - ARM64 (little endian)
arm64_32 - ARM64 (little endian ILP32)
armeb - ARM (big endian)
bpf - BPF (host endian)
bpfeb - BPF (big endian)
bpfel - BPF (little endian)
hexagon - Hexagon
mips - MIPS (32-bit big endian)
mips64 - MIPS (64-bit big endian)
mips64el - MIPS (64-bit little endian)
mipsel - MIPS (32-bit little endian)
ppc32 - PowerPC 32
ppc32le - PowerPC 32 LE
ppc64 - PowerPC 64
ppc64le - PowerPC 64 LE
riscv32 - 32-bit RISC-V
riscv64 - 64-bit RISC-V
systemz - SystemZ
thumb - Thumb
thumbeb - Thumb (big endian)
x86 - 32-bit X86: Pentium-Pro and above
x86-64 - 64-bit X86: EM64T and AMD64
llvm-addr2line
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
llvm-strip, compatible with GNU strip
LLVM (http://llvm.org/):
LLVM version 15.0.7
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
[1] https://mirrors.edge.kernel.org/pub/tools/llvm/
Fixes: google#5385
novitoll
added a commit
to novitoll/syzkaller
that referenced
this issue
Oct 27, 2024
Replace clang, llvm apt packages in syzbot Dockerfile with the stable kernel version. Reduce Dockerfile code (less RUN layers). Use stable LLVM version that Linux kernel doc suggests [1]. [1] https://mirrors.edge.kernel.org/pub/tools/llvm/ These 2 binaries are missing in the archive: "clang-format, clang-tidy", hence install them as the current apt package. This patch increases syzbot docker image size by 230MB, see the comparison between "syz_master" (current master branch built image), and with the applied patch "syz_master_diff". $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE syz_master_diff latest 40b209ef582e 6 hours ago 5.54GB syz_master latest 292faad84de9 13 hours ago 5.31GB The archive is 123M size and I've built the image 3 times with removing the images as the fresh install, and curl downloading for my 500Mbit/s Ethernet downlink took: 1st build: 0 hour 0 min 50 sec 2nd build: 1 hour 3 min 5 sec 3rd build: 0 hour 1 min 5 sec Hereby, I'm concerned if the archive can be mirrored from "mirrors.edge.kernel.org" to more stable/reliable storage with more distributed CDN like dl.google.com, or it's something local with my downlink. Per the 3rd build, the patch also speeds up (~1 min faster) the docker image build time: ... Step 7/27 : RUN curl -L <redacted> ---> Running in ae2853dfc0fc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 123M 100 123M 0 0 1916k 0 0:01:05 0:01:05 --:--:-- 2513k ... Successfully built 387ae7e06dfa Successfully tagged syz_master_diff:latest real 6m23.013s user 0m0.145s sys 0m0.094s and the original "syz_master" fresh build: ... Successfully built 843592c9ce1b Successfully tagged syz_master:latest real 7m18.333s user 0m0.152s sys 0m0.090s Verified the binaries' versions in the container as well: $ docker run --rm --name syz_master_diff -it syz_master_diff bash /🤖 for e in {clang,clang++,clang-tidy,clang-format,ld.lld,llvm-nm,\ > llvm-ar,llvm-objcopy,llvm-objdump,llvm-addr2line,llvm-readelf,\ > llvm-strip}; do echo; $e --version; done Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian LLVM version 15.0.6 Optimized build. Default target: x86_64-pc-linux-gnu Host CPU: skylake Debian clang-format version 15.0.6 ClangBuiltLinux LLD 15.0.7 (compatible with GNU linkers) llvm-nm, compatible with GNU nm LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-objcopy, compatible with GNU objcopy LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Registered Targets: aarch64 - AArch64 (little endian) aarch64_32 - AArch64 (little endian ILP32) aarch64_be - AArch64 (big endian) arm - ARM arm64 - ARM64 (little endian) arm64_32 - ARM64 (little endian ILP32) armeb - ARM (big endian) bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) hexagon - Hexagon mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian) ppc32 - PowerPC 32 ppc32le - PowerPC 32 LE ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE riscv32 - 32-bit RISC-V riscv64 - 64-bit RISC-V systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 llvm-addr2line LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-strip, compatible with GNU strip LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Fixes: google#5385
novitoll
added a commit
to novitoll/syzkaller
that referenced
this issue
Oct 30, 2024
Replace clang, llvm apt packages in syzbot Dockerfile with the stable kernel version. Reduce Dockerfile code (less RUN layers). Use stable LLVM version that Linux kernel doc suggests [1]. [1] https://mirrors.edge.kernel.org/pub/tools/llvm/ These 2 binaries are missing in the archive: "clang-format, clang-tidy", hence install them as the current apt package. This patch increases syzbot docker image size by 230MB, see the comparison between "syz_master" (current master branch built image), and with the applied patch "syz_master_diff". $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE syz_master_diff latest 40b209ef582e 6 hours ago 5.54GB syz_master latest 292faad84de9 13 hours ago 5.31GB The archive is 123M size and I've built the image 3 times with removing the images as the fresh install, and curl downloading for my 500Mbit/s Ethernet downlink took: 1st build: 0 hour 0 min 50 sec 2nd build: 1 hour 3 min 5 sec 3rd build: 0 hour 1 min 5 sec Hereby, I'm concerned if the archive can be mirrored from "mirrors.edge.kernel.org" to more stable/reliable storage with more distributed CDN like dl.google.com, or it's something local with my downlink. Per the 3rd build, the patch also speeds up (~1 min faster) the docker image build time: ... Step 7/27 : RUN curl -L <redacted> ---> Running in ae2853dfc0fc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 123M 100 123M 0 0 1916k 0 0:01:05 0:01:05 --:--:-- 2513k ... Successfully built 387ae7e06dfa Successfully tagged syz_master_diff:latest real 6m23.013s user 0m0.145s sys 0m0.094s and the original "syz_master" fresh build: ... Successfully built 843592c9ce1b Successfully tagged syz_master:latest real 7m18.333s user 0m0.152s sys 0m0.090s Verified the binaries' versions in the container as well: $ docker run --rm --name syz_master_diff -it syz_master_diff bash /🤖 for e in {clang,clang++,clang-tidy,clang-format,ld.lld,llvm-nm,\ > llvm-ar,llvm-objcopy,llvm-objdump,llvm-addr2line,llvm-readelf,\ > llvm-strip}; do echo; $e --version; done Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian LLVM version 15.0.6 Optimized build. Default target: x86_64-pc-linux-gnu Host CPU: skylake Debian clang-format version 15.0.6 ClangBuiltLinux LLD 15.0.7 (compatible with GNU linkers) llvm-nm, compatible with GNU nm LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-objcopy, compatible with GNU objcopy LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Registered Targets: aarch64 - AArch64 (little endian) aarch64_32 - AArch64 (little endian ILP32) aarch64_be - AArch64 (big endian) arm - ARM arm64 - ARM64 (little endian) arm64_32 - ARM64 (little endian ILP32) armeb - ARM (big endian) bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) hexagon - Hexagon mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian) ppc32 - PowerPC 32 ppc32le - PowerPC 32 LE ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE riscv32 - 32-bit RISC-V riscv64 - 64-bit RISC-V systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 llvm-addr2line LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-strip, compatible with GNU strip LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Fixes: google#5385
novitoll
added a commit
to novitoll/syzkaller
that referenced
this issue
Nov 4, 2024
Replace clang, llvm apt packages in syzbot Dockerfile with the stable kernel version. Reduce Dockerfile code (less RUN layers). Use stable LLVM version that Linux kernel doc suggests [1]. [1] https://mirrors.edge.kernel.org/pub/tools/llvm/ These 2 binaries are missing in the archive: "clang-format, clang-tidy", hence install them as the current apt package. This patch increases syzbot docker image size by 230MB, see the comparison between "syz_master" (current master branch built image), and with the applied patch "syz_master_diff". $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE syz_master_diff latest 40b209ef582e 6 hours ago 5.54GB syz_master latest 292faad84de9 13 hours ago 5.31GB The archive is 123M size and I've built the image 3 times with removing the images as the fresh install, and curl downloading for my 500Mbit/s Ethernet downlink took: 1st build: 0 hour 0 min 50 sec 2nd build: 1 hour 3 min 5 sec 3rd build: 0 hour 1 min 5 sec Hereby, I'm concerned if the archive can be mirrored from "mirrors.edge.kernel.org" to more stable/reliable storage with more distributed CDN like dl.google.com, or it's something local with my downlink. Per the 3rd build, the patch also speeds up (~1 min faster) the docker image build time: ... Step 7/27 : RUN curl -L <redacted> ---> Running in ae2853dfc0fc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 123M 100 123M 0 0 1916k 0 0:01:05 0:01:05 --:--:-- 2513k ... Successfully built 387ae7e06dfa Successfully tagged syz_master_diff:latest real 6m23.013s user 0m0.145s sys 0m0.094s and the original "syz_master" fresh build: ... Successfully built 843592c9ce1b Successfully tagged syz_master:latest real 7m18.333s user 0m0.152s sys 0m0.090s Verified the binaries' versions in the container as well: $ docker run --rm --name syz_master_diff -it syz_master_diff bash /🤖 for e in {clang,clang++,clang-tidy,clang-format,ld.lld,llvm-nm,\ > llvm-ar,llvm-objcopy,llvm-objdump,llvm-addr2line,llvm-readelf,\ > llvm-strip}; do echo; $e --version; done Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian LLVM version 15.0.6 Optimized build. Default target: x86_64-pc-linux-gnu Host CPU: skylake Debian clang-format version 15.0.6 ClangBuiltLinux LLD 15.0.7 (compatible with GNU linkers) llvm-nm, compatible with GNU nm LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-objcopy, compatible with GNU objcopy LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Registered Targets: aarch64 - AArch64 (little endian) aarch64_32 - AArch64 (little endian ILP32) aarch64_be - AArch64 (big endian) arm - ARM arm64 - ARM64 (little endian) arm64_32 - ARM64 (little endian ILP32) armeb - ARM (big endian) bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) hexagon - Hexagon mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian) ppc32 - PowerPC 32 ppc32le - PowerPC 32 LE ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE riscv32 - 32-bit RISC-V riscv64 - 64-bit RISC-V systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 llvm-addr2line LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-strip, compatible with GNU strip LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Fixes: google#5385
novitoll
added a commit
to novitoll/syzkaller
that referenced
this issue
Nov 4, 2024
Replace clang, llvm apt packages in syzbot Dockerfile with the stable kernel version. Reduce Dockerfile code (less RUN layers). Use stable LLVM version that Linux kernel doc suggests [1]. [1] https://mirrors.edge.kernel.org/pub/tools/llvm/ These 2 binaries are missing in the archive: "clang-format, clang-tidy", hence install them as the current apt package. This patch increases syzbot docker image size by 230MB, see the comparison between "syz_master" (current master branch built image), and with the applied patch "syz_master_diff". $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE syz_master_diff latest 40b209ef582e 6 hours ago 5.54GB syz_master latest 292faad84de9 13 hours ago 5.31GB The archive is 123M size and I've built the image 3 times with removing the images as the fresh install, and curl downloading for my 500Mbit/s Ethernet downlink took: 1st build: 0 hour 0 min 50 sec 2nd build: 1 hour 3 min 5 sec 3rd build: 0 hour 1 min 5 sec Hereby, I'm concerned if the archive can be mirrored from "mirrors.edge.kernel.org" to more stable/reliable storage with more distributed CDN like dl.google.com, or it's something local with my downlink. Per the 3rd build, the patch also speeds up (~1 min faster) the docker image build time: ... Step 7/27 : RUN curl -L <redacted> ---> Running in ae2853dfc0fc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 123M 100 123M 0 0 1916k 0 0:01:05 0:01:05 --:--:-- 2513k ... Successfully built 387ae7e06dfa Successfully tagged syz_master_diff:latest real 6m23.013s user 0m0.145s sys 0m0.094s and the original "syz_master" fresh build: ... Successfully built 843592c9ce1b Successfully tagged syz_master:latest real 7m18.333s user 0m0.152s sys 0m0.090s Verified the binaries' versions in the container as well: $ docker run --rm --name syz_master_diff -it syz_master_diff bash /🤖 for e in {clang,clang++,clang-tidy,clang-format,ld.lld,llvm-nm,\ > llvm-ar,llvm-objcopy,llvm-objdump,llvm-addr2line,llvm-readelf,\ > llvm-strip}; do echo; $e --version; done Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian LLVM version 15.0.6 Optimized build. Default target: x86_64-pc-linux-gnu Host CPU: skylake Debian clang-format version 15.0.6 ClangBuiltLinux LLD 15.0.7 (compatible with GNU linkers) llvm-nm, compatible with GNU nm LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-objcopy, compatible with GNU objcopy LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Registered Targets: aarch64 - AArch64 (little endian) aarch64_32 - AArch64 (little endian ILP32) aarch64_be - AArch64 (big endian) arm - ARM arm64 - ARM64 (little endian) arm64_32 - ARM64 (little endian ILP32) armeb - ARM (big endian) bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) hexagon - Hexagon mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian) ppc32 - PowerPC 32 ppc32le - PowerPC 32 LE ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE riscv32 - 32-bit RISC-V riscv64 - 64-bit RISC-V systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 llvm-addr2line LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-strip, compatible with GNU strip LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Fixes: google#5385
novitoll
added a commit
to novitoll/syzkaller
that referenced
this issue
Nov 4, 2024
Replace clang, llvm apt packages in syzbot Dockerfile with the stable kernel version. Reduce Dockerfile code (less RUN layers). Use stable LLVM version that Linux kernel doc suggests [1]. [1] https://mirrors.edge.kernel.org/pub/tools/llvm/ These 2 binaries are missing in the archive: "clang-format, clang-tidy", hence install them as the current apt package. This patch increases syzbot docker image size by 230MB, see the comparison between "syz_master" (current master branch built image), and with the applied patch "syz_master_diff". $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE syz_master_diff latest 40b209ef582e 6 hours ago 5.54GB syz_master latest 292faad84de9 13 hours ago 5.31GB The archive is 123M size and I've built the image 3 times with removing the images as the fresh install, and curl downloading for my 500Mbit/s Ethernet downlink took: 1st build: 0 hour 0 min 50 sec 2nd build: 1 hour 3 min 5 sec 3rd build: 0 hour 1 min 5 sec Hereby, I'm concerned if the archive can be mirrored from "mirrors.edge.kernel.org" to more stable/reliable storage with more distributed CDN like dl.google.com, or it's something local with my downlink. Per the 3rd build, the patch also speeds up (~1 min faster) the docker image build time: ... Step 7/27 : RUN curl -L <redacted> ---> Running in ae2853dfc0fc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 123M 100 123M 0 0 1916k 0 0:01:05 0:01:05 --:--:-- 2513k ... Successfully built 387ae7e06dfa Successfully tagged syz_master_diff:latest real 6m23.013s user 0m0.145s sys 0m0.094s and the original "syz_master" fresh build: ... Successfully built 843592c9ce1b Successfully tagged syz_master:latest real 7m18.333s user 0m0.152s sys 0m0.090s Verified the binaries' versions in the container as well: $ docker run --rm --name syz_master_diff -it syz_master_diff bash /🤖 for e in {clang,clang++,clang-tidy,clang-format,ld.lld,llvm-nm,\ > llvm-ar,llvm-objcopy,llvm-objdump,llvm-addr2line,llvm-readelf,\ > llvm-strip}; do echo; $e --version; done Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian clang version 15.0.6 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin Debian LLVM version 15.0.6 Optimized build. Default target: x86_64-pc-linux-gnu Host CPU: skylake Debian clang-format version 15.0.6 ClangBuiltLinux LLD 15.0.7 (compatible with GNU linkers) llvm-nm, compatible with GNU nm LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-objcopy, compatible with GNU objcopy LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Registered Targets: aarch64 - AArch64 (little endian) aarch64_32 - AArch64 (little endian ILP32) aarch64_be - AArch64 (big endian) arm - ARM arm64 - ARM64 (little endian) arm64_32 - ARM64 (little endian ILP32) armeb - ARM (big endian) bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) hexagon - Hexagon mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian) ppc32 - PowerPC 32 ppc32le - PowerPC 32 LE ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE riscv32 - 32-bit RISC-V riscv64 - 64-bit RISC-V systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 llvm-addr2line LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake llvm-strip, compatible with GNU strip LLVM (http://llvm.org/): LLVM version 15.0.7 Optimized build. Default target: x86_64-unknown-linux-gnu Host CPU: skylake Fixes: google#5385
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
Replace clang, llvm apt packages in syzbot Dockerfile with the stable kernel version.
Describe the solution you'd like
We could've used stable LLVM versions that kernel doc suggests? I've found that it's the most stable way rather than installing from the distro's packages.
From https://docs.kernel.org/kbuild/llvm.html:
Do you have any implementation in mind for this feature?
Additional context
this archive contains the most of llvm utils, also llvm-readelf, llvm-strip mentioned in this PR #5367:
Please let me know what you think.
CC: @tarasmadan , @dvyukov , @a-nogikh , @ramosian-glider
Downloading the archive can probably be sped up with uploading to Google drive or some another closer to the requester location.
It took me ~20 mins for 153MB
The text was updated successfully, but these errors were encountered: