How to limit or force a subset of instruction sets?

[reFX-Mike]

In our projects, we can not rely on ALL architectures (AVX, AVX2, etc.) to be present on all cores (Hybrid CPUs exist), and we have no control over which core our code gets run on; it might even switch between function call to function call.

Explanation: We sell plugin products. These are dynamic libraries that expose a certain number of functions. The hosting application might call these functions from any thread, at any time, on any core.

So we can't detect the instruction set on startup, so we set a few function pointers and be done with it. An instruction set that gets detected on core0 doesn't necessarily exist on core13.

We don't even need dynamic dispatch, we want to make sure that on Intel CPUs, no instructions higher than SSE4 (4.1 & 4.2) get emitted.

Are the usual defines OK for that?

CMT_FORCE_GENERIC_CPU=1
CMT_ARCH_SSE4_2=1
CMT_ARCH_SSE42=1
CMT_ARCH_SSE4_1=1
CMT_ARCH_SSE41=1
CMT_ARCH_SSSE3=1
CMT_ARCH_SSE3=1
CMT_ARCH_SSE2=1
CMT_ARCH_SSE=1

Or do we need to set others? What if we want to move away from using the C-API and use the "normal" C++ stuff? Do the same definitions work as well?

I'm sorry to ask this, but the documentation is pretty poor, doesn't explain a whole lot, and has a ton of gaps. It's by far the weakest point and biggest hindrance to the adoption of KFRlib.

[dancazarin]

Do you use cmake? Do you use install feature? It's far easier to set proper architectures with cmake.

To enable only SSE4.1 and disable runtime dispatch you need to use the following flags:

cmake -GNinja -DKFR_ARCH=sse41 -DKFR_ENABLE_MULTIARCH=OFF -DCMAKE_INSTALL_PREFIX=install-dir <other-arguments>
ninja install

This automatically sets all needed c++ defines and flags and ensures that the generated code can run on SSE4.1 cpu.

Note that automatic cpu detection can be turned off at runtime by calling override_cpu with the instruction level that is available on all cores.

Example:

// somehow detect that we are running on Intel hybrid architecture.
// P-cores: AVX, AVX2, AVX512
// E-cores: AVX, AVX2
kfr::override_cpu(kfr::cpu_t::avx2); // common to all cores

CMT_ARCH_* defines are not meant to be defined explicitly. If you don't use cmake and pass c++ flags to compiler directly just set cpu architecture using your compiler default method (-msse4.1 for gcc/clang) and ensure no other arch flags are passed in the command line. The compiler automatically defines __SSE4_2__ macros, which is enough for KFR to define its own internal CMT_ARCH_* macroses.

[reFX-Mike]

Yes, we use cmake, but currently don't use cmake for kfr because of the aforementioned problems. Also because it didn't build originally with MSVC.

For current plugin projects, we actually created a KFRlib_static build, a library we can link to with the functionality we need as simple C functions.

We would like to change that and build KFRlib directly into the projects.

kfr::override_cpu(kfr::cpu_t::avx2); // common to all cores

Is that thread-safe? Our plugins are effectively DLLs on Windows, and any global variables (like function pointers?) are then also shared. I want to avoid us calling override_cpu in our constructor and modifying these pointers while on other threads, the same pointers might get used. Since on a 64-bit system, the pointers are 8 bytes long, they are not atomic by default, and adding mutexes around the function calls seems expensive.

Maybe we can get away with using a std::call_once on a static std::once_flag?

[reFX-Mike]

We already use the clang toolset and overwrite the path, so we use the latest version (18.1.5), which leads to another problem: the build system is MSVC, so the KFR cmake adds the "ZC:/lambda" command-line option, which is unrecognized by clang, because the C++ standard is set to C++20. This now causes warnings for each file during build.

clang-cl : warning : argument unused during compilation: '/Zc:lambda' [-Wunused-command-line-argument]

[dancazarin]

Apart from this warning, the builds works with clang-cl? This should not cause any problem with the build itself.

[reFX-Mike]

Yes, but when you enable "warnings are errors" (which we do for release builds), then this stops the build unless we suppress the warning itself. Ideally the /Zc:lambda shouldn't be emitted unless needed.

[dancazarin]

What exactly KFR version (or commit) do you use?

[reFX-Mike]

For this particular project, the dev branch, freshly pulled today 5c37b7b

[reFX-Mike]

Please also keep in mind that for macOS, we still need Neon support. So disabling multi-arch, does that also disable Neon? Do we need a separate define for that?

[dancazarin]

KFR_ENABLE_MULTIARCH flags makes no sense on ARM targets because dynamic dispatch is not implemented there, NEON is required to be present (in fact it always present on all supported Apple devices) and no further instruction sets available at this moment similar to multiple versions of SSE* and AVX* like intel cpus have.

[reFX-Mike]

So can you tell me the lines I should add to your CMakeLists.txt to make KFR use only up to SSE4 on Intel CPUs, always use Neon on ARM CPUs?

[dancazarin]

You can always refer to how KFR releases are built
https://github.com/kfrlib/kfr/blob/main/.github/workflows/build.yml

The proper way to use KFR is to install it to a separate location (unix-style of building libraries).

Windows

// LLVM must be installed (latest version is preferred)
// Builds SSE41 build with dynamic dispatch turned OFF

REM !!! Replace the paths below !!!
REM execute this in the KFR source tree
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
cmake -B build-release -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=dist -GNinja  -DKFR_ENABLE_CAPI_BUILD=ON -DKFR_ARCH=sse41 -DKFR_ENABLE_MULTIARCH=OFF -DCMAKE_CXX_COMPILER="C:/Program Files/LLVM/bin/clang-cl.exe" -DCMAKE_LINKER="C:/Program Files/LLVM/bin/lld-link.exe" -DCMAKE_AR="C:/Program Files/LLVM/bin/llvm-lib.exe"

This builds MSVC-compatible binaries in build-release/dist directory.

macOS

brew install ninja
cmake -B build-release -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=dist -GNinja  -DCMAKE_OSX_ARCHITECTURES=x86_64 -DCMAKE_OSX_DEPLOYMENT_TARGET=10.13 -DKFR_ENABLE_CAPI_BUILD=ON -DKFR_ARCH=sse41 -DKFR_ENABLE_MULTIARCH=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON
     

[reFX-Mike]

But this builds KFR as a static library we can link to. What if we want to use the C++ functionality? buffer *= 0.5f and similar constructs?

[dancazarin]

This builds both C and C++ libraries and puts both C and C++ headers to include directory. In fact there is no way to build only C library and skip building C++ libraries.

[dancazarin]

Could you try to download prebuilt binaries from here https://github.com/kfrlib/kfr/releases/tag/6.0.2
or build it exactly with the instructions from .github/workflows/build.yml and after you get the directory with lib, include subdirectories follow this guide https://www.kfrlib.com/docs/latest/installation/#cmake-find_package.

CMAKE_PREFIX_PATH must point at the directory with KFR binaries.

[reFX-Mike]

I will try this afternoon (not too familiar with cmake) and get back to you. Thanks.

[dancazarin]

What exactly KFR version (or commit) do you use?

Thank you for reporting this and the feedback.