Skip to content
Jack Gerrits edited this page Jun 4, 2021 · 74 revisions

Getting started > [Dependencies] > Building > Installing > Tutorial


Before building VW from source the following dependencies must be satisfied.

Ubuntu

Ensure you have the source:

git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit

Get dependencies:

sudo apt install libboost-dev libboost-thread-dev libboost-program-options-dev libboost-system-dev libboost-math-dev libboost-test-dev zlib1g-dev cmake g++

# Optional: flatbuffers
sudo apt install libflatbuffers-dev

# Optional: Building Python bindings
sudo apt install libboost-python-dev

WSL Notes

  • If using WSL ensure you clone the repo in WSL and not in Windows to prevent broken line endings.
  • If building on WSL on an NTFS drive, also be sure that the drive is mounted with the "metadata" option, or CMake will not complete successfully. (See: here)

Next step: building on Linux

MacOS

Ensure you have the source:

git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit

HomeBrew

brew install cmake boost zlib

# Optional: flatbuffers
brew install flatbuffers 

# Optional: Building Python bindings
brew install boost-python3

The rest of the Linux instructions should apply to MacOS too.

Next step: building on Linux

Windows

Ensure you have the source:

git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
  • Either Visual Studio 2017 or 2019 can be used, VS2019 is recommended
  • Visual Studio 2017 (v141) toolset must be installed if using VS 2019
  • 10.0.16299.0 Windows 10 SDK must be installed
  • Nuget is used for other dependencies. Open vowpalwabbit/vw.sln in Visual Studio and restore Nuget dependencies

If you are not interested in using the flatbuffer support you can skip to: Next step: building on Windows

If using Visual Studio 2017 (v141) flatbuffers can be installed via Vcpkg by running:

vcpkg install flatbuffers:x64-windows
  • Set VcpkgIntegration environment variable to vcpkg.targets file on your machine. Example: VcpkgIntegration=<vcpkg_root>\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets
  • Set BUILD_FLATBUFFERS environment variable to BUILD_FLATBUFFERS on you machine. Should look like: BUILD_FLATBUFFERS=BUILD_FLATBUFFERS
  • Add the flatbuffers tool directory to your PATH: <vcpkg_root>\installed\x64-windows\tools\flatbuffers OR set flatcPath environment variable to the flatc.exe on your machine. Example: flatcPath=<vcpkg_root>\installed\x64-windows\tools\flatbuffers\flatc.exe

If using Visual Studio 2019 or if Visual Studio 2019 (v142) is installed then additional steps will need to be taken to avoid linker errors due to version incompatibilities. Flatbuffers will need to be installed via Vcpkg using v141:

  • From the triplets directory in your Vcpkg installation (e.g. <vcpkg_root>\triplets)
cp x64-windows.cmake x64-windows-v141.cmake
  • Set the VCPKG_PLATFORM_TOOLSET variable to v141 by adding set(VCPKG_PLATFORM_TOOLSET v141) to the new x64-windows-v141.cmake file
vcpkg install flatbuffers:x64-windows-v141
  • Set VcpkgIntegration environment variable to vcpkg.targets file on your machine. Example: VcpkgIntegration=<vcpkg_root>\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets
  • Set BUILD_FLATBUFFERS environment variable to BUILD_FLATBUFFERS on you machine. Should look like: BUILD_FLATBUFFERS=BUILD_FLATBUFFERS
  • Set VcpkgPlatformToolset environment variable to v141
  • Add the flatbuffers tool directory to your PATH: <vcpkg_root>\installed\x64-windows-v141\tools\flatbuffers OR set flatcPath environment variable to the flatc.exe on your machine. Example: flatcPath=<vcpkg_root>\installed\x64-windows-v141\tools\flatbuffers\flatc.exe

Next step: building on Windows

Experimental: CMake build system on Windows

Note: The CSharp projects are not yet converted to CMake for Windows. So the CMake generated solution is only for C++ projects for the time being. For this reason the existing solution can not yet be deprecated.

When using CMake for Windows, dependencies are easily managed through Vcpkg. Install the following dependencies with Vcpkg:

vcpkg --triplet x64-windows install zlib boost-system boost-program-options boost-test boost-align boost-foreach boost-python boost-math python3 flatbuffers

Next step: building on Windows with CMake

Details

A vendored dep is provided by the project and built as part of building VW. A system dep is externally provided by the OS or a package manager. All dependencies must be available as as system dependency and can optionally be provided vendored.

Dependency Available as vendored? Can be system? Optional?
boost No Yes, default No
zlib No Yes, default No
rapidjson Yes, default Yes, -DRAPIDJSON_SYS_DEP=On in CMake No
flatbuffers No Yes, default Yes, off by default. -DBUILD_FLATBUFFERS=On in CMake to enable
spdlog Yes, default No* No
fmt Yes, default No* No

* TODO - enable as system dependency

Clone this wiki locally