-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Dependencies
Getting started > [Dependencies] > Building > Installing > Tutorial
Before building VW from source the following dependencies must be satisfied.
Ensure you have the source:
git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
Get dependencies:
sudo apt install libboost-dev libboost-thread-dev libboost-program-options-dev libboost-system-dev libboost-math-dev libboost-test-dev zlib1g-dev cmake g++
# Optional: flatbuffers
sudo apt install libflatbuffers-dev
# Optional: Building Python bindings
sudo apt install libboost-python-dev
- If using WSL ensure you clone the repo in WSL and not in Windows to prevent broken line endings.
- If building on WSL on an NTFS drive, also be sure that the drive is mounted with the "metadata" option, or CMake will not complete successfully. (See: here)
Ensure you have the source:
git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
brew install cmake boost zlib
# Optional: flatbuffers
brew install flatbuffers
# Optional: Building Python bindings
brew install boost-python3
The rest of the Linux instructions should apply to MacOS too.
Ensure you have the source:
git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
- Either Visual Studio 2017 or 2019 can be used, VS2019 is recommended
- Visual Studio 2017 (v141) toolset must be installed if using VS 2019
-
10.0.16299.0
Windows 10 SDK must be installed - Nuget is used for other dependencies. Open
vowpalwabbit/vw.sln
in Visual Studio and restore Nuget dependencies
If you are not interested in using the flatbuffer support you can skip to: Next step: building on Windows
If using Visual Studio 2017 (v141) flatbuffers can be installed via Vcpkg by running:
vcpkg install flatbuffers:x64-windows
- Set
VcpkgIntegration
environment variable tovcpkg.targets
file on your machine. Example:VcpkgIntegration=<vcpkg_root>\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets
- Set
BUILD_FLATBUFFERS
environment variable toBUILD_FLATBUFFERS
on you machine. Should look like:BUILD_FLATBUFFERS=BUILD_FLATBUFFERS
- Add the flatbuffers tool directory to your
PATH
:<vcpkg_root>\installed\x64-windows\tools\flatbuffers
OR setflatcPath
environment variable to theflatc.exe
on your machine. Example:flatcPath=<vcpkg_root>\installed\x64-windows\tools\flatbuffers\flatc.exe
If using Visual Studio 2019 or if Visual Studio 2019 (v142) is installed then additional steps will need to be taken to avoid linker errors due to version incompatibilities. Flatbuffers will need to be installed via Vcpkg using v141:
- From the triplets directory in your Vcpkg installation (e.g.
<vcpkg_root>\triplets
)
cp x64-windows.cmake x64-windows-v141.cmake
- Set the
VCPKG_PLATFORM_TOOLSET
variable tov141
by addingset(VCPKG_PLATFORM_TOOLSET v141)
to the newx64-windows-v141.cmake
file
vcpkg install flatbuffers:x64-windows-v141
- Set
VcpkgIntegration
environment variable tovcpkg.targets
file on your machine. Example:VcpkgIntegration=<vcpkg_root>\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets
- Set
BUILD_FLATBUFFERS
environment variable toBUILD_FLATBUFFERS
on you machine. Should look like:BUILD_FLATBUFFERS=BUILD_FLATBUFFERS
- Set
VcpkgPlatformToolset
environment variable tov141
- Add the flatbuffers tool directory to your
PATH
:<vcpkg_root>\installed\x64-windows-v141\tools\flatbuffers
OR setflatcPath
environment variable to theflatc.exe
on your machine. Example:flatcPath=<vcpkg_root>\installed\x64-windows-v141\tools\flatbuffers\flatc.exe
Next step: building on Windows
Note: The CSharp projects are not yet converted to CMake for Windows. So the CMake generated solution is only for C++ projects for the time being. For this reason the existing solution can not yet be deprecated.
When using CMake for Windows, dependencies are easily managed through Vcpkg. Install the following dependencies with Vcpkg:
vcpkg --triplet x64-windows install zlib boost-system boost-program-options boost-test boost-align boost-foreach boost-python boost-math python3 flatbuffers
Next step: building on Windows with CMake
A vendored dep is provided by the project and built as part of building VW. A system dep is externally provided by the OS or a package manager. All dependencies must be available as as system dependency and can optionally be provided vendored.
Dependency | Available as vendored? | Can be system? | Optional? |
---|---|---|---|
boost | No | Yes, default | No |
zlib | No | Yes, default | No |
rapidjson | Yes, default | Yes, -DRAPIDJSON_SYS_DEP=On in CMake |
No |
flatbuffers | No | Yes, default | Yes, off by default. -DBUILD_FLATBUFFERS=On in CMake to enable |
spdlog | Yes, default | No* | No |
fmt | Yes, default | No* | No |
* TODO - enable as system dependency
- Home
- First Steps
- Input
- Command line arguments
- Model saving and loading
- Controlling VW's output
- Audit
- Algorithm details
- Awesome Vowpal Wabbit
- Learning algorithm
- Learning to Search subsystem
- Loss functions
- What is a learner?
- Docker image
- Model merging
- Evaluation of exploration algorithms
- Reductions
- Contextual Bandit algorithms
- Contextual Bandit Exploration with SquareCB
- Contextual Bandit Zeroth Order Optimization
- Conditional Contextual Bandit
- Slates
- CATS, CATS-pdf for Continuous Actions
- Automl
- Epsilon Decay
- Warm starting contextual bandits
- Efficient Second Order Online Learning
- Latent Dirichlet Allocation
- VW Reductions Workflows
- Interaction Grounded Learning
- CB with Large Action Spaces
- CB with Graph Feedback
- FreeGrad
- Marginal
- Active Learning
- Eigen Memory Trees (EMT)
- Element-wise interaction
- Bindings
-
Examples
- Logged Contextual Bandit example
- One Against All (oaa) multi class example
- Weighted All Pairs (wap) multi class example
- Cost Sensitive One Against All (csoaa) multi class example
- Multiclass classification
- Error Correcting Tournament (ect) multi class example
- Malicious URL example
- Daemon example
- Matrix factorization example
- Rcv1 example
- Truncated gradient descent example
- Scripts
- Implement your own joint prediction model
- Predicting probabilities
- murmur2 vs murmur3
- Weight vector
- Matching Label and Prediction Types Between Reductions
- Zhen's Presentation Slides on enhancements to vw
- EZExample Archive
- Design Documents
- Contribute: