-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Dependencies
Getting started > [Dependencies] > Building > Installing > Tutorial
Before building VW from source the following dependencies must be satisfied.
Ensure you have the source:
git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
Get dependencies:
# Core
sudo apt install libboost-dev libboost-thread-dev libboost-program-options-dev libboost-system-dev libboost-math-dev libboost-test-dev zlib1g-dev cmake g++
# Install flatbuffers
wget https://github.com/google/flatbuffers/archive/v1.12.0.tar.gz
tar -xzf v1.12.0.tar.gz
cd flatbuffers-1.12.0
mkdir build
cd build
cmake -G "Unix Makefiles" -DFLATBUFFERS_BUILD_TESTS=Off -DFLATBUFFERS_INSTALL=On -DCMAKE_BUILD_TYPE=Release -DFLATBUFFERS_BUILD_FLATHASH=Off ..
make install -j$(nproc)
# Optional: test
pip install six
# Optional: Python bindings
sudo apt install libboost-python-dev
# Optional: Java, Ubuntu 16.04 or newer
sudo apt install default-jdk
Note: If using WSL and you cloned the repo in Windows you will need to run git submodule update --init --recursive
before running cmake. Otherwise the submodule will be downloaded with git in bash and the line endings will break git commands from Windows. Otherwise, submodules are automatically downloaded by cmake.
If attempting to build on WSL on an NTFS drive, also be sure that the drive is mounted with the "metadata" option, or cmake will not complete successfully. (See: https://github.com/microsoft/WSL/issues/4257#issuecomment-507799403)
Ensure you have the source:
git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
brew install cmake
brew install boost
brew install flatbuffers
# If you want to statically link VW
brew install zlib
# If you want to build with python 2 support
brew install boost-python
# If you want to build with python 3 support
brew install boost-python3
# Build Boost for Mac OS X 10.8 and below
port install boost +no_single -no_static +openmpi +python27 configure.cxx_stdlib=libc++ configure.cxx=clang++ flatbuffers
# Build Boost for Mac OS X 10.9 and above
port install boost +no_single -no_static +openmpi +python27 flatbuffers
The rest of the Linux instructions should apply to MacOS too.
Ensure you have the source:
git clone --recursive https://github.com/VowpalWabbit/vowpal_wabbit.git
cd vowpal_wabbit
- Either Visual Studio 2017 or 2019 can be used, VS2019 is recommended
- Visual Studio 2017 (v141) toolset must be installed if using VS 2019
-
10.0.16299.0
Windows 10 SDK must be installed - Nuget is used for other dependencies. Open
vowpalwabbit/vw.sln
in Visual Studio and restore Nuget dependencies
If using Visual Studio 2017 (v141) flatbuffers can be installed via Vcpkg by running:
vcpkg install flatbuffers:x64-windows
- Set
VcpkgIntegration
environment variable tovcpkg.targets
file on your machine. Example:VcpkgIntegration=<vcpkg_root>\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets
- Add the flatbuffers tool directory to your
PATH
:<vcpkg_root>\installed\x64-windows\tools\flatbuffers
OR setflatcPath
environment variable to theflatc.exe
on your machine. Example:flatcPath=<vcpkg_root>\installed\x64-windows\tools\flatbuffers\flatc.exe
If using Visual Studio 2019 or if Visual Studio 2019 (v142) is installed then additional steps will need to be taken to avoid linker errors due to version incompatibilities. Flatbuffers will need to be installed via Vcpkg using v141:
- From the triplets directory in your Vcpkg installation (e.g.
<vcpkg_root>\triplets
)
cp x64-windows.cmake x64-windows-v141.cmake
- Set the
VCPKG_PLATFORM_TOOLSET
variable tov141
by addingset(VCPKG_PLATFORM_TOOLSET v141)
to the newx64-windows-v141.cmake
file
vcpkg install flatbuffers:x64-windows-v141
- Set
VcpkgIntegration
environment variable tovcpkg.targets
file on your machine. Example:VcpkgIntegration=<vcpkg_root>\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets
- Set
VcpkgPlatformToolset
environment variable tov141
- Add the flatbuffers tool directory to your
PATH
:<vcpkg_root>\installed\x64-windows-v141\tools\flatbuffers
OR setflatcPath
environment variable to theflatc.exe
on your machine. Example:flatcPath=<vcpkg_root>\installed\x64-windows-v141\tools\flatbuffers\flatc.exe
Next step: building on Windows
Note: The CSharp projects are not yet converted to CMake for Windows. So the CMake generated solution is only for C++ projects for the time being. For this reason the existing solution can not yet be deprecated.
When using CMake for Windows, dependencies are easily managed through Vcpkg. Install the following dependencies with Vcpkg:
vcpkg --triplet x64-windows install zlib boost-system boost-program-options boost-test boost-align boost-foreach boost-python boost-math python3
- Home
- First Steps
- Input
- Command line arguments
- Model saving and loading
- Controlling VW's output
- Audit
- Algorithm details
- Awesome Vowpal Wabbit
- Learning algorithm
- Learning to Search subsystem
- Loss functions
- What is a learner?
- Docker image
- Model merging
- Evaluation of exploration algorithms
- Reductions
- Contextual Bandit algorithms
- Contextual Bandit Exploration with SquareCB
- Contextual Bandit Zeroth Order Optimization
- Conditional Contextual Bandit
- Slates
- CATS, CATS-pdf for Continuous Actions
- Automl
- Epsilon Decay
- Warm starting contextual bandits
- Efficient Second Order Online Learning
- Latent Dirichlet Allocation
- VW Reductions Workflows
- Interaction Grounded Learning
- CB with Large Action Spaces
- CB with Graph Feedback
- FreeGrad
- Marginal
- Active Learning
- Eigen Memory Trees (EMT)
- Element-wise interaction
- Bindings
-
Examples
- Logged Contextual Bandit example
- One Against All (oaa) multi class example
- Weighted All Pairs (wap) multi class example
- Cost Sensitive One Against All (csoaa) multi class example
- Multiclass classification
- Error Correcting Tournament (ect) multi class example
- Malicious URL example
- Daemon example
- Matrix factorization example
- Rcv1 example
- Truncated gradient descent example
- Scripts
- Implement your own joint prediction model
- Predicting probabilities
- murmur2 vs murmur3
- Weight vector
- Matching Label and Prediction Types Between Reductions
- Zhen's Presentation Slides on enhancements to vw
- EZExample Archive
- Design Documents
- Contribute: