User guide

Overview

AITrack is an independent tracker which, using artificial intelligence, recognizes your face and calculates it's 3D position, which can be used by lots of games for translating your head motion into the virtual world.

At the moment, AITrack can't "talk" directly to your games, because of that, Opentrack is needed. So, how does it work?

The pipeline is pretty simple:

1. Your camera captures the video
2. AITrack recognizes your face and estimates its 3D position (x, y, z, yaw, pitch and roll).
3. AITrack sends over UDP the recognized position to Opentrack.
4. Opentrack relays that data to your game.

Installation

1. Download and install Visual C++ Redistributable x64 in case you don't already have it.
2. Download Opentrack from Opentrack's releases page
3. Download the most recent version of the tracker from the releases page (.zip file) and extract its content.
4. Launch Opentrack and make sure to select Input = UDP over network. If you click the hammer, next to the Input dropdown, you should have "4242" as defaut. Then, click "Start". Opentrack then will be listening for the data AITrack will send.
5. Run AITrack.exe click "Start tracking".
6. Look around! If the pink octopus in Opentrack moves, that's good signal!

If the octopus doesn't move and both programs are tracking, then you probably aren't using Port=4242 or your Windows firewall is blocking the connection.

You're pretty much done!

Initial configuration

The key to obtaining good results with this program lies on tweaking all the parameters (both on AITrack and Opentrack).

The most important parameter AITrack has is the "Distance" parameter, which is the distance, in meters, at which your webcam is in straight line from your face. If you have problems with your recognized position, try fine-tuning it. However, it doesn't need to be ultra precise (no need to get a ruler ;)).

In order to mitigate vibrations, you should add and configure a filter in Opentrack (this is done with the Filter dropdown). The one I recommend it's Acella. Select it and then click the hammer to configure it. Although you should find the values that you're most comfortable with, here are the ones I use with XPlane 11:

• Smoothing (Rotation filtering) = 1.55 degree
• Deadzone (Rotation filtering) = 0.03 degree
• Smoothing (Position filtering) = 0.8 mm
• Deadzone (Position filtering) = 0.85 mm

The last thing we're going to take a look at are the Opentrack "curves". They represent the mapping between how much you move your head in the real life and how much the view is moved in game. You can configure them by clicking "Mapping" in Opentrack.

These mappings are very user specific (and even between games!), so I'm not going to stick mine here because they're probably not comfortable to you. It's better to learn to create your own. Take a look at this video: Tweaking your curves in Opentrack, which explains the process pretty well. As a final recommendation, try to add a deadzone near 0 on every axis in order to have a comfortable neutral head position.

Opentrack configuration example

The following screenshots will show you an example template of Opentrack config you could use with AITrack in case you're a beginner.

Some additional notes:

• Make sure you use same port for opentrack and aitrack (default should be 4242 for both).
• Look at center of monitor when starting Opentrack with center at startup option.
• Yaw scales raw 30degrees to 120degrees equivalent to looking over shoulder
• Pitch scales raw 30 degrees to 90 degrees, equivalent to looking up and down.
• In this example config, X and Y mapping are disabled to be always centered as if strapped into chair. Although you can enable them for more realism by raising a little bit the points on the right (also add a dead zone at the leftmost part).
• Z mapping is enabled to be able to look closer at instrument panel.

For more realism:

• Remove or minimize the center null zone but this may add jitter and make it more difficult to read instruments.

Tips

Based on the testing made so far, here are some recommendations for getting the best performance:

• Configure well your movement curves on Opentrack. Leave a little "dead zone" at the beginning of each curve and use the asymetric mapping feature for "Pitch".
• Position your camera at about 0.5-2 meters horizontally from your face.
  • It's better if the camera is directly in front of you, but it doesn't mattter if you have some lateral offset.
• The camera should be, approximately, at about your nose-level. Good positions are on top of our monitor, or at its base.
• If you find the tracking is not accurate, fine-tune "Distance" parameter only to the actual value (distance from your face to the camera, in meters).
• Use an Acella or Hamilton filter with Opentrack and set the smoothing near the maximum values (both rotation and position).
• Increase FPS as far as you camera can handle it. The higher the FPS, the more responsiveness (60fps is quite well).
• Using lower camera resolutions and lowering FPS can save CPU. Don't worry, the tracking quality will be the same on 90% of the cases.