summer2020.md

layout	title	subtitle	show_sidebar	hide_footer	permalink	hero_image
page	AirLab Summer School	6-10 July 2020	false	false	/summer2020/	/img/tartan2.png

General Information

The goal of this boot camp is to get each student started with the basics, broaden their horizon beyond their own research topic, understand the tradeoffs of different approaches in our context (real-time, flying robots, etc.) and make our students more “dangerous” with some code/tools that they can use to jump start their research.

Sessions cover topics in Perception, State Estimation, Action, and Infrastructure. Each session will consist of a lecture followed by hands-on exercises.

Click on a session in the overview below to jump to its summary, video, outline, and links.

Sessions Overview

Session	Host	Links
1.1 Graph Learning	Chen	slides | repo
1.2 Object Detection and Tracking	Sourish & Anish & Andrew	slides | repo
1.3 Stereo Basics and Advanced Stereo	Yaoyu	slides | repo
2.1 Visual Odometry	Yafei	slides | repo
2.2 SLAM / GTSAM	Weikun	slides | repo
3.1 Motion Planning Tutorial (Basics)	Basti	slides | repo
3.2 Motion Planning (Advanced)	Jay & Brady	slides | repo
3.3 Planning for Exploration	Rohit	slides |
3.5 Formulating and Enforcing Safety	Cherie & Mohammad	slides | repo
3.6 Modeling and Control	Azarakhsh	slides | repo
4.1 Setup in Cloud and Cluster	Kat	slides | repo
4.2 Air Lab Core Stack	John	slides | repo
4.3 Simulation & 3.4 Reinforcement Learning	Wenshan & Sourish	slides | repo

Dependencies

Various labs will require these specifications:

• Ubuntu 18.04
• Matlab installed from CMU with all the toolboxes
• Python: Pytorch, torchvision, opencv-python, opencv-contrib-python
• ROS Melodic
• Docker

Session Contents

Perception

---

1.1 Graph Learning An introduction to graph neural network and its application in robotics, with an exercise and implementation in Github Classroom Graph Convolutional Networks Graph Sampling Methods Application and PyTorch Implementation slides | repo

---

1.2 Object Detection and Tracking An overview of popular methods used for object detection and tracking with a brief introduction to platform tools for inference Review of object detection methods based on two flavors Single-shot detectors (YOLO, SSD, etc.) Region-based networks (R-CNN, R-FCN, FPN, etc.) Review of tracking MOSSE, MedianFlow, KCF, CSRT On-platform tools: OpenVino Lab: Google colab session for implementation of a tiny object detector and tracker from scratch Load dataset, train model, and deploy detection slides | tracking slides | tools slides detection colab 1 | detection colab 2 | tracking tutorial

---

1.3 Stereo Basics and Advanced Stereo Revision and implementation of the recent binocular stereo methods including non-leanring and learning-based Review Fundamentals of stereo vision Review of introductory CV courses Review stereo camera calibration. The secret for real-world testing ROS camera_calibration and OpenCV api, MATLAB. Kalibr Public stereo dataset and benchmark Middlebury, Scene Flow, NYU, KITTI, etc Stereo reconstruction by OpenCV SGBM CUDA BM SPS-stereo Reconstruction with sparse depth heuristic Confidence measure Deep-learning models for stereo vision Related topics Multi-view stereo Monocular depth, Unsupervised learning Optical flow slides | C++ | Deep models

---

State Estimation

---

2.1 Visual Odometry (Traditional) Feature based visual odometry Pose estimation method from 2D to 2D Pose estimation method from 3D to 2D Pose estimation method from 3D to 3D Direct based visual odometry Optical Flow Epipolar Search slides | repo

---

2.2 SLAM / GTSAM Intro to factor graph Example to solve factor graph Derivation of Gauss Newton Tools for back-end SLAM: GTSAM/Ceres slides, notes, and practice problems

---

Action

---

3.1 Motion Planning Tutorial (Basics) An introduction to the unique challenges of motion planning for flying robots with interactive Matlab exercises Introduction to motion planning Problem formulation Introduction to motion planning Abstraction and approaches Regular graph search: A*-grid search Sampling-based: RRT* Trajectory optimization: CHOMP Results Matlab exploration of planners slides | repo

---

3.2 Motion Planning (Advanced) An overview of newer motion planning methods as well as a walkthrough on how to use OMPL Review of motion planning basics Overview of newer planners and development FMT* Informed RRT* BIT* RABIT* Open Motion Planning Library (OMPL) overview Exercises using OMPL slides | repo

---

3.3 Planning for Exploration Part I - Mapping Occupancy Grid Maps Ray Casting Underlying data structure - OpenVDB Example 2D/3D occupancy grid maps Hands-on session Part II - Frontier Based Exploration Frontiers for exploration planning Example Drone Autonomy Architecture - I Example Drone Autonomy Architecture - II Overview of modules Hands-on session slides

---

3.5 Formulating and Enforcing Safety Purpose: Building intuition on different safety tools and where they fail Where things fit in? Common use cases… I want to make a safety claim on my planning stack AACUS - Emergency maneuver library Key tools Reachability Curse of dimensionality Control barrier functions How does CBF makes a safety claim? Interactive Lab Session with ECBF Single robot (with safe control and without) with fill-ins on base QP optimization Multirobot Problems with CBF Deadlock and how to address Unsolvable cases slides | repo

---

3.6 Modeling and Control A practical introduction to modeling and control for Multirotor Aerial Vehicles with interactive MATLAB exercises Introduction Modeling Multirotors Multirotor Controller Architecture (with PX4 as an example) PID Controllers Underactuated vs. Fully-Actuated UAVs Matlab exercises Attitude Controller and PID Tuning Position Controller Fully-Actuated Multirotors slides | repo

---

Infrastructure

---

4.1 Setup in Cloud and Cluster An intro to setting up infrastructure on the cloud & cluster and an overview of various AirLab development operations tools Docker Cluster & SLURM Azure Continuous Intergration Git, Ansible, Deployer slides | repo

---

4.2 AirLab Core Autonomy Stack An overview of the AirLab Core Autonomy Stack, including the setup, walkthrough, and running examples Behavior Trees Global and Local Planners Trajectory, Pose, and Velocity Controllers State Estimation Gazebo Sim Exercises slides | repo

---

4.3 & 3.4 Simulation and Reinforcement Learning An introduction of doing drone simulation in AirSim, using the core stack package in the simulation, as well as doing reinforcement learning in it Presentation Unreal Engine and AirSim basics Basic concepts Installation procedure Basic operations, navigate, add, move, scale, rotation etc. Use AirSim w/ ROS AirSim interfaces AirSim ROS adapter Joystick/ keyboard Use AirSim on cluster w/ docker Integrate AirSim with the core stack and px4 Use AirSim on Azure Lab A toy example of using AirSim w/ ROS People Avoidance using Reinforcement Learning Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations Integrate AirSim with PX4 slides | repo-RL | repo-AirSim-CoreStack repo-AirSim-ROS

---