Syllabus

Books

  1. CORKE - Robotics, Vision and Control: Fundamental Algorithms in PYTHON, by Peter Corke, 3rd edition, 2023. This book is required and is free to download.

  2. THRUN - Probabilistic Robotics, by Sebastian Thrun, Wolfram Burgard, and Dieter Fox, 2005.

  3. AIMA - Artificial Intelligence: A Modern Approach, by Stuart Russell, 4th edition, 2021 and also here..

Planned Schedule

Lecture 1: We start with an introduction to AI and robotics from a systems perspective with emphasis on the autonomous vehicles application domain. We explain the various systems that need to be engineered to allow safe and efficient self-driving and review prerequisites on programming (Python) as well as linear algebra and probability theory. With the help of the TAs & other tutorial videos, we also ensure that students have setup a programming environment necessary for the projects and assignments of the course. Reading: selected pages from the course web site.

Part I: Robotic Perception

Lecture 2: The perception subsystem is the first stage of the processing chain in robotics. It processes and fuses multi-modal sensory inputs and is implemented using deep neural networks. Our focus here is to understand how prediction can be engineered by taking the maximum likelihood optimization principle and its associated cross entropy loss function and applying it to neural networks - in this lecture focusing on fully connected and subsequently convolutional architectures for supervised regression and classification tasks. Reading: AIMA Chapter 19 and 21.

Lecture 3: Naturally the two tasks of classification and regression will be combined to form object detectors. Initially we cover YOLOv8 and MaskRCNN as representatives of single and two-stage architectures and introduce the engineering aspects of building object detection models with pre-trained feature extractors and using transfer learning for making predictions outside the scope of the pretrained models - a typical setting in robotic applications. AIMA Chapter 25 (in part).

Lecture 4: In this lecture we expand on object detection and provide the agent with the additional capabilities of semantic and instance segmentation that are essential for completing planning tasks in complex scenes. AIMA Chapter 25.

Lecture 5: In this lecture we introduce probabilistic models that process the perceptive predictions over time and understand how the agent will track / update its time-varying belief about the state of the environment. This is achieved with recursive state estimation algorithms acting on a problem setting represented with dynamic bayesian networks. This lecture introduces Bayesian filters in discrete and continuous state spaces (Kalman filters). All robots employ such filters.  Reading: AIMA Chapters 12, 13 & 14.

Part II: Localization and Mapping

Lecture 6: In Part I we built a static agent that can track moving objects - we expand on agents that move themselves (egomotion). We present well established probabilistic motion models such as the velocity and odometry model and introduce localization as the problem of estimating the agent’s pose in the environment. We evaluate various algorithms that can solve such an estimation problem. Reading: CORKE Chapter 4, THRUN Chapter 5 and 7

Lecture 7: Up to this point, the mobile agent faced a localization estimation task assuming the knowledge of the environment’s map. We now expand on agents that can do Simultaneous Localization and Mapping (SLAM). Due to the plethora of the SLAM methods, we focus here on one that brings together Part I and Part II: Visual SLAM demonstrating how even monocular cameras can be used to move the agent safely in a dynamic environment. Reading: CORKE Chapter 6, THRUN Chapters 9 & 10.

Part III: Motion Planning and Control

Lecture 8: After the successive enhancements of Part I and Part II, the agent has now a clear view of the environment state such as what and where the objects that surround it are, its able to track them as they move and know where is located as itself is moving. In this lecture we introduce the agents short and longer term goals and how, optimal planning under uncertainty solutions, produces the best sequence of actions to reach its goal state. Although our final destination is optimal action taking in dynamic stochastic environments. we start by connecting planning and problem solving and with the assumption that the goal state is feasible and the environment is known and deterministic. In this case, the planning problem becomes that of a search and cover a range of search-based algorithms such as A, D, RRT* and PRM. Readings: Reading: CORKE Chapter 2, AIMA Chapters 3 & 4 and 11.

Lecture 9: We finally reached the point where we need to execute the plan such as a trajectory that was planned over the next few hundreds of ms for a . In this lecture we introduce the control subsystem that is responsible for executing the planned actions. We start with the basics of control theory and introduce the PID controller. We then move to more advanced control algorithms such as LQR and MPC. Reading: Course notes.

Part IV: Reinforcement Learning, Imitation Learning and Sim2Real

With Part III, we have implemented the basic so-called decoupled architecture for autonomous robots that decouples perception from planning and control. In this last part we look at end to end diffential architectures,

Lecture 10: We now make a considerable extension to our assumptions: the utility of the agent now depends on a sequence of decisions and, further, the stochastic environment offers a feedback signal to the agent called reward. We review how the agent’s policy, the sequence of actions, can be calculated when it fully observes its current state (MDP) and also when it can only partially do so (POMDP). The discussion here is on fundamentals and goes through an understanding of the Bellman expectation backup and optimality equations following David Silver’s (DeepMind) lecture series at Oxford. Readings: AIMA Chapter 16 & 17.

Lecture 11: The prediction and control algorithms that approximate the optimal MDP policies using either a world model or a model-free approach are known as Reinforcement Learning (RL). In this lecture we establish the connection between MDP and RL and dive into deep learning especially focusing on Deep RL algorithms that have been shown to be applicable in robotics - mostly model-free methods. Reading: AIMA Chapter 22.

Lecture 12: Simulation-to-reality transfer has emerged as a popular and highly successful method to train robotic control policies for a wide variety of tasks. In this concluding topic we review the various methods that have been proposed to bridge the gap between simulation and reality such as domain adaptation and demonstrate a real-worl use case based on PyBullet - a popular sim2real tool. Reading: Course notes.

Lecture 13:: Review - last lecture before the final.

Back to top