Autonomous Driving

This post explores collision loss design for end-to-end autonomous driving training, focusing on extending PLUTO’s binary occupancy map approach to handle probabilistic maps. The method enables safer autonomous driving by providing smooth, uncertainty-aware collision avoidance while maintaining computational efficiency. Using Signed Distance Map with Binary Occupancy Map How PLUTO Builds the Loss Map [2] Vehicle Model In PLUTO and other planning algorithms, the vehicle is modelded as a series of overlapping discs. ...

What is SLAM? SLAM demo. SLAM stands for Simultaneous Localization and Mapping. It is a computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. Applications Object Detection Parking Lot Annotation Lane Annotation Lane Reprojection HD Map [source] SLAM has various applications, including: ...

In this post, we will discuss the perspective-n-point (PnP) problem. We will start with the problem definition. Then, gradient-based optimization methods will be introduced. Finally, we will discuss two global optimization methods. Problem Formulation The core task of the Perspective-n-Point (PnP) problem is to determine the pose—specifically, the rotation and translation—of a calibrated camera in 3D space. This is achieved by using a set of known 3D points in the world and their corresponding 2D projections observed on the camera’s image sensor. ...

Given a starting point, starting direction, ending point, and ending direction, the goal is to generate a feasible and “natural” path that connects the two points. This path should adhere to vehicle dynamics and avoids obstacles. However, it is hard to define what is “natural”. Path Planning In this section, we review some classical path planning methods. We ignore the algorithmic details and focus on the resulting path shapes to provide an overview. The demonstration images are generated using the code repository [1]. ...

In this post, we will discuss the coordinate systems commonly used in autonomous driving. In practice, different positioning providers may define their own coordinate systems. This post will introduce the fundamental concepts of these coordinate systems and explain how to convert between them for your specific needs. Pose on the Earth When we talk about pose, we refer to the position and orientation of an object in the world. In the context of autonomous driving, the world is the Earth. Therefore, pose describes the position and orientation of an object relative to the Earth. ...

In this post, we will discuss the end-of-line (EOL) camera calibration, especially for camera bird’s-eye view (BEV) extrinsic calibration. Suggested Pipeline Do single camera intrinsic calibration Measure each corner in the world coordinate Refine the corners’ coordinates according to board constraints (plane, parallel, equally spaced) Find the plane equation in the world coordinate Initialize each camera’s extrinsic parameters by solving the perspective-n-point (PnP) problem Estimate the 3D coordinates by the intersection of the image ray and board plane ...

Tracking ego pose is critical in autonomous driving. In this article, we will discuss how to fuse the IMU, wheel encoder, GPS, etc. to track the ego pose. We will derive the pose tracking algorithm based on the iterative extended Kalman filter. This document mainly follows [1] and [2]. Preliminaries Let $\mathcal{M}$ be the manifold of dimension $n$ in consideration (e.g., $\mathcal{M} = SO(3)$). Since manifolds are locally homeomorphic to $\mathbb{R}^n$, we can establish a bijective mapping from a local neighborhood on $\mathcal{M}$ to its tangent space $\mathbb{R}^n$ via two encapsulation operators $\boxplus$ and $\boxminus$: ...

When tracking an object’s position, it is typically done from the ego vehicle’s perspective. However, the ego’s motion makes tracking the object somewhat difficult. The basic idea is to perform the tracking in the world coordinate system, then transform the results into the ego-car’s coordinate system. In this post, we will discuss how to concisely incorporate the ego’s motion into the object tracking process. Continuous Form Coordinate Definitions The target’s movement in the world coordinate: $\mathbf{o}(t)$; ego-car movement in the world coordinate: $\mathbf{g}(t)$; ego-car’s heading angle: $\theta(t)$; observed target’s coordinate relative to the ego-car: $\mathbf{x}(t)$. ...

Unlike global shutter cameras that capture the entire frame at once, rolling shutter cameras capture each row sequentially, leading to image distortions when there is motion. This paper discusses the rolling shutter effect in cameras and methods to handle it. Fundamentals of Rolling Shutter Camera Global Shutter Camera v.s. Rolling Shutter Camera. (a) Global Shutter Camera. (b) Rolling Shutter Camera. (figure from [6]) To efficiently capture and read the image, the time constraints for rolling shutter camera are: ...