Autonomous Driving

Introduction A training pool for a driving planner consists of logged frames annotated with scenario tags, such as lane changes, ramps, cut-ins and stops at red lights. Scenarios that are rare in the logs may nevertheless be important for the model. The pool is therefore resampled before training: frames from under-represented scenarios are repeated, while those from over-represented scenarios may be removed where permitted. Because each frame can carry several tags, repeat factors must be chosen jointly. A frame tagged with both ramp and is_decel, for example, contributes to both scenario counts, so changing one target affects the other. Assigning one weight to each tag combination allows the scenario targets to be met while minimising changes to the original pool. In survey statistics, this procedure is known as raking, or calibration of sampling weights to known marginal totals [1], [2]. Related work addresses calibration of design weights [3] and generalized raking for regression in two-phase samples [4]. ...

Centering Suppose the ego position relative to the center lane is denoted as $X$. We do random position augmentation along the lateral direction. Let $Y$ denote the position after augmentation as $Y = X+ Z$, where $Z$ is the random position augmentation. Assume $X\sim p_x$, $Z\sim p_z$ with its probability density function $f_X(x)$ and $f_Z(z)$, respectively. So, the distribution of $Y$ can be computed as below. First, we will compute the accumulated distribution of $Y$, then can compute the density distribution. Let $F_Y(y)$ denote the accumulated distribution of $Y$, then ...

Occasionally, we discover that the LiDAR extrinsic parameters are inaccurate. In such cases, we aim to recalibrate the SLAM poses and maps based on the updated parameters, without the need to rerun the entire SLAM process. By doing so, we can keep the annotations and the original SLAM map, which saves human effort and computational resources. Raw Data Given LiDAR points in the LiDAR coordinate system $$\mathbf{p}^{orig} = \bigcup_{t=0:T}\{\mathbf{p}_{t,i}\}_{i=1}^{N_t}$$Where $t$ is the time index, $i$ is the point index at time $t$, and $N_t$ is the number of points at time $t$. ...

In this post, we will discuss the post-processing of LiDAR SLAM. We mainly focus on its problem formulation. The content of this post follows papers [1] and [2]. Problem Formulation Factor graph representation of bundle adjustment formulation. (Fig. 1 of [2]) With LiDAR poses, each denoted by $\mathbf{T}_j = (\mathbf{R}_j,\mathbf{t}_j)$ $(j=1,\ldots,M_p)$, the bundle adjustment refers to simultaneously determining all the LiDAR poses (denoted by $\mathbf{T} = (\mathbf{T}_1,\cdots,\mathbf{T}_{M_p})$) and feature parameters (denoted by $\boldsymbol{\pi} = (\pi_1,\cdots,\pi_{M_f})$), such that the reconstructed map agrees with the LiDAR measurements to the best extent. Denote $c(\pi_i,\mathbf{T})$ the map consistency due to the $i$-th feature; a straightforward BA formulation is ...

This post explores collision loss design for end-to-end autonomous driving training, focusing on extending PLUTO’s binary occupancy map approach to handle probabilistic maps. The method enables safer autonomous driving by providing smooth, uncertainty-aware collision avoidance while maintaining computational efficiency. Using Signed Distance Map with Binary Occupancy Map How PLUTO Builds the Loss Map [2] Vehicle Model In PLUTO and other planning algorithms, the vehicle is modelded as a series of overlapping discs. ...

What is SLAM? SLAM demo. SLAM stands for Simultaneous Localization and Mapping. It is a computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it. Applications Object Detection Parking Lot Annotation Lane Annotation Lane Reprojection HD Map [source] SLAM has various applications, including: ...

In this post, we will discuss the perspective-n-point (PnP) problem. We will start with the problem definition. Then, gradient-based optimization methods will be introduced. Finally, we will discuss two global optimization methods. Problem Formulation The core task of the Perspective-n-Point (PnP) problem is to determine the pose—specifically, the rotation and translation—of a calibrated camera in 3D space. This is achieved by using a set of known 3D points in the world and their corresponding 2D projections observed on the camera’s image sensor. ...

Given a starting point, starting direction, ending point, and ending direction, the goal is to generate a feasible and “natural” path that connects the two points. This path should adhere to vehicle dynamics and avoids obstacles. However, it is hard to define what is “natural”. Path Planning In this section, we review some classical path planning methods. We ignore the algorithmic details and focus on the resulting path shapes to provide an overview. The demonstration images are generated using the code repository [1]. ...

In this post, we will discuss the coordinate systems commonly used in autonomous driving. In practice, different positioning providers may define their own coordinate systems. This post will introduce the fundamental concepts of these coordinate systems and explain how to convert between them for your specific needs. Pose on the Earth When we talk about pose, we refer to the position and orientation of an object in the world. In the context of autonomous driving, the world is the Earth. Therefore, pose describes the position and orientation of an object relative to the Earth. ...

This article develops end-of-line (EOL) camera extrinsic calibration and calibration-board placement for autonomous-vehicle camera rigs, including bird’s-eye-view applications. There are two linked optimization problems: estimate camera and board poses from measurements, then choose physical board positions and orientations that make the camera estimates more precise. Camera intrinsics are assumed known, and absolute extrinsics require a world-frame anchor. The presentation follows this order: General calibration problem formulation: variables, coordinate conventions, measurements, priors, and identifiability. Solver: sparse factor-graph bundle adjustment for the pose estimates. Board-position and orientation optimization formulation: camera uncertainty, design variables, constraints, and optimization methods. Experiments from simple to complex: one camera and one board, two-camera cases, a multi-camera workspace, and a vehicle rig. Geometric preprocessing, ray-plane derivations, and optional design formulations are collected in appendices. The detailed ADMM formulation and solver benchmarks remain in the independent comparison. ...