Posts

Given a starting point, starting direction, ending point, and ending direction, the goal is to generate a feasible and “natural” path that connects the two points. This path should adhere to vehicle dynamics and avoids obstacles. However, it is hard to define what is “natural”. Path Planning In this section, we review some classical path planning methods. We ignore the algorithmic details and focus on the resulting path shapes to provide an overview. The demonstration images are generated using the code repository [1]. ...

In this post, we will discuss the Kalman filters from two perspectives: classical parameter estimation and Bayesian estimation. Each perspective provides a unique way to derive the Kalman filter. Parameter estimation is more flexible as it allows you to easily add constraints, revise the transition and observation models, and derive other related smoothing and filtering methods. Meanwhile, Bayesian estimation is more intuitive and provides a clearer probabilistic interpretation, helping us understand the underlying principles better. By examining both approaches, we can gain a more comprehensive understanding of how Kalman filters work. ...

I have taken some courses related to reinforcement learning during my Ph.D. study. However, I have not touched it for a long time. Recently, I am working on end-to-end autonomous driving and the success of reinforcement learning in large language models brings me back to this topic. In this post, we will introduce the basic concepts and commonly used algorithms in reinforcement learning. More detailed information can be found in the reference book [1] and OpenAI RL page. ...

This post is an application of the EOL calibration described in the EOL calibration article. Detect Image Corners Detecting the corners of the fisheye images on its original image is challenging due to its severe distortion. So we resort to detecting the corners on the BEV image and then project the corners back onto the original image to further refine the corners. Set initial extrinsics (referring to installation parameters (angles and positions) or parameters from joint calibration with LiDAR), construct a 20m×20m grid with resolution of 0.01m in the ego coordinate system’s ground plane (z=0), and generate a BEV projected image; ...

In this post, we will discuss the coordinate systems commonly used in autonomous driving. In practice, different positioning providers may define their own coordinate systems. This post will introduce the fundamental concepts of these coordinate systems and explain how to convert between them for your specific needs. Pose on the Earth When we talk about pose, we refer to the position and orientation of an object in the world. In the context of autonomous driving, the world is the Earth. Therefore, pose describes the position and orientation of an object relative to the Earth. ...

This article develops end-of-line (EOL) camera extrinsic calibration and calibration-board placement for autonomous-vehicle camera rigs, including bird’s-eye-view applications. There are two linked optimization problems: estimate camera and board poses from measurements, then choose physical board positions and orientations that make the camera estimates more precise. Camera intrinsics are assumed known, and absolute extrinsics require a world-frame anchor. The presentation follows this order: General calibration problem formulation: variables, coordinate conventions, measurements, priors, and identifiability. Solver: sparse factor-graph bundle adjustment for the pose estimates. Board-position and orientation optimization formulation: camera uncertainty, design variables, constraints, and optimization methods. Experiments from simple to complex: one camera and one board, two-camera cases, a multi-camera workspace, and a vehicle rig. Geometric preprocessing, ray-plane derivations, and optional design formulations are collected in appendices. The detailed ADMM formulation and solver benchmarks remain in the independent comparison. ...

I highly recommend reading the two papers: [5] and [3]. [5] provides a comprehensive description of the splatting process. [3] provides an efficient implementation of the splatting process. Problem Formulation Volume Rendering with Radiance Fields [1, 2] figure from https://www.cs.cornell.edu/courses/cs5670/2022sp/lectures/lec22_nerf_for_web.pdf. The volume density $\sigma(\mathbf{x})$ can be interpreted as the differential probability of a ray terminating (being absorbed or scattered) at an infinitesimal particle at location $\mathbf{x}$. ...

In this document, we provide detailed derivations complementing those presented in [2]. Details of the Derivation Discrete Model Based on the $\boxplus$ operation defined above, we can discretize the continuous model in (1) at the IMU sampling period $\Delta t$ using a zero-order holder. The resultant discrete model is $$ \mathbf{x}_{i+1} = \mathbf{x}_i \boxplus (\Delta t f(\mathbf{x}_i, \mathbf{u}_i, \mathbf{w}_i))$$where $i$ is the index of IMU measurements, and the function $f$, state $\mathbf{x}$, input $\mathbf{u}$, and noise $\mathbf{w}$ are defined below: ...

In this post, we will discuss the Iterative Closest Point (ICP) problem: from point-to-point and point-to-plane ICP to generalized ICP. Problem Formulation Let two 3D point-sets $\mathcal{X} = \{\mathbf{x}_i\}, i = 1, \ldots, N$ and $\mathcal{Y} = \{\mathbf{y}_j\}, j = 1, \ldots, M$, where $\mathbf{x}_i, \mathbf{y}_j \in \mathbb{R}^3$ are point coordinates, be the data point-set and the model point-set respectively. The goal is to estimate a rigid motion with rotation $\mathbf{R} \in SO(3)$ and translation $\mathbf{t} \in \mathbb{R}^3$ that minimizes the following $L_2$-error $E$: ...

System Diagram See Appendix.VII Figure 1: System diagram of a typical 4D mmWave radar signal processing chain (figure from [1]) Single Object Tx-Rx Model Below is a mathematical formalization of each major step in the traditional 4D mmWave Frequency Modulated Continuous Wave (FMCW) radar signal processing chain, from transmitted signals through to point-cloud generation. Please note that these equations represent a general framework; actual implementations may vary slightly depending on specific system parameters and design choices. ...