In this blog post, we will look at the principles behind how 360-degree cameras create images of the vehicle’s surroundings and correct distortion to help the driver’s safety.
There are various devices that help drivers when parking or driving on narrow roads. Among these, the device that uses images taken by cameras mounted on the front, rear, and sides of the vehicle to create a 360-degree view of the vehicle’s surroundings and provides it to the driver through the monitor in the vehicle is particularly noteworthy. This device helps the driver to drive and park safely by allowing them to grasp the surrounding situation at a glance. Now, let’s take a look at how these images are provided to the driver.
First, a grid of checkerboards is laid out on the floor around the vehicle and photographed by a camera. The cameras used in this device are usually equipped with wide-angle lenses, which provide a large field of view and help the driver to secure a clear view by reducing blind spots. However, wide-angle lenses have the problem of image distortion due to the inherent curvature that occurs when light passes through the lens. The center of the image is convex, and the further away from the center, the more curved it becomes, which is called the distortion of the image caused by the lens. The characteristics of the camera itself that affect this distortion are called internal variables, and they are expressed as distortion coefficients. If the internal variables are accurately identified, distortion can be corrected by setting a distortion model.
The process of correcting distortion requires a very sophisticated operation. Only when the distortion in the video taken by the camera is minimized will the video that the driver sees be as close as possible to the actual situation. To do this, a distortion correction algorithm is used, and in this process, the characteristics of the lens, as well as the position and angle of the camera mounted on the vehicle, play an important role. The cause of distortion caused by the tilt of the camera mounted on the vehicle is called an external variable. By comparing the captured image with the real-world grating, the camera’s tilt angle can be determined by the angle at which the grating rotates in the image or the change in the grating’s position. Based on this, the distortion can be corrected by modifying the external variable.
Once the distortion correction is complete, the next step is to estimate the 3D real-world points corresponding to the points in the image, and then perform perspective transformation to obtain an image with the perspective effect removed. Generally, when a camera projects the 3D real world onto a 2D image, objects of the same size appear smaller the farther they are from the camera. However, it is important to remove this perspective effect because the image from the point of view from above should not have any changes in the size of objects depending on the distance.
If the positions of a few points in the image obtained by the viewpoint transformation and the corresponding points on the real-world grating are known, the correspondence between all the points in the image and the points on the grating can be described using a virtual coordinate system. Using this relationship, if the dots of the image are placed on a plane so that the shape of the grid and the relative size between the grids remain the same as in the real world, it will appear as a two-dimensional image. The image obtained in this way is the image from the top-down view. By synthesizing images from each direction in this way, the driver can see a 360° image on the monitor as if looking down on the vehicle from above.
The technology used in this process is very complex and precise, but the results are very helpful to the driver. In particular, in narrow parking spaces or complex road conditions, these devices play an important role in ensuring the safety of the driver. This technological advancement is greatly improving the safety and convenience of driving a vehicle and will be used as an important basic technology for the development of autonomous vehicles in the future.
