Active Obstacle Detection System Based on Video Recognition and Lidar Information Fusion

In terms of the requirements for obstacle detection in the rail transit application field, an architecture and implementation method for active obstacle detection system based on the fusion of video recognition and lidar information is proposed. The studies on the video recognition algorithms based on deep learning neural network and lidar for orbit area recognition, pedestrian vehicle recognition

Traditionally, the driver is responsible for looking ahead. When an obstacle appears in front of the driver, he will adopt braking measures timely to avoid the collision accident. However, because the driver is prone to fatigue after driving for a long time, which may cause problems such as slow response and negligence, there is still the possibility of accidents due to that there is no time to deal with the intrusion.
With the continuous maturity of autonomous driving technology and the improvement of system reliability, "fully automatic driverless driving" has become the main goal of the metro control system. When there is no longer a driver in the driver's cabin to keep lookout, there must be a system that can replace the driver's eyes, timely sense whether there are obstacles in the space boundary in front of the traffic, make accurate identification, respond in time, and control the braking and deceleration of the train so as to prevent collision accidents. How to design and implement such an obstacle detection system is the subject to be studied and solved in this paper.

Current situation and problems
At present, in the rail transit industry, many relevant companies and universities have carried out relevant studies on the demand for obstacle detection. The achievements mainly include two categories, i.e., "passive" and "active". Currently, "passive obstacle detection system" has been applied in some metro lines, where the "detection rod" is installed in front of the wheel set at a certain height from the rail. When the detection rod touches the obstacle ahead, it pushes the associated sensor and timely triggers the train "emergency braking" to prevent the accident from further expanding. The system is limited to passive and contact detection of obstacles in front of the vehicle. When the system detects an obstacle, the vehicle has touched the obstacle and the collision has already happened. It can only play a role in reducing the degree of accident damage but cannot prevent the accident in advance.
Another idea is "active obstacle detection". It generally adopts sensors such as camera, millimeter wave radar, and lidar for non-contact detection of obstacles within a certain distance in front of the vehicle and makes response in time. At present, in the field of automatic driving obstacle recognition, the main sensors adopted include camera, millimeter wave radar, ultrasonic radar, lidar, etc. The detection range and working characteristics of these sensors are different (see Tab. 1), which need to be selected according to the characteristics of the application scenarios of rail transit. and "dense fog".
Lidar has the ability to scan quickly, and generally  Millimeter wave radar has been widely used in the automotive field, and also has some application cases in the rail transit field. Forward detection radar generally adopts 77 GHz. In recent years, millimeter wave radar has a longer detection range with the cost falling rapidly. Its resolution is not comparable to that of lidar, which is easy to be interfered with by clutter and thus produces false alarms. Millimeter wave radar is not affected by light conditions. However, heavy rain will interfere with its operation.
Ultrasonic radar is mainly used for short-range

Research process and key technologies
For the identification of "foreign object intrusion boundary", the critical first step is to accurately identify the "track area". The second step is to identify the targets near the boundary of the track area. The third step is to determine whether there is a foreign object invading the boundary and determine whether alarm and related trigger actions are required according to the relevant rules.
In order to achieve the above goals, we have tried many methods. Initially, we adopt simple image processing algorithms to perform operations such as "edge search" and "Hough transform" on the image to extract and match its features, by which part of the track area on the image can be identified, as shown in         At present, the lidar host used in autonomous driving are mainly two types, i.e., "mechanical scanning" lidar and "solid state" lidar. The mechanical scanning lidar turns the laser beam from "line" to "plane" by continuously rotating the transmitter head, and arranges multiple laser beams in the vertical direction (i.e., 32 or 64 line radar) to form multiple surfaces to achieve the purpose of dynamic 3D scanning.
However, it has the shortcomings of "big, heavy, and We use high-density MEMS solid-state lidar as the sensor of this system to ensure its long service life and good shock resistance. With an angular resolution of 0.1º, fine point cloud data can be formed for the obstacle perception. For objects with the reflectivity of 50%, the detection distance can be up to 300 m.
In order to make the lidar and the camera data coordinate system coincide, coordinate joint calibration should be carried out. The result of the coordinate calibration is to project the 3D coordinate system (xw, yw, zw), where the lidar is located, onto the 2D discrete coordinate system (u, v), where the image is located. The transformation from 3D to 2D coordinates has undergone processes such as rotation and translation, projection, and coordinate dispersion.
New Metro (2020) 1(1): 11-21 Yaodong (2020) 19 Figure 14. MEMS mechanisms. The unremitting pursuit of "fully automated and unmanned driving" and "smart transportation" in the rail transit field has given birth to the extensive application of artificial intelligence technology in many fields related to rail transit. The development of deep learning and neural network algorithms is changing with each passing day, the accuracy rate is New Metro (2020) 1(1): 11-21 Yaodong (2020) 21 constantly improving, and the real-time performance of calculations is also constantly improving. The large-scale localization of lidar has greatly reduced the price of lidar devices, making it more acceptable for customers. It is foreseeable that the active obstacle detection system described in this paper can be used as a beneficial supplement to the current communication based train control (CBTC) system, to equip vehicles with their own "eyes", especially for "driverless" application scenarios. The system will replace the lookout of drivers, undertake the tasks of obstacle detection and collision avoidance, and become an important active safety protection system.