In Autonomous Vehicles (AVs), Multi-Sensor Fusion (MSF) is used to combine perception results from multiple sensors such as LiDARs (Light Detection And Ranging) and cameras to achieve overall higher accuracy and robustness. In this work, we design MSF-ADV, the first attack that can fundamentally defeat MSF-based AV perception by generating 3D adversarial objects. In this website, we provide:

• Overview of our attack design

• Visualization of the generated 3D adversarial objects

• Video demonstrations of the attack effectiveness in both industry-grade autonomous vehicle simulator and the physical world

• More detailed information about the project such as write-ups, project team, and acknowledgements

The figure below illustrates the attack pipeline of MSF-ADV. First, the attacker follows our novel MSF-ADV attack method (more details in our research write-up) to generate a 3D adversarial object, print it using a 3D printer, and places it in the middle of the road. Next, the victim autonomous vehicle drives towards such adversarial object, and its LiDAR and camera sensors collect the corresponding adversarial point cloud and adversarial image. Due to our attack design, the adversarial point cloud and image will fool the DNN-based object detection models for both LiDAR and camera. Such adversarial object thus becomes completely invisible to the victim autonomous vehicle no matter what MSF algorithms are used, which thus fundamentally defeats any types of MSF algorithms. The victim autonomous vehicle thus will ignore such object and directly crash into it.

Visualization for Adversarial & Benign Cube

The figure below visualizes both adversarial and benign cube from different angles to illustrate their differences.

In the section, we present video demos showing the end-to-end attack effect on an AV system, Baidu Apollo, in an industry-grade autonomous vehicle simulator LGSVL. Apollo is a representative industry-grade autonomous driving system that is open source and already providing self-driving taxi services in China. The LGSVL simulator provides the required live sensors data for Apollo and the Apollo sends back the decision and control to the LGSVL. In these demos, we use LGSVL 2019.11 and Apollo-5.0.

The two video demos below are generated when both the LiDAR- and camera-based object detections are used in Baidu Apollo with its MSF-based perception process:

• Benign Traffic Cone (left): As shown, at the 12th second the autonomous vehicle is able to correctly detect the benign traffic cone and thus comes to a full stop before the cone at the 14th second.

• Adversarial Traffic Cone (right): As shown, at the 12th second, the autonomous vehicle fails to detect the adversarial traffic cone and thus directly crashes into it.

When Only LiDAR is Used

The two video demos below show when only LiDAR-based object detection is used in the Baidu Apollo with its MSF-based perception process:

• Benign Traffic Cone (left). As shown, at the 12th second the autonomous vehicle is able to correctly detect the benign traffic cone and thus comes to a full stop before the cone at the 14th second.

• Adversarial Traffic Cone (right). As shown, at the 12th second, the autonomous vehicle fails to detect the adversarial traffic cone on LiDAR-based object detection and thus directly crashes into it.

When Only Camera is Used

The two video demos below show when only camera-based object detection is used in the Baidu Apollo with its MSF-based perception process:

• Benign Traffic Cone (left). As shown, at the 11th second the autonomous vehicle is able to correctly detect the benign traffic cone and thus comes to a full stop before the cone at the 15th second.

• Adversarial Traffic Cone (right). As shown, at the 10th second, the autonomous vehicle fails to detect the adversarial traffic cone on camera-based object detection and thus directly crashes into it.

In the section, we present video demos showing the physical-world experiments on Baidu Apollo with a real autonomous vehicle. Our preliminary experiments are performed when only LiDAR is used. To avoid damaging the vehicle, a driver is sitting in the autonomous vehicle to prevent the actual collision.

Experiment Settings

The figures below illustrate the settings of our physical-world experiments. As shown, it includes the adversarial object we generated and 3D-printed, a benign object, the autonomous vehicle, and the physical-world road we used for the experiments.

Adversarial & Benign Object in the Middle of the Road

The two video demos below are generated when the LiDAR-based object detection is used in Baidu Apollo with its MSF-based perception process:

• Benign Object (left). As shown, the LiDAR-based object detection can always detect the benign object.

• 3D-Printed Adversarial Object (right). As shown, the adversarial object is rarely detected.

Adversarial & Benign Object on the Right of the Road

The two video demos below show the detection results from different angles (e.g., from the left side of the object) to demonstrate the robustness of our attack:

• Benign Object (left). As shown, the LiDAR-based object detection can always detect the benign object.

• 3D-Printed Adversarial Object (right). As shown, the adversarial object is never detected.

In the section, we present video demos showing the MSF-based perception results on Apollo by driving manually in the LGSVL simulator. We construct multiple driving scenarios (e.g., drive straight forward, drive forward from the left, etc.) to demonstrate the robustness of our attack. Note that due to the delay between the perception output and its visualization, some results may not be synchronized well.

For the demos in this section, the benign cases (videos on the left) show that the MSF-based perception in Apollo can always detect the benign traffic cone, while the adversarial cases (videos on the right) show that the adversarial traffic cone is never detected.

Drive Forward

The two video demos below show the MSF-based perception results in Apollo when driving the car straight forward to the traffic cone:

• Benign Traffic Cone (left). As shown, at the 7th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Backward

The two video demos below show the MSF-based perception results in Apollo when driving the car straight backward from the traffic cone:

• Benign Traffic Cone (left). As shown, until the 11th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Forward from Left

The two video demos below show the MSF-based perception results in Apollo when driving the car forward from the left side to the traffic cone:

• Benign Traffic Cone (left). As shown, at the 6th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Backward from Left

The two video demos below show the MSF-based perception results in Apollo when driving the car backward on the left side from the traffic cone:

• Benign Traffic Cone (left). As shown, until the 11th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Forward from Right

The two video demos below show the MSF-based perception results in Apollo when driving the car forward from the right side to the traffic cone:

• Benign Traffic Cone (left). As shown, at the 8th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Backward from Right

The two video demos below show the MSF-based perception results in Apollo when driving the car backward on the right side from the traffic cone:

• Benign Traffic Cone (left). As shown, until the 16th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Around from Left

The two video demos below show the MSF-based perception results in Apollo when driving the car around the traffic cone from the left side:

• Benign Traffic Cone (left). As shown, at the 4th second the autonomous vehicle is able to correctly detect the benign traffic cone.

• Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Drive Randomly

The two video demos below show the MSF-based perception results in Apollo when driving the car randomly turning left and right towards the traffic cone:

Benign Traffic Cone (left). As shown, at the 13th second the autonomous vehicle is able to correctly detect the benign traffic cone.

Adversarial Traffic Cone (right). As shown, the autonomous vehicle can never detect the adversarial traffic cone.

Research Write-up

[MLSys'20 Demo] 3D Adversarial Object against MSF-based Perception in Autonomous Driving

Yulong Cao* , Ningfei Wang*, Chaowei Xiao*, Dawei Yang*, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li (*contributed equally)

Third Conference on Machine Learning and Systems (MLSys), Mar 2020

[Demo Paper] [Demo Poster] [Demo Website]

Acknowledgements