BEVHeight++: Toward Robust Visual Centric 3D Object Detection

Lei Yang · Tao Tang · Jun Li · Kun Yuan · Peng Chen · Li Wang · Yi Huang · Lei Li · Xinyu Zhang · Kaicheng Yu

BEVHeight++ is a new vision-based 3D object detector specially designed for both roadside and vihicle-side scenarios. On popular 3D detection benchmarks of roadside cameras, BEVHeight++ surpasses all previous vision-centric methods by a significant margin. In terms of the ego-vehicle scenario, our BEVHeight++ also possesses superior over depth-only methods.

News

• [2025/02/03] Release the pre-trained models!
• [2024/09/06] Both arXiv and codebase are released!

Table of Contents

1. Getting Started
2. Acknowledgment
3. Citation

Getting Started

• Installation
• Prepare Dataset

Train BEVDepth / BEVHeight / BEVHeight++ with 8 GPUs

python [EXP_PATH] --amp_backend native -b 8 --gpus 8

Eval BEVDepth / BEVHeight / BEVHeight++ with 8 GPUs

python [EXP_PATH] --ckpt_path [CKPT_PATH] -e -b 8 --gpus 8

• For more specific training and evaluation commands, please refer to the train_scripts directory.

• For experiments on nuScenes dataset, please refer to the nuscense_bevheight_plus directory and this document .

Experimental Results

• DAIR-V2X-I Dataset

Method	Config	Car			Pedestrain			Cyclist
		[email protected]			[email protected]			[email protected]
		Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
BEVDepth	config	75.50	63.58	63.67	34.95	33.42	33.27	55.67	55.47	55.34
BEVHeight	config	77.48	65.46	65.53	41.22	39.29	39.46	60.23	60.08	60.54
BEVHeight++	config	79.31	68.62	68.68	42.87	40.88	41.06	60.76	60.52	61.01

• Rope3D Dataset

Method	Config	Car | [email protected]		Big Vehicle | [email protected]		Car | [email protected]		Big Vehicle | [email protected]
		AP	Rope	AP	Rope	AP	Rope	AP	Rope
BEVDepth	config	69.63	74.70	45.02	54.64	42.56	53.05	21.47	35.82
BEVHeight	config	74.60	78.72	48.93	57.70	45.73	55.62	23.07	37.04
BEVHeight++	config	76.12	80.91	50.11	59.92	47.03	57.77	24.43	39.57

• KITTI Dataset

Method	AP|3D			AP|BEV			Config	model ckpt
	Easy	Mod.	Hard	Easy	Mod.	Hard
BEVDepth	10.69	7.31	5.88	35.14	23.22	19.33	config	\
BEVHeight	10.61	6.97	5.51	34.58	22.05	17.96	config	\
BEVHeight++	11.37	8.06	6.35	36.81	25.49	20.81	config	model_ckpt

• KITTI-360 Dataset

Method	AP3D (IoU=0.5)			AP3D (IoU=0.25)			config	model ckpt
	AP (Lrg)	AP (Car)	mAP	AP (Lrg)	AP (Car)	mAP
BEVDepth	2.17	42.01	22.09	30.52	59.84	45.18	config	model_ckpt
BEVHeight	1.78	41.76	21.77	33.02	56.69	44.85	config	model_ckpt
BEVHeight++	2.35	46.84	24.59	30.72	65.78	48.25	config	model_ckpt

• Waymo Dataset

• BEVDepth: config, model_ckpt
• BEVHeight: config, model_ckpt
• BEVHeight++: config, model_ckpt

IoU3D	Difficulty	Method	AP3D				APH3D
			All	0-30	30-50	50- $\infty$	All	0-30	30-50	50- $\infty$
0.7	Level_1	BEVDepth	2.86	7.51	1.13	0.12	2.84	7.45	1.11	0.12
		BEVHeight	2.62	6.78	1.35	0.08	2.60	6.71	1.33	0.08
		BEVHeight++	3.10	8.04	1.41	0.13	3.07	7.96	1.40	0.13
0.7	Level_2	BEVDepth	2.68	7.50	1.09	0.10	2.66	7.43	1.08	0.10
		BEVHeight	2.46	6.77	1.30	0.07	2.44	6.70	1.29	0.07
		BEVHeight++	2.91	8.03	1.36	0.12	2.88	7.95	1.35	0.11
0.5	Level_1	BEVDepth	13.22	31.67	6.01	1.43	13.05	31.27	5.93	1.41
		BEVHeight	12.73	29.87	5.90	1.33	12.55	29.45	5.83	1.30
		BEVHeight++	14.00	32.59	6.31	1.86	13.81	32.16	6.25	1.83
0.5	Level_2	BEVDepth	12.41	31.61	5.81	1.25	12.25	31.21	5.73	1.23
		BEVHeight	11.95	29.81	5.70	1.16	11.78	29.40	5.63	1.14
		BEVHeight++	13.14	32.53	6.10	1.62	12.97	32.10	6.04	1.59

Acknowledgment

This project is not possible without the following codebases.

• BEVHeight
• BEVDepth
• DAIR-V2X
• BEVDet
• mmdetection3d
• pypcd
• SeaBird

Citation

If you use BEVHeight++ in your research, please cite our work by using the following BibTeX entry:

@article{yang2023bevheight++,
  title={Bevheight++: Toward robust visual centric 3d object detection},
  author={Yang, Lei and Tang, Tao and Li, Jun and Chen, Peng and Yuan, Kun and Wang, Li and Huang, Yi and Zhang, Xinyu and Yu, Kaicheng},
  journal={arXiv preprint arXiv:2309.16179},
  year={2023}
}