Skip to main content

Table 3 Ablation study result scores on the validation set

From: Object detection for automotive radar point clouds – a comparison

Method

IOU=0.3

IOU=0.5

 

APped

APgrp

APcyc

APcar

APtrk

mAP

mLAMR

F1,pt

F1,obj

APped

APgrp

APcyc

APcar

APtrk

mAP

mLAMR

F1,pt

F1,obj

PointNet++ Reference

33.58

51.46

52.15

53.12

38.98

45.86

59.94

58.07

52.62

30.68

48.49

51.16

46.08

32.16

41.72

63.57

58.06

49.90

PointNet++ Normal DBSCAN

31.78

50.17

53.66

50.38

29.66

43.13

62.61

57.49

50.40

30.09

44.56

52.04

44.25

25.60

39.31

64.99

57.45

48.22

PointNet++ Normal Filter

28.70

50.66

52.46

52.32

44.60

45.75

59.66

54.43

53.36

25.67

42.81

49.13

42.79

35.03

39.09

65.24

54.15

48.90

LSTM Reference

22.19

48.41

61.32

63.07

66.98

52.39

52.67

55.29

59.61

21.63

45.95

61.29

56.58

60.03

49.10

54.69

54.95

58.15

LSTM OVO Ranking

22.91

41.72

61.94

65.55

57.65

49.95

53.66

54.21

58.74

22.86

40.51

61.94

63.71

54.62

48.73

55.29

54.21

57.45

DBSCAN/RandomForest

19.17

41.71

49.86

57.70

59.87

45.66

59.50

50.85

53.11

19.14

39.57

49.81

51.24

58.47

43.65

61.47

50.85

51.70

LSTM++ Reference

28.90

58.81

58.16

63.53

67.32

55.34

50.47

56.50

61.46

27.55

54.39

58.16

60.92

62.81

52.77

52.74

56.43

59.84

LSTM++ OVO Ranking

30.45

56.35

58.37

63.43

67.25

55.17

50.83

56.03

61.73

29.39

52.41

58.37

60.95

62.57

52.74

52.87

56.08

60.24

LSTM PointNet++ Clustering

25.77

50.68

42.87

40.81

58.81

43.79

62.55

49.27

50.19

25.49

44.18

41.81

34.48

53.43

39.88

65.36

49.41

47.93

YOLOv3 Reference

37.04

57.77

55.89

69.12

64.83

56.93

51.37

56.23

61.13

35.04

54.66

55.76

62.89

56.63

53.00

54.59

55.56

58.79

YOLOv3 No Scale

29.95

58.69

56.33

69.18

64.73

55.78

51.67

54.88

60.63

31.21

56.22

54.73

62.22

58.15

52.51

55.12

54.76

58.29

YOLOv3 No Blurr

38.00

58.25

54.46

67.73

63.16

56.32

52.37

55.44

61.16

35.97

55.51

53.46

61.56

56.49

52.60

55.57

55.27

58.85

YOLOv3 No Scale No Blurr

36.70

59.23

55.68

68.46

61.13

56.24

52.14

54.73

61.23

36.09

55.30

54.13

61.61

53.68

52.16

55.52

54.82

58.80

YOLOv3 Rotated Boxes

34.44

55.71

57.35

70.24

61.89

55.93

51.42

55.06

62.02

31.22

50.70

54.18

61.83

55.06

50.60

56.33

54.46

58.19

PointPillars Reference

14.82

27.94

40.50

57.80

54.87

39.19

67.10

48.69

44.90

14.33

26.25

39.05

51.35

50.63

36.32

69.43

48.69

42.91

PointPillars Rotated Boxes

11.15

22.52

38.81

54.16

53.68

36.06

70.26

48.43

40.50

5.29

21.73

38.39

50.36

51.93

33.54

71.03

48.67

39.68

PointPillars++

23.39

44.52

48.75

63.92

57.53

47.62

58.92

55.99

54.37

22.59

41.59

48.10

58.83

53.64

44.95

61.68

56.10

52.25

  1. For all scores except mLAMR higher means better. LSTM++ denotes the combined LSTM method with PointNet++ filtering during clustering. Bold font indicates the best experiment variant of each ablation study