Skip to content

Commit

Permalink
fixup! Add package to evaluate the impact of the beam skipping featur…
Browse files Browse the repository at this point in the history 
…e in nav2
  • Loading branch information
@glpuga
glpuga committed Dec 5, 2024
1 parent 2818971 commit 816ba4b
Show file tree
Hide file tree
Showing 3 changed files with 139 additions and 9 deletions.
4 changes: 2 additions & 2 deletions src/benchmarks/beam_skipping_evaluation/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ cd {LAMBKIN_ROOT_DIR}
```
- Clone `beluga-datasets`. **This repository uses LFS**, so make sure you have it installed.
```bash
cd src/benchmarks/beam_skipping_evaluation \
&& mkdir -p playground && cd playground
cd src/benchmarks/beam_skipping_evaluation
mkdir -p playground && cd playground
```
- Copy in that folder the datasets from the `/srv/datasets/beluga_evaluation_datasets` directory in the beefy machine.
```sh
Expand Down
3 changes: 3 additions & 0 deletions src/benchmarks/beam_skipping_evaluation/launch/beam_skipping_evaluation.launch
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@

<param name="robot_model_type" value="$(var robot_model_type)" />

<!-- we use this one for reference in case of failures -->
<param name="tf_broadcast" value="true" />

<remap from="amcl_pose" to="/nav2_amcl_likelihood/pose" />
<remap from="particle_cloud" to="/nav2_amcl_likelihood/particle_cloud" />
</node>
Expand Down
141 changes: 134 additions & 7 deletions src/benchmarks/beam_skipping_evaluation/scripts/nominal.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,143 @@ Suite Setup Setup Nav2 Beam Skipping Evaluation benchmark suite
Suite Teardown Teardown Nav2 Beam Skipping Evaluation benchmark suite
Test Template Run Nav2 Beam Skipping Evaluation benchmark case for each ${dataset} ${basedir} ${odom_frame} ${map_frame} ${base_frame} ${scan_topic} ${iterations} ${initial_pose_x} ${initial_pose_y} ${initial_pose_yaw} ${robot_model_type}


*** Variables ***

@{DATASETS_DIR} simulated_datasets
@{DIFF_DRIVE_SIM_BAGS} simulated_hq_kobuki
@{OMNI_DRIVE_SIM_BAGS} simulated_bookstore_robomaster
@{MAGAZINO_DIR} magazzino_ros2_localization_only
@{MAGAZINO_BAGS} hallway_localization
... hallway_return

@{OPENLORIS_DIR} openloris_office_ros2_localization_only
@{OPENLORIS_OFFICE_BAGS} office1-1
... office1-2
... office1-3
... office1-4
... office1-5
... office1-6
... office1-7

${TORWIC_MAPPING_DIR} torwic_mapping_dataset_ros2_localization_only
@{TORWIC_MAPPING_BAGS} baseline
... 1-1_top_row_shift
... 1-2_larger_shift
... 1-3_longitudinal_shift
... 1-4_all_boxes_shift
... 1-5_all_boxes_rotate
... 1-6_inner_shrink
... 1-7_all_shrink
... 1-8_alternating_in_and_out
... 1-9_stepping_outward
... 2-1_remove_top_row
... 2-2_remove_corners
... 2-3_remove_all_boxes
... 2-4_remove_fences
... 3-1_top_row_shift_with_fences
... 3-2_top_row_shift_with_sides
... 3-3_fences_move_outward
... 4-1_cover_the_fences_with_boxes

${TORWIC_SLAM_DIR} torwic_slam_dataset_ros2_localization_only
@{TORWIC_SLAM_BAGS} 2022-06-15_aisle_ccw_run_1
... 2022-06-15_aisle_ccw_run_2
... 2022-06-15_aisle_cw_run_1
... 2022-06-15_aisle_cw_run_2
... 2022-06-15_hallway_full_cw_parts_1_and_2
... 2022-06-15_hallway_full_ccw_parts_1_and_2
... 2022-06-15_hallway_straight_ccw_parts_1_and_2
... 2022-06-23_aisle_ccw_run_1
... 2022-06-23_aisle_ccw_run_2
... 2022-06-23_aisle_cw_run_1
... 2022-06-23_aisle_cw_run_2
... 2022-06-23_hallway_full_cw_parts_1_and_2
... 2022-06-23_hallway_straight_ccw_parts_1_and_2
... 2022-06-23_hallway_straight_cw_parts_1_and_2
... 2022-10-12_aisle_ccw
... 2022-10-12_aisle_cw
... 2022-10-12_hallway_full_cw_run1
... 2022-10-12_hallway_full_cw_run2
... 2022-10-12_hallway_straight_ccw
... 2022-10-12_hallway_straight_cw

${WILLOW_DSET_DIR} willow_garage_dataset_ros2_localization_only
# # 2011-08-03-16-16-43 is not in the list because it's missing the scan topic
@{WILLOW_DSET_BAGS} 2011-08-03-20-03-22
... 2011-08-04-12-16-23
... 2011-08-04-14-27-40
... 2011-08-04-23-46-28
... 2011-08-05-09-27-53
... 2011-08-05-12-58-41
... 2011-08-05-23-19-43
... 2011-08-08-09-48-17
... 2011-08-08-14-26-55
... 2011-08-08-23-29-37
# ... 2011-08-09-08-49-52
# ... 2011-08-09-14-32-35
# ... 2011-08-09-22-31-30
# ... 2011-08-10-09-36-26
# ... 2011-08-10-14-48-32
# ... 2011-08-11-01-31-15
# ... 2011-08-11-08-36-01
# ... 2011-08-11-14-27-41
# ... 2011-08-11-22-03-37
# ... 2011-08-12-09-06-48
# ... 2011-08-12-16-39-48
# ... 2011-08-12-22-46-34
# ... 2011-08-15-17-22-26
# ... 2011-08-15-21-26-26
# ... 2011-08-16-09-20-08
# ... 2011-08-16-18-40-52
# ... 2011-08-16-20-59-00
# ... 2011-08-17-15-51-51
# ... 2011-08-17-21-17-05
# ... 2011-08-18-20-33-16
# ... 2011-08-18-20-52-30
# ... 2011-08-19-10-12-20
# ... 2011-08-19-14-17-55
# ... 2011-08-19-21-35-17
# ... 2011-08-22-10-02-27
# ... 2011-08-22-14-53-33
# ... 2011-08-23-01-11-53
# ... 2011-08-23-09-21-17
# ... 2011-08-24-09-52-14
# ... 2011-08-24-15-01-39
# ... 2011-08-24-19-47-10
# ... 2011-08-25-09-31-05
# ... 2011-08-25-20-14-56
# ... 2011-08-25-20-38-39
# ... 2011-08-26-09-58-19
# ... 2011-08-29-15-48-07
# ... 2011-08-29-21-14-07
# ... 2011-08-30-08-55-28
# ... 2011-08-30-20-49-42
# ... 2011-08-30-21-17-56
# ... 2011-08-31-20-29-19
# ... 2011-08-31-20-44-19
# ... 2011-09-01-08-21-33
# ... 2011-09-02-09-20-25
# ... 2011-09-06-09-04-41
# ... 2011-09-06-13-20-36
# ... 2011-09-08-13-14-39
# ... 2011-09-09-13-22-57
# ... 2011-09-11-07-34-22
# ... 2011-09-11-09-43-46
# ... 2011-09-12-14-18-56
# ... 2011-09-12-14-47-01
# ... 2011-09-13-10-23-31
# ... 2011-09-13-13-44-21
# ... 2011-09-14-10-19-20
# ... 2011-09-15-08-32-46

@{LONG_DURATION_DIR} long_duration_bags_ros2_localization_only
@{OMNI_DRIVE_SIM_BAGS} simulated_bookstore_robomaster_24hs

*** Test Cases *** DATASET BASEDIR ODOM_FRAME MAP_FRAME BASE_FRAME SCAN_TOPIC ITERATION INITIAL_POSE_X INITIAL_POSE_Y INITIAL_POSE_YAW ROBOT_MODEL_TYPE
Magazino Datasets ${{MAGAZINO_BAGS}} ${{MAGAZINO_DIR}} odom map base_footprint /scan_front 1 0.0 0.0 0.0 nav2_amcl::DifferentialMotionModel
Openloris Office ${{OPENLORIS_OFFICE_BAGS}} ${{OPENLORIS_DIR}} base_odom map base_link /scan 5 0.0 0.0 0.0 nav2_amcl::DifferentialMotionModel
TorWIC Mapping ${{TORWIC_MAPPING_BAGS}} ${{TORWIC_MAPPING_DIR}} odom map base_link /front/scan 1 0.0 0.0 0.0 nav2_amcl::DifferentialMotionModel
TorWIC SLAM ${{TORWIC_SLAM_BAGS}} ${{TORWIC_SLAM_DIR}} odom map base_link /front/scan 1 0.0 0.0 0.0 nav2_amcl::DifferentialMotionModel
Willow Garage ${{WILLOW_DSET_BAGS}} ${{WILLOW_DSET_DIR}} odom_combined map base_footprint /base_scan 1 0.0 0.0 0.0 nav2_amcl::OmniMotionModel
Omni Drive Sim 24hs ${{OMNI_DRIVE_SIM_BAGS}} ${{LONG_DURATION_DIR}} odom map base_link /scan 1 3.2 9.0 0.7 nav2_amcl::OmniMotionModel

*** Test Cases *** DATASET BASEDIR ODOM_FRAME MAP_FRAME BASE_FRAME SCAN_TOPIC ITERATION INITIAL_POSE_X INITIAL_POSE_Y INITIAL_POSE_YAW ROBOT_MODEL_TYPE
Diff Drive Sim 60m ${{DIFF_DRIVE_SIM_BAGS}} ${{DATASETS_DIR}} odom map base_link /scan 1 0.0 2.0 0.0 nav2_amcl::DifferentialMotionModel
Omni Drive Sim 30m ${{OMNI_DRIVE_SIM_BAGS}} ${{DATASETS_DIR}} odom map base_link /scan 1 3.2 9.0 0.7 nav2_amcl::OmniMotionModel

*** Keywords ***
Nav2 Beam Skipping Evaluation benchmark suite
Expand Down

0 comments on commit 816ba4b

Please sign in to comment.