Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visualize RAI task plan #214

Open
adamdbrw opened this issue Sep 9, 2024 · 0 comments
Open

Visualize RAI task plan #214

adamdbrw opened this issue Sep 9, 2024 · 0 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@adamdbrw
Copy link
Member

adamdbrw commented Sep 9, 2024

Is your feature request related to a problem? Please describe.
RAI is typically successful with task plans such as go to A, ask about something, observe etc, go to B. However, it would be great for it to be able to support visualisation of tasks to help people understand what it is going to do. It makes most sense to visualise on a mobile device / website (webviz etc.).

Describe the solution you'd like
Produce visualization / markers for the plan and during it's execution:

  • In general, we need to be able to see the map as it understands it. This is existing feature to be included in rviz2 config or similar.
  • For a navigation task, nav stack waypoints and plan should be used. This is existing feature to be included in rviz2 config or similar.
  • (RAI addition) Task icons for tools that are used (say something, wait for response, observe), and a general task icon.
  • As a bonus, the tool could visualize keyframes from issue Key frames node for camera #213.

As a bonus, if it makes sense, try to make RAI additionally self-aware of this plan as a means of validation, by feeding the plan visually back to RAI (through a VLM description).

This could also be a part of RAI control UX in the future.

@adamdbrw adamdbrw added the enhancement New feature or request label Sep 9, 2024
@adamdbrw adamdbrw changed the title Visualise RAI task plan Visualize RAI task plan Sep 9, 2024
@adamdbrw adamdbrw added the help wanted Extra attention is needed label Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant