diff --git a/CalibrationRoutines/MCC/mcc_with_iVA.md b/CalibrationRoutines/MCC/mcc_with_iVA.md index d2a7930..d7c2e0d 100644 --- a/CalibrationRoutines/MCC/mcc_with_iVA.md +++ b/CalibrationRoutines/MCC/mcc_with_iVA.md @@ -2,7 +2,7 @@ Motion Camera Calibration is a camera calibration software provided by ifm based on an ifm specific checkerboard that estimates the rotational parameters of the camera head with respect to the user coordinate system. -This application is available free of charge and included in the embedded O3R firmware (`version >= 1.1.30`) and the GUI ifm Vision Assistant (`version >= 2.7.6`). To use this application, users must install the ifmVisionAssistant tool, which can be downloaded [here](https://www.ifm.com/us/en/product/OVP800?tab=documents). To get started with the ifmVisionAssistant, please refer to this [documentation](index_iVA.md). +This application is available free of charge and included in the embedded O3R firmware (`version >= 1.1.30`) and the GUI ifm Vision Assistant (`version >= 2.7.6`). To use this application, users must install the ifmVisionAssistant tool, which can be downloaded [here](https://www.ifm.com/us/en/product/OVP800?tab=documents). To get started with the ifmVisionAssistant, please refer to this [documentation](../../SoftwareInterfaces/iVA/index_iVA.md). ## Pre-requisites diff --git a/CompatibilityMatrix/compatibility_matrix.md b/CompatibilityMatrix/compatibility_matrix.md index 1e80059..042ec5d 100644 --- a/CompatibilityMatrix/compatibility_matrix.md +++ b/CompatibilityMatrix/compatibility_matrix.md @@ -10,7 +10,7 @@ | OVP800
M04239 | O3R222
03R225 | FW 1.0.14 | 1.2.6 | 2.6.14 | :::{note} -- Other combinations of versions than the ones listed in the table above could work but are not officially supported by ifm. [See table below](#suggested-hardware) +- Other combinations of versions than the ones listed in the table above could work but are not officially supported by ifm. [See table below](#recommended-hardware) - Newer API versions are aimed to be backward compatible. Please verify via the API release notes and changelog. ::: diff --git a/Firmware/ReleaseNotes/FW_1.10.x/FW_1.10.x_release_notes.md b/Firmware/ReleaseNotes/FW_1.10.x/FW_1.10.x_release_notes.md new file mode 100644 index 0000000..66f3bd0 --- /dev/null +++ b/Firmware/ReleaseNotes/FW_1.10.x/FW_1.10.x_release_notes.md @@ -0,0 +1,73 @@ +# Firmware 1.10.13 release notes +The following release note provides an overview of the features of the **Firmware 1.10.13** version. + +The firmware update file for the OVP81x devices is available on [ifm.com](https://www.ifm.com). + +## Compatibility +### Previous releases +The previous version is v1.4.30. + +### Compatible software versions +This firmware release works with the following software package versions. + +| Software | Version | +| ------------------------ | -------- | +| ifmVisionAssistant | >= 2.9.9 | +| ifm3d library | 1.5.3 | +| ifm3d-ros ROS(1) wrapper | >= 1.1.2 | +| ifm3d-ros2 ROS2 wrapper | >= 1.2.0 | + +### Compatible processing platforms +:::{warning} +FW 1.10.13 does not support the OVP80x family at the moment. +**Do not update an OVP80x with this firmware. Such a FW update will fail, and the device will reboot to the previous FW version installed.** +::: + +This firmware release can be applied to the following ifm video processing platforms: + +| Article | Description | +| ------- | --------------------------------------------------------------------------- | +| OVP810 | Series product with TX2-NX NVIDIA board. | +| OVP811 | Series product with TX2-NX NVIDIA board, including an ODS license. | +| OVP812 | Series product with TX2-NX NVIDIA board, including a PDS license. | +| OVP813 | Series product with TX2-NX NVIDIA board, including ODS and PDS licenses. | + +### Supported camera articles +This firmware release supports the following ifm camera articles: + +| Camera Article | Description | +| -------------- | ----------------------------------------------------------- | +| O3R222 | 3D: 38k 224x172, 60°x45° IP50
2D: 1280x800, 127°x80° | +| O3R225 | 3D: 38k 224x172, 105°x78° IP50
2D: 1280x800, 127°x80° | + +## Base device +### Changed +- The default `channelValue` for 3D port number X [0..5] is changed to 2*X. +- The L4T kernel version is updated to r32.7.5. + +## ODS application +### Added +- **Diagnostics for decalibrated camera ports:** A new error, `ERROR_ODSAPP_CAMERA_DECALIBRATED`, is raised if any active camera port is likely to be decalibrated. In this case, the camera calibration should be verified. +- A new parameter, `rangeOfInterest`, allows users to define the maximum range of vision in meters (in robot coordinates). This setting directly impacts the grid shape, which is always square with each dimension containing (`rangeOfInterest`/0.05) × 2 cells at the default resolution of 5 cm. +- ODS supports polygons on the zone interface, allowing for the definition of both convex and concave shapes. +- A preset feature is implemented where the user can load a predefined zone configuration. + +### Changed +- The maximum number of points to define a zone is increased from 6 to 16. +- The ODS diagnosis sources are now attached to the specific camera port rather than the overall ODS application, for the following diagnostic messages: + - `ERROR_ODSAPP_FOV_INSUFFICIENT_FOR_NEGATIVE_OBSTACLES` + - `ERROR_ODSAPP_CAMERA_DECALIBRATED` + - `ERROR_ODSAPP_EXTR_DI_CALIB_IMPLAUSIBLE` + - `ERROR_ODSAPP_VO_EXTR_DI_CALIB_IMPLAUSIBLE` +- A maximum of one ODS instance is allowed. It is expected to use a single ODS application, which is always in `RUN` mode, and to switch the `activePorts` to change which cameras are used. +- The conditions for raising the `ERROR_ODSAPP_VO_EXTR_DI_CALIB_IMPLAUSIBLE` diagnostic message were relaxed. + +### Known issues +- The diagnostics activated by the ODS application during the `RUN` state will not be deactivated when the application is set to the `CONF` or `IDLE` state. +- If the negative obstacle feature is enabled, it may happen that the diagnostic message `ERROR_ODSAPP_FOV_INSUFFICIENT_FOR_NEGATIVE_OBSTACLES` is raised even thought the field of view conditions are fulfilled. In this case, negative obstacles will not be properly detected. To fix this, the application should be set to `CONF`, and then to `RUN` again. + +## PDS application + +- A new Pick and Drop System (PDS) application is added. This is a new feature with firmware 1.10.13. Refer to [the PDS documentation](../../../PDS/index_pds.md) for an overview of the available features. + + diff --git a/Firmware/ReleaseNotes/index.md b/Firmware/ReleaseNotes/index.md index 3105e73..cd21b29 100644 --- a/Firmware/ReleaseNotes/index.md +++ b/Firmware/ReleaseNotes/index.md @@ -5,6 +5,7 @@ Release notes for firmware version 0.16.23 and lower are available upon request. ::: :::{toctree} +OVP81x: version 1.10.x OVP81x: version 1.4.30 OVP81x: version 1.1.41 OVP80x: version 1.4.32 diff --git a/Firmware/fw_update.md b/Firmware/fw_update.md index 2c3175d..89c5a2b 100644 --- a/Firmware/fw_update.md +++ b/Firmware/fw_update.md @@ -39,15 +39,19 @@ In the instructions below, we expect that the user extracted the downloaded ZIP - Select the file and the update process will start. - Once the update is complete, the device will reboot. -### With the ifm3d API or the web interface -#### Reboot to recovery -When the starting firmware is version 1.0.0 and above, a reboot to recovery state is necessary to perform an update. +:::{note} +The firmware update using the iVA Linux AppImage may fail on the first attempt, causing the VPU to remain in recovery mode. +If this happens, refer to [the instructions below](#rebooting-to-productive-mode) to reboot to productive mode, using the ifm3d API. +::: +### With the ifm3d API + +In the instructions below, replace `` with the path to the firmware file you downloaded from [ifm.com](https://www.ifm.com/). :::::{tabs} ::::{group-tab} CLI ```bash -$ ifm3d reboot --recovery +$ ifm3d swupdate --file= ``` :::: ::::{group-tab} c++ @@ -57,11 +61,10 @@ $ ifm3d reboot --recovery ... auto o3r = std::make_shared(); auto sw = std::make_shared(o3r); -sw->RebootToRecovery(); -if (sw->WaitForRecovery()) { - std::cout << "System in recovery mode" << std::endl; +if (sw->FlashFirmware("")){ + sw->WaitForProductive(); + std::cout << "System ready!" << std::endl; } -... ``` :::: ::::{group-tab} Python @@ -70,50 +73,6 @@ from ifm3dpy.swupdater import SWUpdater from ifm3dpy.device import O3R o3r = O3R() sw = SWUpdater(o3r) -sw.reboot_to_recovery() -if sw.wait_for_recovery(): - print("System in recovery mode) -``` -:::: -::::: - -:::{note} -If you happen to be stuck in recovery mode, the device will not be "ping-able." To reboot to productive, you have two options: -- You can update the system again. If the update is successful, the system will reboot to productive, or, -- You can use the ifm3d API to reboot to productive without updating. With the command line interface, you can use `ifm3d swupdate -r`. In Python you can use the [`reboot_to_productive` function](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.swupdater.SWUpdater.html#ifm3dpy.swupdater.SWUpdater.reboot_to_productive) and in C++ the [`RebootToProductive` function](https://api.ifm3d.com/stable/cpp_api/classifm3d_1_1SWUpdater.html#a5ed7d927b9ff35a6808394345bdced8e). -::: - -#### With the web interface - -Once the device is in recovery mode (see section above), you can open the web interface: - -1. Open [http://192.168.0.69:8080/](http://192.168.0.69:8080/) in web browser. The `SWUpdate` web interface is shown. -2. Drag and drop the `*.swu` firmware file into the `software update`-window. The upload procedure starts. - -The system will automatically reboot in productive mode. The web interface will not be available anymore (it is only available in recovery mode). - -#### With ifm3d - -Once the device is in recovery mode, you can use ifm3d to update the firmware. -In the instructions below, replace `` with the path to the firmware file you downloaded from [ifm.com](https://www.ifm.com/). -The code below is continued from the "reboot to recovery" section. - -:::::{tabs} -::::{group-tab} CLI -```bash -$ ifm3d swupdate --file= -``` -:::: -::::{group-tab} c++ -```cpp -if (sw->FlashFirmware("")){ - sw->WaitForProductive(); - std::cout << "System ready!" << std::endl; -} -``` -:::: -::::{group-tab} Python -```python if sw.flash_firmware(''): sw.wait_for_productive() print("System ready!") @@ -146,6 +105,11 @@ o3r.get(["/device/swVersion/firmware"]) :::: ::::: +#### Rebooting to productive mode +If you happen to be stuck in recovery mode, the device will not be "ping-able." To reboot to productive, you have two options: +- You can update the system again. If the update is successful, the system will reboot to productive, or, +- You can use the ifm3d API to reboot to productive without updating. With the command line interface, you can use `ifm3d swupdate -r`. In Python you can use the [`reboot_to_productive` function](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.swupdater.SWUpdater.html#ifm3dpy.swupdater.SWUpdater.reboot_to_productive) and in C++ the [`RebootToProductive` function](https://api.ifm3d.com/stable/cpp_api/classifm3d_1_1SWUpdater.html#a5ed7d927b9ff35a6808394345bdced8e). + #### The full example script We provide a full Python script to help you update the firmware using the API. You can find this script [in the `ifm3d-examples` repository](https://github.com/ifm/ifm3d-examples/blob/main/ovp8xx/python/ovp8xxexamples/core/fw_update_utils.py). diff --git a/GettingStarted/ifmVisionAssistant/iVA_with_wine.md b/GettingStarted/ifmVisionAssistant/iVA_with_wine.md deleted file mode 100644 index 88ffb87..0000000 --- a/GettingStarted/ifmVisionAssistant/iVA_with_wine.md +++ /dev/null @@ -1,54 +0,0 @@ -# ifmVisionAssistant on Linux - -All stable versions of ifmVisonAssistant are natively compatible with Windows OS if not marked otherwise. - -A stable version of ifmVisionAssistant for Linux OS and will be released soon. -As a workaround until the ifmVisionAssistant will be released for Linux Debian based systems, use Wine("Wine is Not an Emulator") to "emulate" ifmVisionAssistant. - -Wine("Wine is Not an Emulator") is a compatibility layer capable of running Windows applications on several POSIX-compliant operating systems, such as Linux, macOS, & BSD. Instead of simulating internal Windows logic like a virtual machine or emulator, Wine translates Windows API calls into POSIX calls on the fly, eliminating the performance and memory penalties of other methods and allowing you to cleanly integrate Windows applications into your desktop. - -Please follow the installation instructions documented on the [Wine website](https://wiki.winehq.org/Ubuntu) - -To check the installation of wine execute the following command: - -```bash -$wine --version -``` - -**Optional: Configuring Wine to resembles Windows 10** - -By default, the wine configuration resembles Windows 7 environment. The user can configure -```bash -$wine winecfg -``` - -We suggest as an optional step to change your Wine configuration to resemble a Windows 10 environment. - -## ifmVisionAssistant - -1. Download the latest ifmVisionAssistant from the [official ifm website](https://www.ifm.com/de/en/product/OVP800?tab=documents) -2. Extract the downloaded zip file -3. Navigate to the folder where the ifmVisionAssistant files are extracted -4. Execute the following command in a bash terminal to launch the ifmVisionAssistant application. - -```bash -$sudo wine ifmVisionAssistant.exe -``` - -![wine_iVA.gif](resources/wine_iVA.gif) - - -*Most Wine related warnings can be ignored* - -### ifmVisionAssistant logging -If you experience any unexpected GUI related issues please launch the ifmVisionAssistant application with the additional flag `log`. - -```bash -$wine ifmVisionAssistant.exe -log - -This prints the output to STD OUT and also generates an additional log files in your .wine directory under `/home/raxxxx/.wine/dosdevices/c:/users/rasheed/AppData/Roaming/ifm electronic/ifmVisionAssistant/logs`. -Please include those logs when reporting any issues. - -### Known limitations: -+ Sensor search might be limited depending on your machine and network broadcast settings. -+ Saving and loading files from your hard drive may only be possible the wine instance has been started with elevated rights - `sudo`. \ No newline at end of file diff --git a/GettingStarted/ifmVisionAssistant/index_iVA.md b/GettingStarted/ifmVisionAssistant/index_iVA.md deleted file mode 100644 index d0a22b7..0000000 --- a/GettingStarted/ifmVisionAssistant/index_iVA.md +++ /dev/null @@ -1,8 +0,0 @@ -# Getting started with the ifm Vision Assistant - -:::{toctree} -Introduction and installation -ifmVisionAssistant on Linux -Connect to the O3R -First steps -::: \ No newline at end of file diff --git a/GettingStarted/index_sw_stack.md b/GettingStarted/index_sw_stack.md index 700f48c..f6fc02b 100644 --- a/GettingStarted/index_sw_stack.md +++ b/GettingStarted/index_sw_stack.md @@ -5,7 +5,7 @@ Multiple software stacks are available: pick your favorite one below and follow :::{toctree} :maxdepth: 1 -ifmVisionAssistant +ifmVisionAssistant <../SoftwareInterfaces/iVA/index_iVA> Python C++ ::: diff --git a/ODS/Configuration/configuration.md b/ODS/Configuration/configuration.md index dcfed19..a08c693 100644 --- a/ODS/Configuration/configuration.md +++ b/ODS/Configuration/configuration.md @@ -1,24 +1,35 @@ # Configuration - -| Parameter | Description | - ------------ | ---------- | -| `name` | Providing a custom name for the ODS application| -| `ports` | The ports that can be used by the application (camera heads and `IMU`)| -| `state` | The current app state and dependent camera hardware sates - "RUN" or "CONF"| -| `configuration/activePorts` | The list of camera ports that are active and being used by the application at one time. The number cannot exceed `maxNumSimultaneousCameras`. | -| `configuration/maxNumSimultaneousCameras` | The maximum number of cameras used simultaneously by ODS. This parameter requires a change to "CONF." | -| `grid/maxHeight` | Ceiling value above which all obstacles are ignored. Applies to the occupancy grid and all zones.| -| `grid/overhangingLoads` | A static region defined in the vehicle coordinate system which is excluded from the obstacle detection region (see [the overhanging loads documentation](../OverhangingLoads/overhanging_loads.md)). | -| `grid/temporalConsistencyConstraint` | Ensures that artifacts caused by dust particles are ignored (see [the dust mitigation documentation](../DustMitigation/dust_mitigation.md)). | -| `portX/acquisition/channelValue` | The camera channel value [-100, 100] used for a specific port to mitigate interference. Channel values should differ of at least 2.| -| `portX/negObst/enableNegativeObstacles` | Enable or disable the negative obstacle detection (see [the negative obstacle documentation](../NegativeObstacles/negative_obstacles.md)).| -| `portX/seg/minObjectHeight` | Minimum object height in [m], for example 0.025 [m]. This does not mean that any objects of this height will always be detected, but that objects below this height will be excluded from any detection. | -| `vo/voPorts`| Visual odometry - define a port which is the main camera stream for the visual odometry feature. Define multiple ports if a single reference port for visual odometry will not always be active, for example when switching between forward and backward looking cameras.| -| `zones`| Define protection zones to be used - [See `ods zone definition` for a complete description](../Zones/zones.md)| - + | +| Property | Type | Description | Default | Minimum | Maximum | +| :------------------------------------------------------ | :------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------- | :------ | :------ | +| `configuration/activePorts` | array | List of active ports. At least one port listed in vo/ports must be in the list. | ['port2', 'port3'] | N/A | N/A | +| `configuration/grid/maxHeight` | number | Maximum detection height in [m]. | 2.0 | 0.25 | 10.0 | +| `configuration/grid/overhangingLoads` | array | The overhanging loads are ignored during object detection. For more details, please refer [overhanging loads](../OverhangingLoads/overhanging_loads.md) | [] | N/A | N/A | +| `configuration/grid/rangeOfInterest`new | number | Maximum range of vision in [m] (given in robot coordinates). This parameter affects the grid shape. The grid shape is always quadratic with at least (rangeOfInterest/0.05)*2 cells for the default resolution of 5cm. | 5.0 | 1.0 | 10.0 | +| `configuration/grid/temporalConsistencyConstraint` | number | Increase number of frames needed for valid object by the factor. Use factors > 1 for dusty conditions. | 1.0 | 1.0 | 5.0 | +| `configuration/maxNumSimultaneousCameras` | integer | The maximum number of simultaneously cameras in activePorts. Increase this setting if your ODS application uses 3 cameras at the same time. | 2 | 1 | 3 | +| `configuration/port**X**/acquisition/channelValue` | integer | N/A | 4 | -100 | 100 | +| `configuration/port**X**/negObst/enableNegativeObstacles` | boolean | Enable the detection of negative obstacles | False | N/A | N/A | +| `configuration/port**X**/seg/minObjectHeight` | number | minimum height above ground in [m] for segmented object pixels | 0.025 | 0.0 | 1.0 | +| `configuration/version/major` | integer | Major part of the version number. | N/A | 0 | N/A | +| `configuration/version/minor` | integer | Minor part of the version number. | N/A | 0 | N/A | +| `configuration/version/patch` | integer | Patch part of the version number. | N/A | 0 | N/A | +| `configuration/vo/voPorts` | array | List of ports suitable for visual odometry. VO will be applied with the first active port in the list. | ['port2', 'port3', 'port4'] | N/A | N/A | +| `configuration/zones/zoneConfigID` | integer | Will be looped through to the output and can be used to identify zone configurations. | 0 | N/A | N/A | +| `configuration/zones/zoneCoordinates` | array | The coordinates of the configured zones. | [] | N/A | N/A | +| `configuration/zones/zoneType`new | string | If set to convexHull, the zone is defined as the convex hull of the given list of points. If set to polygon, the zone is defined as a filled polygon. In all cases, cells are considered to be part of the zone if and only if the center of the cell is inside the region. | convexHull | N/A | N/A | +| name | string | User-defined application name | N/A | N/A | N/A | +| `ports` | array | The ports that can be used by the application (camera heads and `IMU`) | ['port6', 'port0'] | N/A | N/A | +| `presets/command`new | string | Preset command to be executed. For more details, please refer [presets documentation](../Presets/presets.md#command) | nop | N/A | N/A | +| `presets/definitions`new | object | Preset definitions | {} | N/A | N/A | +| `presets/load/identifier`new | integer | Preset identifier to be applied | 0 | 0 | 127 | +| `presets/location`new | string | Location at which presets are applied | N/A | N/A | N/A | +| `state` | string | Application state | CONF | N/A | N/A | :::{warning} -All connected heads have their own specific configuration. However, all heads referenced by the ODS are also configured by the application itself. **Only `channelValue` and `minObjectHeight` can be changed by the user and for each port separately**. The general configuration of ports (`{"ports":{"port2":...}}`) must not be changed by the user. +- All connected heads have their own specific configuration. However, all heads referenced by the ODS are also configured by the application itself. **Only `channelValue` and `minObjectHeight` can be changed by the user and for each port separately**. +- From the FW 1.10.13, the default channel values will be set to 2*. +- The general configuration of ports (`{"ports":{"port2":...}}`) must not be changed by the user. ::: ## Channel value @@ -62,18 +73,26 @@ Increasing this value might prevent false positives caused by the floor and/or s ::: ```JSON title="minObjectHeight" -"configuration": { - "port2": { - "acquisition": { - "seg": { - "minObjectHeight": 0.025 - } - } - }, - "port3": { - "acquisition": { - "seg": { - "minObjectHeight": 0.040 +{ + "applications":{ + "instances":{ + "appX":{ + "configuration": { + "port2": { + "acquisition": { + "seg": { + "minObjectHeight": 0.025 + } + } + }, + "port3": { + "acquisition": { + "seg": { + "minObjectHeight": 0.040 + } + } + } + } } } } @@ -85,8 +104,8 @@ Increasing this value might prevent false positives caused by the floor and/or s To activate/deactivate the application use the following definition: -- Active: "RUN" -- Inactive: "CONF" +- Active: `RUN` +- Inactive: `CONF` :::{note} To ensure that restarting the data stream is as quick as possible, we recommend keeping the application always in "RUN" state. For an inactive application, set the `activePorts` list to an empty list `[]`. diff --git a/ODS/Mounting/mounting.md b/ODS/Mounting/mounting.md index 9b8ed49..636b4fd 100644 --- a/ODS/Mounting/mounting.md +++ b/ODS/Mounting/mounting.md @@ -6,7 +6,7 @@ Review the general mounting instructions to comply with [heat dissipation guidel Generally, we recommend mounting the lowest camera at a height in the range of 250 to 700 mm from the ground. -Additionally, at least one of the active cameras should have the floor in its field of view, with visible floor up to 1 m, to allow for optimal performance in motion. +Additionally, at least one of the active cameras should have the floor in its field of view. For more details review the [visual odometry](#visual-odometry). ::: The positioning of the cameras on the AGV has a great impact on the performance of ODS. @@ -16,9 +16,9 @@ First, review [the general mounting instructions](../../Technology/Hardware/Moun Then, there are a couple additional things to take into account specifically for ODS cameras. -### Mounting position +## Mounting position -#### Mounting height recommendations +### Mounting height recommendations Mounting height is important for all cameras, but it is especially critical for cameras that will be used to detect small objects on the floor. @@ -34,7 +34,7 @@ With these three considerations in mind, we recommend that the lowest cameras be For cameras that are only used to detect cantilevered objects and are not expected to segment small objects from the ground, the mounting requirements are relaxed and the camera can be mounted higher and pointed upward. -#### Examples +### Examples Below are two examples of mounting positions for ODS. The first example shows a typical AMR platform, with one camera on each side. @@ -53,30 +53,31 @@ The lower camera is an O3R222, which is the narrow field of view camera. It is t The upper camera is an O3R225, which is the wide field of view camera. It is tilted up and is used to detect overhanging obstacles that could damage the top part of the vehicle. Both cameras are mounted at around 70 cm height. -| | | -| ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| | | +| ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | ![Example of camera mounting on a fork truck, general view](./img/fork-lift-1.png) | ![Example of camera mounting on a fork truck, close up view](./img/fork-lift-2.png) | :::{note} These images are intended as examples only and do not represent an actual at-scale model. ::: -### Visual odometry +## Visual odometry -Visual odometry is used in ODS along with IMU data to calculate the ego motion (linear velocity, angular velocity) of the vehicle. +Visual odometry is used in ODS along with IMU data to calculate the ego motion (linear velocity, angular velocity) of the vehicle. For a fully functional system, the visual odometry information needs to be available for at least one active camera at all times. -Typically, visual odometry requires that pixels associated with the ground be visible up to about 1 meter under typical driving conditions. + +If the camera head which is responsible for visual odometry is not calibrated or doesn't see enough floor pixels then a diagnostic `ERROR_ODSAPP_VO_EXTR_DI_CALIB_IMPLAUSIBLE` will be raised. This may require angling the camera down to see the ground in front of or to the side of the vehicle. -### Clearance area +## Clearance area It is worth repeating that special attention should be paid to the clearance requirements for each camera. Any object obstructing the field of view or the clearance area will have a noticeable negative impact on ODS performance. Refer to the [clearance area guidelines](../../Technology/Hardware/Mounting/clearance_area.md). -### Dead zones +## Dead zones Depending on the mounting position, dead zones may exist. ODS does not retain a memory of objects that have disappeared from the field of view into a dead zone. It is important to take this into account and define a strategy to either cover the dead zones with additional cameras, or to incorporate these dead zones into the effective breaking distance calculation. \ No newline at end of file diff --git a/ODS/NegativeObstacles/negative_obstacles.md b/ODS/NegativeObstacles/negative_obstacles.md index 16cf3dd..1413962 100644 --- a/ODS/NegativeObstacles/negative_obstacles.md +++ b/ODS/NegativeObstacles/negative_obstacles.md @@ -8,6 +8,12 @@ ODS has the capability to detect negative obstacles, that is obstacles that are ## Prerequisites and limitations +The camera used for detecting negative obstacles should mounted in such a way that minimum radial distance from camera to the ground floor must be less than 1 meter. Otherwise, the camera is mounted too high to be able to detect the floor and raises the diagnostic `ERROR_ODSAPP_FOV_INSUFFICIENT_FOR_NEGATIVE_OBSTACLES`. + +:::{note} +ODS application relies on an accurate extrinsic calibration parameters to detect the ground plane. +::: + To the ODS app, a negative obstacle is defined as: - At least 20 centimeter large along the X axis, - At least 20 centimeter large along the Y axis, diff --git a/ODS/OccupancyGrid/occupancy_grid.md b/ODS/OccupancyGrid/occupancy_grid.md index 530f4c0..972589a 100644 --- a/ODS/OccupancyGrid/occupancy_grid.md +++ b/ODS/OccupancyGrid/occupancy_grid.md @@ -1,5 +1,13 @@ # Occupancy grid +## Range of Interest + +The default occupancy grid size is 10 x 10 m2 with the calibrated reference coordinate system (RCS) positioned at the center of grid with X-range<=5 meters. + +If large vehicles are used for ODS with translation offsets > 1.5 m with respect to the RCS, the occupancy grid size is limiting the detection range of ODS. The detection range on large objects is close to 4 m, which would get clipped by the available 3.5 m range in occupancy grid. + +To address this issue, from firmware versions >= 1.10.13 a new parameter is introduced, `rangeOfInterest`, within the `grid` configuration. This parameter allows users to extend the grid size to accommodate larger translation offsets, ensuring optimal performance of the ODS in such scenarios. + ## Output | Name | Type | Description | diff --git a/ODS/Performance/PerformanceBenchmark/ods_performance_benchmark.md b/ODS/Performance/PerformanceBenchmark/ods_performance_benchmark.md index 0d58c45..0595778 100644 --- a/ODS/Performance/PerformanceBenchmark/ods_performance_benchmark.md +++ b/ODS/Performance/PerformanceBenchmark/ods_performance_benchmark.md @@ -36,7 +36,7 @@ The reflectivity and type of the floor surface impacts the amount of light refle Even though the O3R cameras utilize the infrared spectrum, a good rule of thumb is that floors that appear "reflective" for the human eye will most likely also be reflective in infrared. Three main things are impacted by the reflectivity and surface type of the floor: -- Floors that are very smooth without visible marks, like freshly painted floors, can be challenging for the [visual odometry](#visual-odometry) estimation. +- Floors that are very smooth without visible marks, like freshly painted floors, can be challenging for the visual odometry estimation. - Highly reflective floors can create stray light artifacts when the camera is mounted low. In this case, stray light can negatively impact the detection of small or low reflectivity objects. - To reliably detect small objects (smaller than 10cm) on the floor, ODS relies partially on detecting the floor in the vicinity of the object. For highly reflective floors, the visible floor range is typically smaller. This means that performance detection for small objects might be degraded on highly reflective floors. For objects larger than 10 cm (vertically), this limitation does not apply. diff --git a/ODS/Presets/presets.md b/ODS/Presets/presets.md new file mode 100644 index 0000000..a54e751 --- /dev/null +++ b/ODS/Presets/presets.md @@ -0,0 +1,211 @@ +# Presets + +## Overview + +The presets feature in the Obstacle Detection System (ODS) allows the users to define and manage up to 128 predefined zone configurations. This feature facilitates quick switching between different configurations by loading a preset definition through a single command. +This is especially useful in dynamic environments requiring frequent zone setup changes. + +## Key features + +1. Predefined configurations: users can define up to 128 presets. +2. Quick activation: presets can be activated using the load command with a specific definition identifier. +3. Customizable definitions: each preset includes: + 1. A set of active camera ports. + 2. Zone configurations with a unique + 1. Zone configuration ID + 2. Zone type + 3. Zone coordinates. + +:::{note} +Currently, it is not possible to configure the presets using the ifmVisionAssistant. The presets are only configured using the Python or C++ ifm3d API. +::: + +## Parameter description +Within the ODS application parameters, the presets follow the structure below. +All presets are contained under the `presets/definitions` parameter. + + +```json +{ + "applications": { + "instances": { + "app0": { + "class": "ods", + "presets": { + "definitions":{ + "0": { + "description": "description of the preset", + "preset": { + "activePorts": [/* List of active camera ports */], + "zones": { + "zoneConfigID":/* Unique identifier for zone configuration */, + "zoneType":/* Type of zone (e.g., convexHull, Polygon) */, + "zoneCoordinates": [/* Coordinates defining the zone */] + } + } + } + }, + "load": {"identifier": 0}, + "location": "/applications/instances/app0/configuration", + "command": "nop" + } + } + } + } +} + +``` +All the parameters can be found in the `applications/instances/appX/presets` section of the JSON configuration. + +| Parameter | Description | +| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `/presets/definition/X` | The identifier of the first preset, the first identifier must start with value `"0"` (string). A maximum of 128 presets can be configured. | +| `/presets/definition/X/preset/activePorts` | The ports that are used by this preset | +| `/presets/definition/X/preset/zones` | Define the used protection zones - [See `ods zone definition` for a complete description](../Zones/zones.md). A maximum of three zones can be configured in one preset. | +| `/presets/load/identifier` | Value of the identifier of the preset that will be loaded when a `load` command is triggered | +| `/presets/location` | A read-only parameter specifying the JSON path to the ODS configuration, where the preset will be loaded when the `load` command is triggered. | +| `/presets/command` | Specifies the command to be executed. It can have the value `nop` (do nothing) or `load` (trigger loading preset). | +| `/presets/definitions/X/description` | String specifying a description for the preset | + +:::{note} +- The preset identifier is a string parameter. +- The command parameter is per default `nop` and will return to this default again once the operation is completed. +::: + +## Load command +To load a preset we use the `load` command, which will set the preset having the identifier matching the value of `/presets/load/identifier`. The load command is located under `/presets/command` and can have the following values: +- `nop`: Default value, meaning do nothing. +- `load`: Triggers loading the preset. + +The `load` command execution time mainly depends on the `activePorts` in the preset. If we load a preset with different `activePorts`, the newly added ports need to change from `IDLE` to `RUN`. +Changing only the zones while keeping the same `activePorts` is completed within one to two ODS frames, which corresponds to 50 to 100 ms with the default framerate of 20Hz. + +The following table documents the approximate load runtime: + +| Change | Time to update result | +| ---------------------------------------------------------------------------------------------------------- | --------------------- | +| Changing only the zone: `activePorts`: `["port2"]` → `["port2"]`, `zones`: `[zones_i]` → `[zones_j]` | 50 to 100 ms | +| Adding another port: `activePorts`: `["port2"]` → `["port2", "port3"]`, `zones`: `[zones_i]` → `[zones_j]` | ~600 ms | +| Mutually exclusive ports: `activePorts`: `["port2"]` → `["port3"]`, `zones`: `[zones_i]` → `[zones_j]` | ~600 ms | + +## Configuration procedure + +Please follow these steps to configure the ODS presets system: + +1. Create an ODS application instance and set up the ODS parameters. Please ensure that you list all the ports used across all presets in the ODS `/appX/ports` parameter. Refer to [the port configuration documentation](../Instantiation/instantiation.md#ports-selection) for detailed instructions. + +2. Currently, the presets can only be configured via the ifm3d API. Set a JSON configuration which contains the requested presets. Ensure each preset has **a unique identifier (`presets/definition/X`) matching the `zoneConfigID` values**. + :::{note} + JSON validation is only performed during the configuration of presets and not when they are loaded using the load command. + Each preset identifier should have a unique value starting from 0 up to 999, with a maximum of 128 presets. + To avoid confusion, set the `zoneConfigID` to match the identifier of each preset. + This is especially important when using the PLC application, as the global value displayed to the PLC is the `zoneConfigID`, **not** the preset identifier. + ::: +3. Set the ODS application to RUN state. This can be also done before setting the ODS presets. + :::{note} + - At this step, no preset will be automatically loaded. The default value of `applications/instances/appX/configuration/zones/zoneConfigID` is `0`. This does not mean that to the preset containing `zoneConfigID` `0` has been loaded. Only after manually executing a `load` command will the `zoneConfigID` parameter be overwritten. + - When ODS is in `RUN` state and a preset is added to the configuration, the currently used preset will not change until a `load` command is issued. + ::: +4. Set the desired preset via a `load` command. Make sure to use the identifier to load the appropriate preset. + +## Example Configuration + +Below is an example of an ODS configuration including presets. +This configuration includes three presets: + +1. Forward driving preset: optimized for detecting objects in front of the vehicle with three detection zones. +2. Backward driving preset: designed for rear detection with two zones. +3. Left turn preset: configured for detecting objects during left turns with three zones. + +These presets are provided as examples and can be customized or extended based on the specific needs of the customer. + +```json +{ + "applications": { + "instances": { + "app0": { + "class": "ods", + "ports": [ + "port2", + "port3", + "port6" + ], + "state": "CONF", + "presets": { + "definitions":{ + "0": { + "description": "Forward preset with three zones and one active camera", + "preset": { + "activePorts": [ + "port2", + ], + "zones": { + "zoneConfigID": 0, + "zoneType":"convexHull", + "zoneCoordinates": [ + [[0, -0.3], [2.5, -0.3], [2.5, 0.3], [0, 0.3]], + [[2.5, -0.3], [3.5, -0.3], [3.5, 0.3], [2.5, 0.3]], + [[3.5, -0.3], [4.5, -0.3], [4.5, 0.3], [3.5, 0.3]] + ] + } + } + }, + "1": { + "description": "Backward preset with two zones and one active camera", + "preset": { + "activePorts": [ + "port3" + ], + "zones": { + "zoneConfigID": 1, + "zoneType":"polygon", + "zoneCoordinates": [ + [[-0.5, -0.3], [-3.0, -0.3], [-3.0, 0.3], [-0.5, 0.3]] + [[-3.0, -0.3], [-4.0, -0.3], [-4.0, 0.3], [-3.0, 0.3]] + ] + } + } + }, + "2": { + "description": "Left turn preset with three zones and two active cameras", + "preset": { + "activePorts": [ + "port2", + "port4" + ], + "zones": { + "zoneConfigID": 2, + "zoneType":"polygon", + "zoneCoordinates": [ + [[0.5, 0.3], [1.5, 0.3], [1.5, 0.9], [0.5, 0.9]], + [[1.5, 0.3], [2.5, 0.3], [2.5, 0.9], [1.5, 0.9]], + [[2.5, 0.3], [3.5, 0.3], [3.5, 0.9], [2.5, 0.9]] + ] + } + } + } + } + } + } + } + } +} +``` + +## Best practices + +- As a first configuration step, include all required cameras in the JSON `ports` parameter. You can only define `activePorts` for each preset that are included in the global `ports` parameter. For example, if using two cameras for forward driving and two cameras for backward driving, specify all four camera ports in the global `ports` parameter. +- Ensure that each preset has a unique identifier to prevent one preset from being overwritten by another. Define a unique value for the identifier starting from 0 up to 999, with a maximum of 128 presets. +- We highly recommend to set the `zoneConfigID` value matches the identifier of the preset to avoid confusion. +- When loading a preset, the identifier of the preset is going to be used for the load command. The `zoneConfigID` displayed is the one defined in the loaded preset, at `presets/definitions/X/zones/zoneConfigID`. + +## Known issues: +- The `zoneConfigID` is not enforced to be identical with the presets identifier. +- If no preset is set, the `zoneConfigID` will be displayed as the default value 0, but this doesn't mean that preset 0 is active. +- `IDLE` state will be set for the application if a preset containing an empty `activePorts` is loaded. The ODS will not update the data and the output returned will be empty. +- The default value of the output if no zones or presets are set is `[0,0,0]`. So please ensure that a preset is loaded before operation. + + +## Code examples + +Code examples are available in the [ifm3d-examples](https://github.com/ifm/ifm3d-examples) repository: see [the Python ODS preset example](https://github.com/ifm/ifm3d-examples/ovp8xx/python/ovp8xxexamples/ods/ods_presets.py). diff --git a/ODS/Zones/zones.md b/ODS/Zones/zones.md index 6044747..9071c8c 100644 --- a/ODS/Zones/zones.md +++ b/ODS/Zones/zones.md @@ -1,32 +1,50 @@ # Zones -## Parameters +A zone is a user-defined region within the ODS’s detection field where obstacle presence is continuously monitored. Zones are geometric areas that can be configured based on the operational requirements of the AGV or AMR. -A zone maps the probability information of all grid cells within the zone extent to a binary value: `true` means that an object is detected, `false` that no object is detected. -This mapping is based on the internally computed convex hull (2D) of the user input `zoneCoordinates` and the `maxHeight` parameter. +Each zone has a simple binary state: -Additionally, the `minObjectHeight` parameter allows editing the expected minimum object height above the ground (`Z==0` plane) per camera. The `minObjectHeight` parameter is handled on a per-camera basis to allow fine-tuning for multi-camera setups with different camera mounting heights. For higher mounted cameras the `minObjectHeight` can be lower compared to lower mounted cameras to allow the user to see further away and lower objects relative to the floor plane. +- **1 (True):** Indicates that an obstacle has been detected in the zone. +- **0 (False):** Indicates the zone is clear of obstacles. -The `maxHeight` parameter is a global parameter for all zones. The 3D extent (maximum) limits along the vertical axis are therefore the same for all zones. +:::{important} + The zone output is influenced not only by the defined zone boundaries but also by `grid` parameters such as `maxHeight`, `overHangingLoads`. -![Zone Configuration](img/set_example_zones.gif) + Additionally, the `minObjectHeight` parameter allows editing the expected minimum object height above the ground (`Z==0` plane) per camera port. The `minObjectHeight` parameter is handled on a per-camera basis to allow fine-tuning for multi-camera setups with different camera mounting heights. For higher mounted cameras the `minObjectHeight` can be lower compared to lower mounted cameras to allow the user to see further away and lower objects relative to the floor plane. +::: + +## Parameters + +### `zoneConfigID` + +The user-defined ID for the currently used zone configuration. This information is passed as an output info so that user can know which zone configuration is currently used by ODS. + +### `zoneCoordinates` -### Zone extent +The `ODS` offers the flexibility to define up to three distinct zones for monitoring. These zones are configured using polygonal coordinates, offering versatility in shape and size. To define a zone, a minimum of three points is required (forming a triangle), while the maximum number of points depends on the firmware version: up to 16 points for firmware versions 1.10.13 or higher, and up to 6 points for earlier versions. -The 3D zone extent is defined via the 2D (convex) hull of the points in space as defined by the list `zoneCoordinates`. Its volume is handled via the combined `maxHeight` parameter and `minObjectHeight` parameters. If configured, [overhanging loads](../OverhangingLoads/overhanging_loads.md) will also be taken into account. +### `zoneType` -### Number of zones -An ODS application is limited to a maximum of three zones. +From the firmware versions 1.10.13 or higher, user can choose the zone type from `convexHull` or `polygon`. + +When configured as `convexHull`, the zone is automatically defined as the convex hull encompassing the provided list of points, forming the smallest convex boundary that includes all specified points. When set to `polygon`, the zone is defined as a filled polygon precisely shaped by the given coordinates. + +:::{important} + Irrespective of `zoneType`, a grid cell is considered part of the zone only if the center of the cell lies within the defined region, ensuring accurate and consistent detection boundaries. +::: + +![Zone Configuration](img/set_example_zones.gif) ## Output + The Output of the Zones is contained in `buffer_id.O3R_ODS_INFO`, which is formatted as follows: -| Name | Type | Description | -| ------------ | ------- | ---------------------------------------------------------------------------- | +| Name | Type | Description | +| -------------- | ------- | ----------------------------------------------------------------------------------- | | `timestamp_ns` | uint64 | timestamp of occupancy grid in nanoseconds - NTP time if NTP server is synchronized | -| `zoneOccupied` | int8[3] | a flag for each zone describing whether it is occupied or free | -| `zoneConfigID` | uint32 | the user-defined ID for the zone configuration | +| `zoneOccupied` | int8[3] | a flag for each zone describing whether it is occupied or free | +| `zoneConfigID` | uint32 | the user-defined ID for the zone configuration | The output of the zones can be easily deserialized, for more information please visit [the ifm3d deserializer documentation](https://api.ifm3d.com/stable/examples/o3r/deserialize/deserialize.html). @@ -38,17 +56,10 @@ You can also view the zones' output at the bottom left corner of the `Applicatio Every ODS frame also contains a timestamp in nanoseconds. If a `NTP-server` is provided, the timestamp is synchronized. Learn more about timestamps in [the timestamps documentation](../../Technology/VPU/Timestamps/timestamps.md). -### `zoneConfigID` - -The user-defined ID for the currently used zone configuration. - -### `zoneOccupied` - -The state information for all zones is consistently represented as an array of three elements, regardless of whether three or fewer zones have been configured. - - ## Examples + ### Only one zone + ```json { "zoneCoordinates": [[[1.0,1.0], [1.0,0.0], [-1.0, 0.0], [-1.0, 1.0]]] @@ -59,6 +70,7 @@ The JSON shown here is the convex hull of one zone, the red zone presented in th The width (lateral size, that is in Y-direction) is 2 m. The length (longitudinal size, that is in X-direction) is 1 m. ### Three zones + ```json { "zoneCoordinates": [ @@ -68,7 +80,9 @@ The width (lateral size, that is in Y-direction) is 2 m. The length (longitudina ] } ``` + ### Example output + In current FW versions the configuration and output of zones is limited to three. Trying to set 4 zones in the JSON configuration will result in a JSON schema error: ifm3d / ifm3dpy custom error. @@ -76,9 +90,10 @@ The zone evaluation output is also limited to three zones. Independently of how The first element pertains to the first zone, the second element to the second zone, etc. The example array below shows the output of 10 consecutive frames buffered into one array. + ```python array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8) -``` \ No newline at end of file +``` diff --git a/ODS/iVA/index_ifmODS_iVA.md b/ODS/iVA/index_ifmODS_iVA.md index 63e8291..083074c 100644 --- a/ODS/iVA/index_ifmODS_iVA.md +++ b/ODS/iVA/index_ifmODS_iVA.md @@ -1,6 +1,6 @@ # ODS with the iVA -Before reading this section, make sure to read [how to get started with the iVA](../../GettingStarted/ifmVisionAssistant/index_iVA.md). +Before reading this section, make sure to read [how to get started with the iVA](../../SoftwareInterfaces/iVA/index_iVA.md). :::{toctree} iVA_diagnostic diff --git a/ODS/index_ods.md b/ODS/index_ods.md index 29aaf61..322838b 100644 --- a/ODS/index_ods.md +++ b/ODS/index_ods.md @@ -13,7 +13,9 @@ Getting started Mounting Instantiation Configuration +Presets Zones +Presets Occupancy grid Recording Changing views @@ -30,7 +32,7 @@ C++ -:::{raw} html +:::{raw} HTML

The ODS journey

::: @@ -43,7 +45,7 @@ Each step in the diagram will guide you to the relevant documentation. :reverse: ::::{grid-item} -:::{raw} html +:::{raw} HTML :file: img/ods_journey.drawio.svg ::: :::: diff --git a/PDS/Calibration/pds_calibration.md b/PDS/Calibration/pds_calibration.md index e2f0ff1..e0688f3 100644 --- a/PDS/Calibration/pds_calibration.md +++ b/PDS/Calibration/pds_calibration.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Camera calibration for PDS The PDS application always returns the position of an object with respect to the calibrated coordinate system. Typically, the calibrated coordinate system corresponds to the fork tines, so that the coordinate system will go up and down with the forks and the pallet position will always be provided with reference to the current position of the forks. diff --git a/PDS/Configuration/configuration.md b/PDS/Configuration/configuration.md index 51bc258..c52fbe0 100644 --- a/PDS/Configuration/configuration.md +++ b/PDS/Configuration/configuration.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Configuration The PDS application is instantiated with a set of JSON parameters. @@ -18,9 +14,6 @@ A minimal instantiation is as follows, where `"app0"` can be replaced by another } } ``` -:::{note} -The user can refer to the [JSON schema](../../Technology/configuration.md#json-schema) after instantiating the application for more details on the available parameters. -::: This is equivalent to creating a PDS app in the Vision Assistant: @@ -31,19 +24,63 @@ This is equivalent to creating a PDS app in the Vision Assistant: The app will be instantiated with all parameters set to their default values. The parameters can be further configured and are described below. +## General parameters + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :-------------------------------------- | :------ | :------------------------------------------------ | :-------- | :------ | :------ | :----------------------------- | +| `class` | string | Application class | N/A | N/A | N/A | `['mcc', 'ods', 'pds', 'plc']` | +| `configuration.version.X` | integer | Version number for the parameters. | 0 | 0 | N/A | N/A | +| `data.algoDebugConfiguration.version.X` | integer | Version number for algo-debug. | N/A | 0 | N/A | N/A | +| `data.availablePCICOutput` | array | PCIC outputs that are produced by the application | N/A | N/A | N/A | N/A | +| `data.pcicTCPPort` | integer | PCIC output TCP/IP port of the application | N/A | N/A | N/A | N/A | +| `name` | string | User-defined application name | N/A | N/A | N/A | N/A | +| `ports` | array | N/A | ['port2'] | N/A | N/A | N/A | +| `state` | string | Application state | CONF | N/A | N/A | `['CONF', 'IDLE', 'ERROR']` | + +## Customization parameters at `configuration/customization` + +These parameters are available at the `configuration/customization` path in the JSON configuration. These parameters allow users to quickly adapt the detection system to their environment with minimal input. + +**Recommended for:** + +- Users who want to set up the system with minimal adjustments. +- Situations where the detection environment is consistent, and default settings are sufficient. +- Configuring high-level properties like pallet type, detection order, or defining the Volume of Interest (VOI). + +**Example:** + +For example, to detect a standard pallet in a warehouse when the approximate distance from the camera is known, you would just need to adjust the `depthHint` and select the appropriate `palletIndex`. + +**Parameters:** + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :--------------------- | :----- | :------------------------------------------------------------------------------------------------------------------------------------------ | :------ | :------ | :------ | :-------------------------------------------- | +| `command` | string | PDS command to be executed. When the execution has finished, this field will automatically switch back to nop (No Operation) | nop | N/A | N/A | `['nop', 'getPallet', 'getRack', 'volCheck']` | +| `getPallet.PARAMETERS` | | Customization parameters for the `getPallet` command. Refer to [the `getPallet` documentation](../GetPallet/getPallet.md) for more details. | | | | | +| `getRack.PARAMETERS` | | Customization parameters for the `getRack` command. Refer to [the `getRack` documentation](../GetRack/getRack.md) for more details. | | | | | +| `volCheck.PARAMETERS` | | Customization parameters for the `volCheck` command. Refer to [the `volCheck` documentation](../VolCheck/volCheck.md) for more details. | | | | | + +## Fine-tuning parameters at `configuration/parameter` + +These parameters are available at the `configuration/parameter` path in the JSON configuration. + +These parameters should be used when fine-tuning the detection algorithm for optimal performance in complex or variable environments. + +**Recommended usage**: + +- Users who need to improve accuracy and speed in challenging detection scenarios. +- Pallets of varying sizes or material. +- Racks of non-standard sizes. +- etc. + +**Example:** For example, the user can adjust the bounding box dimensions with `getPallet/x/orthoProjection/voi/xMax` and `xMin` to focus the detection on specific areas and therefore increase the detection speed. + +**Parameters:** +`getPallet`, `getRack` and `volCheck` can all be fine tuned in specific ways. +For more details on each of the parameters, refer to the respective documentation: [`getPallet` parameters](../GetPallet/getPallet.md), [`getRack` parameters](../GetRack/getRack.md) and [`volCheck` parameters](../VolCheck/volCheck.md). + +## Port parameters at `configuration/port` -| Parameter | Description | -| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `name` | Providing a custom name for the PDS application | -| `ports` | The port that is used by the PDS application | -| `state` | The current app state (default: `CONF`) | -| `configuration/customization/command` | The command to be executed by PDS (`getPallet`, `getRack`, `getItem`, `volCheck` or `nop`). `nop` corresponds to no operation. After a command is executed by the PDS, the command parameter is set back to `nop` value. | | -| `configuration/customization/getPallet` | Configure the `getPallet` command. Refer to [the `getPallet` documentation](../GetPallet/getPallet.md) for further details on the relevant parameters. | -| `configuration/customization/getRack` | Configure the `getRack` command. Refer to [the `getRack` documentation](../GetRack/getRack.md) for further details on the relevant parameters. | -| `configuration/customization/getItem` | Configure the `getItem` command. Refer to [the `getItem` documentation](../GetItem/getItem.md) for further details on the relevant parameters. | -| `configuration/customization/volCheck` | Configure the `volCheck` command. Refer to [the `volCheck` documentation](../VolCheck/volCheck.md) for further details on the relevant parameters. | -| `configuration/port/mode` | To designate the measurement range: 2 or 4 meters. This parameter is configurable only in `CONF` state. | -| `configuration/port/acquisition/channelValue` | Channel value where each channel corresponding to a different modulation frequency. This parameter is configurable only in `CONF` state. | -| `configuration/port/acquisition/exposureLong` | Parametrize the long exposure time. This parameter is configurable only in `CONF` state. The default value works for the majority of use cases and we do not recommend changing this value. | -| `configuration/port/acquisition/exposureShort` | Parametrize the short exposure time. This parameter is configurable only in `CONF` state. The default value works for the majority of use cases and we do not recommend changing this value. | -| `configuration/port/acquisition/offset` | Shifts the starting point of the measured range. This parameter is configurable only in `CONF` state. The default value works for the majority of use cases and we do not recommend changing this value. | \ No newline at end of file +These parameters are available at the `configuration/port` path in the JSON configuration. +These parameters mirror the port parameters available at the `/ports/portX` path in the JSON configuration. +They are used to configure acquisition settings and filtering for the port. Typically, these settings to not need to be updated, as the application provide reasonable defaults for most use cases. \ No newline at end of file diff --git a/PDS/Configuration/resources/default_app.png b/PDS/Configuration/resources/default_app.png index bcbc029..83fa65f 100644 Binary files a/PDS/Configuration/resources/default_app.png and b/PDS/Configuration/resources/default_app.png differ diff --git a/PDS/Configuration/resources/instantiate_app.png b/PDS/Configuration/resources/instantiate_app.png index 3a69327..88f32b6 100644 Binary files a/PDS/Configuration/resources/instantiate_app.png and b/PDS/Configuration/resources/instantiate_app.png differ diff --git a/PDS/Cpp/CMakeLists.txt b/PDS/Cpp/CMakeLists.txt index f6662b3..5a7933d 100644 --- a/PDS/Cpp/CMakeLists.txt +++ b/PDS/Cpp/CMakeLists.txt @@ -1,6 +1,6 @@ cmake_minimum_required(VERSION 3.11) -project(o3r_examples CXX) +project(pds_examples CXX) #Global compiler flags set(CMAKE_BUILD_TYPE Release) # Release or Debug @@ -13,35 +13,9 @@ find_package(ifm3d 1.5.3 CONFIG REQUIRED COMPONENTS device framegrabber deserialize ) -## Volume check example -add_executable(vol_check vol_check.cpp) -target_link_libraries(vol_check - ifm3d::device - ifm3d::framegrabber - ) - -## Get pallet example -add_executable(get_pallet get_pallet.cpp) -target_link_libraries(get_pallet - ifm3d::device - ifm3d::framegrabber - ) - -## Get item example +# Get item example add_executable(get_item get_item.cpp) target_link_libraries(get_item ifm3d::device ifm3d::framegrabber - ) -## Get rack example -add_executable(get_rack get_rack.cpp) -target_link_libraries(get_rack - ifm3d::device - ifm3d::framegrabber - ) -## Get rack example -add_executable(get_flags get_flags.cpp) -target_link_libraries(get_flags - ifm3d::device - ifm3d::framegrabber - ) \ No newline at end of file + ) \ No newline at end of file diff --git a/PDS/Cpp/get_flags.cpp b/PDS/Cpp/get_flags.cpp deleted file mode 100644 index b975e51..0000000 --- a/PDS/Cpp/get_flags.cpp +++ /dev/null @@ -1,118 +0,0 @@ -#include -#include -#include -#include -#include - -using namespace std::chrono_literals; - -void FlagsCallback(ifm3d::Frame::Ptr frame) { - if (frame->HasBuffer(ifm3d::buffer_id::O3R_RESULT_ARRAY2D)){ - std::cout << "Received a frame" << std::endl; - auto result_buffer = frame->GetBuffer(ifm3d::buffer_id::O3R_RESULT_ARRAY2D); - // The result_buffer is a 1D bytes array, so we need to - // reformat it into a 2D array of uint16_t - std::vector result_1D_array = std::vector(result_buffer.ptr(0), result_buffer.ptr(0) + result_buffer.size()/sizeof(uint16_t)); - std::vector> result_array(172, std::vector(224)); - for (int i = 0; i < 172; ++i) { - for (int j = 0; j < 224; ++j) { - result_array[i][j] = result_1D_array[i * 224 + j]; - } - } - std::cout << "Flag for pixel (100, 100): " - << result_array[100][100] - << std::endl; - } -} - -int main(){ - //////////////////////////////// - // Device specific configuration - //////////////////////////////// - std::string IP = "192.168.0.69"; - std::string CAMERA_PORT = "port2"; - std::string APP_PORT = "app0"; - auto o3r = std::make_shared(IP); - - //////////////////////////////// - // Setup the application - //////////////////////////////// - try { - o3r->Reset("/applications"); - } catch (ifm3d::Error &err) { - std::cerr << "Error resetting the camera: " << err.what() << std::endl; - return -1; - } - ifm3d::json ports_config = { - {"ports",{ - {CAMERA_PORT, { - {"processing", { - {"extrinsicHeadToUser", { - {"transX", 0.0}, - {"transY", 0.0}, - {"transZ", 0.2}, - {"rotX", 0.0}, - {"rotY", 1.57}, - {"rotZ", -1.57} - }} - }} - }} - }} - }; - - std::cout << "Setting port configuration:" - << ports_config << std::endl; - o3r->Set(ports_config); - - ifm3d::json app_config = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"class", "pds"}, - {"ports", {CAMERA_PORT}}, - {"state", "IDLE"} - }} - }} - }} - }; - std::cout << "Setting app configuration:" - << app_config << std::endl; - o3r->Set(app_config); - - //////////////////////////////// - // Setup the framegrabber to receive frames - // when the application is triggered. - //////////////////////////////// - auto fg = std::make_shared(o3r, o3r->Port(APP_PORT).pcic_port); - fg->Start({ifm3d::buffer_id::O3R_RESULT_ARRAY2D}); - fg->OnNewFrame(&FlagsCallback); - - //////////////////////////////// - // Trigger the application - //////////////////////////////// - std::this_thread::sleep_for(2s); - ifm3d::json getPallet_parameters = { - {"depthHint", 1.2}, - {"palletIndex", 0} - }; - ifm3d::json getPallet_command = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"configuration", { - {"customization",{ - {"command", "getPallet"}, - {"getPallet", getPallet_parameters} - }} - }} - }} - }} - }} - }; - std::cout << "Triggering the volCheck command" << std::endl; - o3r->Set(getPallet_command); - - std::this_thread::sleep_for(3s); - fg->Stop(); - return 0; -} \ No newline at end of file diff --git a/PDS/Cpp/get_pallet.cpp b/PDS/Cpp/get_pallet.cpp deleted file mode 100644 index 78e66bb..0000000 --- a/PDS/Cpp/get_pallet.cpp +++ /dev/null @@ -1,111 +0,0 @@ -#include -#include -#include -#include -#include - -using namespace std::chrono_literals; - -void PalletCallback(ifm3d::Frame::Ptr frame) { - if (frame->HasBuffer(ifm3d::buffer_id::O3R_RESULT_JSON)){ - std::cout << "Received a frame" << std::endl; - auto result_json_buffer = frame->GetBuffer(ifm3d::buffer_id::O3R_RESULT_JSON); - std::string result_json_string = std::string(result_json_buffer.ptr(0), strnlen(result_json_buffer.ptr(0), result_json_buffer.size())); - ifm3d::json result_json = ifm3d::json::parse(result_json_string); - std::cout << "Detected pallet(s): " - << result_json["getPallet"]["pallet"] - << std::endl; - } -} - -int main(){ - //////////////////////////////// - // Device specific configuration - //////////////////////////////// - std::string IP = "192.168.0.69"; - std::string CAMERA_PORT = "port2"; - std::string APP_PORT = "app0"; - auto o3r = std::make_shared(IP); - - //////////////////////////////// - // Setup the application - //////////////////////////////// - try { - o3r->Reset("/applications"); - } catch (ifm3d::Error &err) { - std::cerr << "Error resetting the camera: " << err.what() << std::endl; - return -1; - } - ifm3d::json ports_config = { - {"ports",{ - {CAMERA_PORT, { - {"processing", { - {"extrinsicHeadToUser", { - {"transX", 0.0}, - {"transY", 0.0}, - {"transZ", 0.2}, - {"rotX", 0.0}, - {"rotY", 1.57}, - {"rotZ", -1.57} - }} - }} - }} - }} - }; - - std::cout << "Setting port configuration:" - << ports_config << std::endl; - o3r->Set(ports_config); - - ifm3d::json app_config = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"class", "pds"}, - {"ports", {CAMERA_PORT}}, - {"state", "IDLE"} - }} - }} - }} - }; - std::cout << "Setting app configuration:" - << app_config << std::endl; - o3r->Set(app_config); - - //////////////////////////////// - // Setup the framegrabber to receive frames - // when the application is triggered. - //////////////////////////////// - auto fg = std::make_shared(o3r, o3r->Port(APP_PORT).pcic_port); - fg->Start({ifm3d::buffer_id::O3R_RESULT_JSON}); - fg->OnNewFrame(&PalletCallback); - - //////////////////////////////// - // Trigger the application - //////////////////////////////// - std::this_thread::sleep_for(2s); - ifm3d::json getPallet_parameters = { - {"depthHint", 1.2}, - {"palletIndex", 0} - }; - ifm3d::json getPallet_command = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"configuration", { - {"customization",{ - {"command", "getPallet"}, - {"getPallet", getPallet_parameters} - }} - }} - }} - }} - }} - }; - std::cout << "Triggering the volCheck command" << std::endl; - o3r->Set(getPallet_command); - - std::this_thread::sleep_for(3s); - fg->Stop(); - return 0; -} \ No newline at end of file diff --git a/PDS/Cpp/get_rack.cpp b/PDS/Cpp/get_rack.cpp deleted file mode 100644 index 8ed6528..0000000 --- a/PDS/Cpp/get_rack.cpp +++ /dev/null @@ -1,113 +0,0 @@ -#include -#include -#include -#include -#include - -using namespace std::chrono_literals; - -void RackCallback(ifm3d::Frame::Ptr frame) { - if (frame->HasBuffer(ifm3d::buffer_id::O3R_RESULT_JSON)){ - std::cout << "Received a frame" << std::endl; - auto result_json_buffer = frame->GetBuffer(ifm3d::buffer_id::O3R_RESULT_JSON); - std::string result_json_string = std::string(result_json_buffer.ptr(0), strnlen(result_json_buffer.ptr(0), result_json_buffer.size())); - ifm3d::json result_json = ifm3d::json::parse(result_json_string); - std::cout << "Detected rack: " - << result_json["getRack"] - << std::endl; - } -} - -int main(){ - //////////////////////////////// - // Device specific configuration - //////////////////////////////// - std::string IP = "192.168.0.69"; - std::string CAMERA_PORT = "port2"; - std::string APP_PORT = "app0"; - auto o3r = std::make_shared(IP); - - //////////////////////////////// - // Setup the application - //////////////////////////////// - try { - o3r->Reset("/applications"); - } catch (ifm3d::Error &err) { - std::cerr << "Error resetting the camera: " << err.what() << std::endl; - return -1; - } - ifm3d::json ports_config = { - {"ports",{ - {CAMERA_PORT, { - {"processing", { - {"extrinsicHeadToUser", { - {"transX", 0.0}, - {"transY", 0.0}, - {"transZ", 0.2}, - {"rotX", 0.0}, - {"rotY", 1.57}, - {"rotZ", -1.57} - }} - }} - }} - }} - }; - - std::cout << "Setting port configuration:" - << ports_config << std::endl; - o3r->Set(ports_config); - - ifm3d::json app_config = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"class", "pds"}, - {"ports", {CAMERA_PORT}}, - {"state", "IDLE"} - }} - }} - }} - }; - std::cout << "Setting app configuration:" - << app_config << std::endl; - o3r->Set(app_config); - - //////////////////////////////// - // Setup the framegrabber to receive frames - // when the application is triggered. - //////////////////////////////// - auto fg = std::make_shared(o3r, o3r->Port(APP_PORT).pcic_port); - fg->Start({ifm3d::buffer_id::O3R_RESULT_JSON}); - fg->OnNewFrame(&RackCallback); - - //////////////////////////////// - // Trigger the application - //////////////////////////////// - std::this_thread::sleep_for(2s); - ifm3d::json getRack_parameters = { - {"depthHint", 1.2}, - {"horizontalDropPosition", "left"}, - {"verticalDropPosition", "interior"}, - {"zHint", -0.4} - }; - ifm3d::json getRack_command = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"configuration", { - {"customization",{ - {"command", "getRack"}, - {"getRack", getRack_parameters} - }} - }} - }} - }} - }} - }; - std::cout << "Triggering the volCheck command" << std::endl; - o3r->Set(getRack_command); - - std::this_thread::sleep_for(3s); - fg->Stop(); - return 0; -} \ No newline at end of file diff --git a/PDS/Cpp/vol_check.cpp b/PDS/Cpp/vol_check.cpp deleted file mode 100644 index ec4b861..0000000 --- a/PDS/Cpp/vol_check.cpp +++ /dev/null @@ -1,115 +0,0 @@ -#include -#include -#include -#include -#include - -using namespace std::chrono_literals; - -void VolumeCallback(ifm3d::Frame::Ptr frame) { - if (frame->HasBuffer(ifm3d::buffer_id::O3R_RESULT_JSON)){ - std::cout << "Received a frame" << std::endl; - auto result_json_buffer = frame->GetBuffer(ifm3d::buffer_id::O3R_RESULT_JSON); - std::string result_json_string = std::string(result_json_buffer.ptr(0), strnlen(result_json_buffer.ptr(0), result_json_buffer.size())); - ifm3d::json result_json = ifm3d::json::parse(result_json_string); - std::cout << "Number of pixels in the volume: " - << result_json["volCheck"]["numPixels"] - << std::endl; - } -} - -int main(){ - //////////////////////////////// - // Device specific configuration - //////////////////////////////// - std::string IP = "192.168.0.69"; - std::string CAMERA_PORT = "port2"; - std::string APP_PORT = "app0"; - auto o3r = std::make_shared(IP); - - //////////////////////////////// - // Setup the application - //////////////////////////////// - try { - o3r->Reset("/applications"); - } catch (ifm3d::Error &err) { - std::cerr << "Error resetting the camera: " << err.what() << std::endl; - return -1; - } - ifm3d::json ports_config = { - {"ports",{ - {CAMERA_PORT, { - {"processing", { - {"extrinsicHeadToUser", { - {"transX", 0.0}, - {"transY", 0.0}, - {"transZ", 0.2}, - {"rotX", 0.0}, - {"rotY", 1.57}, - {"rotZ", -1.57} - }} - }} - }} - }} - }; - - std::cout << "Setting port configuration:" - << ports_config << std::endl; - o3r->Set(ports_config); - - ifm3d::json app_config = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"class", "pds"}, - {"ports", {CAMERA_PORT}}, - {"state", "IDLE"} - }} - }} - }} - }; - std::cout << "Setting app config:" - << app_config << std::endl; - o3r->Set(app_config); - - //////////////////////////////// - // Setup the framegrabber to receive frames - // when the application is triggered. - //////////////////////////////// - auto fg = std::make_shared(o3r, o3r->Port(APP_PORT).pcic_port); - fg->Start({ifm3d::buffer_id::O3R_RESULT_JSON}); - fg->OnNewFrame(&VolumeCallback); - - //////////////////////////////// - // Trigger the application - //////////////////////////////// - std::this_thread::sleep_for(2s); - ifm3d::json volCheck_parameters = { - {"xMin", 1}, - {"xMax", 2}, - {"yMin", -0.5}, - {"yMax", 0.5}, - {"zMin", 0.0}, - {"zMax", 0.4}, - }; - ifm3d::json volCheck_command = { - {"applications",{ - {"instances",{ - {APP_PORT,{ - {"configuration", { - {"customization",{ - {"command", "volCheck"}, - {"volCheck", volCheck_parameters} - }} - }} - }} - }} - }} - }; - std::cout << "Triggering the volCheck command" << std::endl; - o3r->Set(volCheck_command); - - std::this_thread::sleep_for(3s); - fg->Stop(); - return 0; -} \ No newline at end of file diff --git a/PDS/Diagnostics/diagnostics.md b/PDS/Diagnostics/diagnostics.md index e7e76a2..44208ab 100644 --- a/PDS/Diagnostics/diagnostics.md +++ b/PDS/Diagnostics/diagnostics.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Diagnostics | Error code | Diagnostics | Description | diff --git a/PDS/Examples/examples.md b/PDS/Examples/examples.md new file mode 100644 index 0000000..f2038d2 --- /dev/null +++ b/PDS/Examples/examples.md @@ -0,0 +1,3 @@ +# Code examples + +Python and C++ code examples are available in the [ifm3d-examples](https://github.com/ifm/ifm3d-examples) repository. \ No newline at end of file diff --git a/PDS/GetPallet/appendix-changing_projection_volume.md b/PDS/GetPallet/appendix-changing_projection_volume.md index 5b2f012..348c006 100644 --- a/PDS/GetPallet/appendix-changing_projection_volume.md +++ b/PDS/GetPallet/appendix-changing_projection_volume.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Appendix - Changing the projection volume parameters for non-standard detection If the default projection volume doesn't meet your needs, you have two options. @@ -46,7 +42,7 @@ To change the coordinates of the projection volume, the following parameter need Note that this parameter can be edited for each pallet (up to ten pallets can be detected). -To set this parameter in Python or C++, you can use the respective [get_pallet.py](../Python/get_pallet.py) and [get_pallet.cpp](../Cpp/get_pallet.cpp) examples and edit the parameters of the JSON command. For example, for a shift along the Y axis of 50 cm and along the Z axis of 10 cm, you can do: +To set this parameter in Python or C++, you can use the respective [get_pallet.py](https://github.com/ifm/ifm3d-examples/tree/main/ovp8xx/python/ovp8xxexamples/pds/get_pallet.py) and [get_pallet.cpp](https://github.com/ifm/ifm3d-examples/tree/main/ovp8xx/cpp/pds/get_pallet.cpp) examples and edit the parameters of the JSON command. For example, for a shift along the Y axis of 50 cm and along the Z axis of 10 cm, you can do: :::::{tabs} ::::{group-tab} Python ```python @@ -131,7 +127,7 @@ This approach is the only one available in cases where the camera is mounted sid ![sideway](./resources/sideway.svg) -# Appendix - Changing the getPallet parameters: +# Appendix - Changing the `getPallet` parameters: ::: {warning} The following parameter changes shall only be done with care! diff --git a/PDS/GetPallet/getPallet.md b/PDS/GetPallet/getPallet.md index 950993c..6e4960f 100644 --- a/PDS/GetPallet/getPallet.md +++ b/PDS/GetPallet/getPallet.md @@ -1,30 +1,40 @@ ---- -nosearch: true ---- - # `getPallet` -The `getPallet` functionality of PDS is designed to detect the position and orientation of up to 10 pallets in the vicinity of autonomous and semi-autonomous pallet handling vehicles. Typically, such a system has a priori knowledge from warehouse management, such as the approximate distance to the pallet and the type of pallet. +The `getPallet` functionality of PDS is designed to detect the position and orientation of up to 10 pallets in the vicinity of autonomous and semi-autonomous pallet handling vehicles. +Typically, such a system has a priori knowledge from warehouse management, such as the approximate distance to the pallet and the type of pallet. `getPallet` supports the picking operation by determining the exact location and orientation of the pallet. ## Usage guidelines The typical use cases for `getPallet` are pallets with two pockets, either with broad blocks or thin stringers as vertical support structures. -![`getPallet` Usage](resources/getPallet_usage.png) By default, PDS is able to detect pallets with the following characteristics: -- For a block-type pallet: - - The pockets should be between 0.24 and 0.44 m, - - The blocks should be between 0.05 and 0.4 m. -- For a stringer-type pallet: - - The pockets should be between 0.4 and 0.48 m, - - The stringers should be between 0.02 and 0.08 m. -Two pocket pallets with different dimensions than the ones stated above will require specific configuration of the algorithm. Reach out to your ifm representative or to the support team for more details. +| Pallet name | Dimensions | Image | +| ----------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------- | +| Block pallet (`palletIndex=0`) | The pockets width should be between 0.24 and 0.44 m, | ![Block pallet](./resources/block_pallet.png) | +| | The pockets height should be between 0.05 and 0.15 m, | | +| | The blocks should be between 0.05 and 0.40 m. | | +| EPAL side pallet (`palletIndex = 2`) | The pockets width should be between 0.23 and 0.44 m, | ![EPAL side pallet](./resources/EPAL_side_pallet.png) | +| | The pockets height should be between 0.10 and 0.15 m, | | +| | The blocks should be between 0.10 and 0.16 m. | | +| Stringer pallet* (`palletIndex = 1`) | The pockets width should be between 0.40 and 0.55 m, | ![Stringer pallet](./resources/stringer_pallet.png) | +| | The pockets height should be between 0.0.5 and 0.15 m, | | +| | The stringers should be between 0.02 and 0.08 m. | | + +* Stringer pallets generally have thin vertical structures and wider pockets. + +### Best Practices + +- The range for the pocket dimensions are designed to detect large range of pallets. It is always recommended to lower the interval of to your specific pallets for more robust detections and lower the risk of false-positive detections. +- Even the `palletIndex == 2` is standardized for `EPAL_SIDE`, it can also be used for the other pallets without a bottom board if the dimensions are matching to `EPAL_SIDE`. +- Note that only the dimensions of the center block are important. The size of the side blocks can be different from the one specified above and can be parametrized under `configuration/parameter/getPallet//stringer`. +- Two pocket pallets with different dimensions than the ones stated above will require specific configuration of the algorithm. To adjust the dimensions of the pockets or the stringers, edit the predefined templates from [3 .... 10] as [0,1,2] are reserved for standard pallets and set the pocket and stringer sizes. See more details at [Custom pallet templates](#custom-pallet-templates). + +## Configuration -## Input To execute the `getPallet` command, the `command` parameter of the JSON configuration has to be set to `getPallet`. -Additionally, extra parameters can be provided such as the estimated distance to the pallet, `depthHint`, the type of pallet, `palletIndex`, and the order in which the results should be displayed, `palletOrder`. +Additionally, extra parameters can be provided such as the estimated distance to the pallet, `depthHint`, the type of pallet, `palletIndex`, and the order in which the results should be displayed in the list of detected pallets, `palletOrder`. An example is provided below in JSON, assuming an instantiated PDS application `app0`. Note that when using the ifm Vision Assistant, the same structure is reflected in the application parameters. ```json @@ -49,14 +59,23 @@ Note that when using the ifm Vision Assistant, the same structure is reflected i ``` :::{note} Only the `command` parameter has to be provided. The other parameters are optional. -If they are left blank, the default settings will be used. +If they are left blank, the algorithm will use the parameters from most recent execution of that command will be applied, or the set of saved settings for the application, if they exist. ::: -| Name | Description | Default Value | -| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------- | -| `depthHint` | Approximate distance (in meters along the X axis) between the origin of the calibrated coordinate system and the pallet. Providing an accurate depth hint allows the algorithm to target a specific area of the scene for pallet detection and speeds up processing times.
Zero or a negative value can be passed to use automatic distance detection. **Note that automatic detection works best with fully loaded pallets and may fail with empty pallets.** | `1.5` | -| `palletIndex` | Defines the type of pallet to detect.
0: block,
1: stringer,
2: EPAL, side view. | `0` | -| `palletOrder` | If multiple pallets are detected in the field of view, you can set the order of pallets based on three properties:
`scoreDescending` (default): the pallet order will be based on the detection score (highest to lowest), corresponding to how well the pallet matches the expected pallet shape;
`zAscending`/`zDescending`: the pallet order will be based on the height from the floor along the calibrated Z axis (`zAscending`: lower to upper, `zDescending`: upper to lower). | `scoreDescending` | +### Customization parameters at `configuration/customization` + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :---------------------- | :------ | :---------------------------------------------------------------------------------------------------------------------- | :-------------- | :------ | :------ | :------------------------------------------------- | +| `getPallet.depthHint` | number | Estimated distance between pallet and calibrated coordinate system center in meters. Set to <=0 for automatic detection | 1.5 | N/A | N/A | N/A | +| `getPallet.palletIndex` | integer | Index of the parameter set between 0 and 9. Currently configured IDs: | 0 | 0 | 9 | N/A | +| `getPallet.palletOrder` | string | Set the order in the list of detected pallets | scoreDescending | N/A | N/A | `['scoreDescending', 'zDescending', 'zAscending']` | + + +:::{note} +When detecting multiple pallets simultaneously, the system allows only one `palletIndex` value to be specified, meaning that all pallets must be of the same type. +This means that all the pallets should have the same pocket and stringer dimensions. +Alternatively, a relaxed parameter set can be used to encompass multiple types of pallets, using [a custom template](#custom-pallet-templates). This will ensure finding pallets of different dimensions, but will increase the necessary processing time. +::: The default values for each parameter can be different depending on the firmware version used. We recommend to check the current default, min and max values for each parameter using [the schema](../../Technology/configuration.md#json-schema). @@ -64,42 +83,82 @@ The default values for each parameter can be different depending on the firmware Other variants of pallets, having three or more pockets for example, require adjustments of the PDS settings. Reach out to your ifm representative or to the support team for more details. ::: -## Volumes of interest - -When searching for a pallet, the PDS algorithm does not explore the entire space, but focuses the search on two delimited volumes, depending on the case. +### Custom pallet templates + +By default, PDS provides three pallet templates: block (`palletIndex = 0`), stringer (`palletIndex = 1`), and EPAL side (`palletIndex = 2`). +It is possible for the user to define custom pallet templates for any two pocket pallet by modifying the parameters at `configuration/parameter/X`, where `X` is the index chosen for the template number (up to 9). + +Once the template is defined, the corresponding `palletIndex` can be chosen when triggering the `getPallet` command. + + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :------------------------------------------------- | :------ | :---------------------------------------------------------------------------------------------------------------- | :---------- | :------ | :------ | :------------------------------- | +| `getPallet.X.depthEstimation.fallbackDepthHint` | number | default value for Depth Hint, if auto-discovery fails | 1.5 | 0.1 | N/A | N/A | +| `getPallet.X.depthEstimation.minNumPixels` | integer | minimum number of valid pixels in the VOI | 10 | 1 | N/A | N/A | +| `getPallet.X.depthEstimation.voi.xMax` | number | Maximum x-coordinate to estimate depth hint (meters) | 5 | N/A | N/A | N/A | +| `getPallet.X.depthEstimation.voi.xMin` | number | Minmum x-coordinate to estimate depth hint (meters) | 0.5 | N/A | N/A | N/A | +| `getPallet.X.depthEstimation.voi.yMax` | number | Maximum y-coordinate to estimate depth hint (meters) | 0.4 | N/A | N/A | N/A | +| `getPallet.X.depthEstimation.voi.yMin` | number | Minmum y-coordinate to estimate depth hint (meters) | -0.4 | N/A | N/A | N/A | +| `getPallet.X.depthEstimation.voi.zMax` | number | Maximum y-coordinate to estimate depth hint (meters) | 0.4 | N/A | N/A | N/A | +| `getPallet.X.depthEstimation.voi.zMin` | number | Minmum z-coordinate to estimate depth hint (meters) | -0.1 | N/A | N/A | N/A | +| `getPallet.X.detectionPipeline` | string | Pipeline type for detecting pallet features | findCorners | N/A | N/A | `['findCorners', 'findPockets']` | +| `getPallet.X.localizePallets.allowPitchEstimation` | boolean | Whether this pallet template can reliably estimate pitch. Intended for machined, high-quality calibration targets | False | N/A | N/A | N/A | +| `getPallet.X.localizePallets.yawTol` | number | Maximum absolute yaw angle in radians | 0.4 | 0 | N/A | N/A | +| `getPallet.X.name` | string | Descriptive name of the recipe. Note that the name will be truncated to PDS_MAX_PALLET_NAME_STRLEN. | Block | N/A | N/A | N/A | +| `getPallet.X.orthoProjection.voi.xMax` | number | Maximum x-coordinate for projection, relative to the depth hint (meters) | 0.3 | -0.5 | 0.5 | N/A | +| `getPallet.X.orthoProjection.voi.xMin` | number | Minmum x-coordinate for projection, relative to the depth hint (meters) | -0.3 | -0.5 | 0.5 | N/A | +| `getPallet.X.orthoProjection.voi.yMax` | number | Maximum y-coordinate for projection in calibrated coordinate system (meters) | 1 | -2 | 2 | N/A | +| `getPallet.X.orthoProjection.voi.yMin` | number | Minmum y-coordinate for projection in calibrated coordinate system (meters) | -1 | -2 | 2 | N/A | +| `getPallet.X.orthoProjection.voi.zMax` | number | Maximum y-coordinate for projection in calibrated coordinate system (meters) | 0.6 | -2 | 2 | N/A | +| `getPallet.X.orthoProjection.voi.zMin` | number | Minmum z-coordinate for projection in calibrated coordinate system (meters) | -0.6 | -2 | 2 | N/A | +| `getPallet.X.pocket.maxHeight` | number | Maximum pocket height in meters (ignored when finding whole pockets) | 0.15 | N/A | N/A | N/A | +| `getPallet.X.pocket.maxWidth` | number | Maximum pocket width in meters (ignored when finding whole pockets) | 0.44 | N/A | N/A | N/A | +| `getPallet.X.pocket.minHeight` | number | Minimum pocket height in meters | 0.05 | N/A | N/A | N/A | +| `getPallet.X.pocket.minWidth` | number | Minimum pocket width in meters | 0.24 | N/A | N/A | N/A | +| `getPallet.X.stringer.maxWidthCenter` | number | Maximum width of the center stringer in meters | 0.4 | N/A | N/A | N/A | +| `getPallet.X.stringer.minWidthCenter` | number | Minimum width of the center stringer in meters | 0.05 | N/A | N/A | N/A | + +#### Detection pipeline +When defining a custom pallet template, the user has to chose the proper detection pipeline. +Two options are available: `findPockets` and `findCorners`. Depending on the use case, one option might be more suitable than the other. + +`findPocket` relies on a strict pocket dimension. In this case, only the `minWidth` and `minHeight` pocket parameters are used, and the `maxWidth` and `maxHeight` are ignored. +It offers limited flexibility in terms of pocket size, but it is the most robust option when shrink wrap is partially covering a pocket. It is also able to detect open pockets, for example in the case of the EPAL side. + +`findCorners` can handle a wider variety of pocket sizes, as it relies on finding four corners, rather than a pocket shape. However, it cannot handle open pockets. It will also be typically slower to produce a result. + +![`localizePallet` and `detectionPipeline` parameters](./resources/pallet_params_1.png) + +![`pocket` and `stringer` parameters](./resources/pallet_params_5.png) + +#### Volumes of interest + +When searching for a pallet, the PDS algorithm does not explore the entire space, but focuses the search on the `orthoProjection` volume. This is done to reduce processing time by limiting the search area. -These two volumes are the depth estimation volume, which is only used when using an automatic `depthHint`, and the projection volume. -When using the ifm Vision Assistant, the two volumes can be displayed as shown below, where the green box represents the projection volume and the blue box represents the depth estimation volume. +The `orthoProjection` volume is defined in the parameters (at `/applications/instances/appX/configuration/parameter/getPallet/n/orthoProjection`, where `n` is the pallet template number), or automatically if using the automatic depth hint feature (`depthHint = -1`). + +When using the ifm Vision Assistant, the`orthoProjection` volume, and the `depthHint` search volume can be displayed as shown below, where the green box represents the projection volume and the blue box represents the depth estimation volume. ![volumes](./resources/pds_volumes_of_interest.png) -### Depth estimation volume +##### Depth estimation volume The depth estimation volume, `depthEstimationVoi`, defines the volume of interest for automatic depth estimation. Within this volume, a search will be conducted to estimate the distance of the pallet's front face from the reference coordinate system. -The default values for depth estimation volume with respect to reference coordinate system are as follows, in meters: -- X axis: `[1, 3]` -- Y axis: `[-0.4, 0.4]` -- Z axis: `[-0.1, 0.4]` In most cases, we recommend using a predefined depth hint, in which case this volume will not be used. +Alternatively, the depth estimation volume can be adjusted in the parameters, at `/applications/instances/appX/configuration/parameter/getPallet/n/depthEstimation/voi`. + +![`depthEstimation` parameters](./resources/pallet_params_2.png) +![`depthEstimation` parameters](./resources/pallet_params_3.png) -### Projection volume -The projection volume, `projectionVoi`, defines the volume of interest for detecting the pose of the pallet pockets and center beam. +##### Ortho-projection volume +The projection volume, `orthoProjection/voi`, defines the volume of interest for detecting the pose of the pallet pockets and center beam. This volume is centered around the `depthHint`, estimated or provided by the user, which represents the estimated distance to the front face of the pallet. -The default values for the projection volume with respect to the reference coordinate system are as follows, in meters: -- X axis: `[depthHint - 0.2, depthHint + 0.2]` -- Y axis: `[-1, 1]` -- Z axis: `[-0.6, 0.6]` -The default values of the projection volume are displayed in the figure below, using the forks coordinates system as a reference. +Using the default values for the projection volume is typically sufficient. +However, to achieve optimal processing speed, it is recommended to tighten -![projectionVOI](./resources/projectionVOI.drawio.svg) - -:::{warning} -The size of the volumes of interest are protected parameters, which means that they cannot be configured by the user. -The system will display their default values in the output of each result after running the `getPallet` command. -If your application requires different settings, please contact the ifm support team for assistance. -::: +![`orthoProjection` parameters](./resources/pallet_params_4.png) ## Workflow @@ -108,14 +167,16 @@ The `getPallet` command expects the pallet to be in front of the forks and in th ![expected](resources/expected.drawio.svg) The `getPallet` command works as follows: -1. If the `depthHint` is set to a positive value (this is the recommended option), the user is expected to have a priori knowledge about where the pallet is with respect to the forks coordinate system. In this case, the pallet's pose estimation is performed inside the projection volume, which is set at the `depthHint` including +/- 0.2 m. -2. If the `depthHint` is set to zero or a negative value, then the pallet will be searched for in the depth estimation volume. The majority of the pixels inside the depth estimation volume should be on the plane of the front face of the pallet. Once the distance to the pallet is estimated, the projection volume is set at this distance including a +/- 0.2 m buffer in X direction on each side. The pixels in the projection volume will be used to estimate the position of the pallet. +1. If the `depthHint` is set to a positive value (this is the recommended option), the user is expected to have a priori knowledge about where the pallet is with respect to the forks coordinate system. In this case, the pallet's pose estimation is performed inside the ortho-projection volume. +2. If the `depthHint` is set to zero or a negative value, then the pallet will be searched for in the depth estimation volume. The majority of the pixels inside the depth estimation volume should be on the plane of the front face of the pallet. Once the distance to the pallet is estimated, the projection volume is set at this distance including a buffer in X direction on each side. The pixels in the projection volume will be used to estimate the position of the pallet. +3. The algorithm searches for a pallet that fits the template corresponding to the chosen `palletIndex`. If not pallet is found, the user should attempt to relax the settings, to avoid a failed detection and calling for rescue. + For further details of a typical `getPallet` trigger sequence see the flowchart below. ![flowchart](./resources/getPallet_flowchart.drawio.svg) -## Output +## Results The output of a `getPallet` command is formatted in JSON. An example JSON result, where the position of one pallet was identified, is shown below: @@ -154,8 +215,7 @@ An example JSON result, where the position of one pallet was identified, is show "z": 0.07671421766281128 }, "width": 0.3186997175216675 - }, - "score": 0.9556106328964233 + } } ], "depthEstimationVoi": { @@ -181,11 +241,10 @@ The output has three main components: `pallet`, `depthEstimationVoi` and `projec ### `pallet` This component of the JSON result lists all the detected pallets (up to 10). For each pallet, the following information is provided: -| Name | Description | -| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `angles` | Rotations components `rotX`, `rotY` and `rotZ` of the pallet, along the three axis of the calibrated coordinate system, in radians. | -| `center`, `left` and `right` | Position and size of the center beam, the left pocket and the right pocket respectively.
For each, the width and height is provided, as well as the coordinates of the center of the pocket or beam along the X, Y and Z axis. | -| `score` | The score of the pallet. The score corresponds to how well the pallet fits the pallet template.
Note that the score does *not* indicate how well a pallet is detected or whether a pallet is damaged; rather, it shows how closely the pallet matches the model pallet. | +| Name | Description | +| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `angles` | Rotations components `rotX`, `rotY` and `rotZ` of the pallet, along the three axis of the calibrated coordinate system, in radians. | +| `center`, `left` and `right` | Position and size of the center beam, the left pocket and the right pocket respectively.
For each, the width and height is provided, as well as the coordinates of the center of the pocket or beam along the X, Y and Z axis. | :::{note} Per default the pitch estimation `rotY` is disabled and will always display `0`. @@ -194,16 +253,7 @@ Per default the pitch estimation `rotY` is disabled and will always display `0`. ### `depthEstimationVoi` and `projectionVoi` The `depthEstimationVoi` and `projectionVoi` components provide the volumes that were used internally to detect the pallet: -| Name | Description | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `depthEstimationVoi` | Volume used in the algorithm to approximate the position of the front face of the pallet. | -| `projectionVoi` | Volume where the precise estimation of the positions of the two pockets, the center beam, and the rotation angle was performed. The pixels outside this area are discarded. | - -## Python example - -To initialize and configure the PDS application to execute the `getPallet` command, please see the code example below. - -:::{literalinclude} ../Python/get_pallet.py -:caption: getPallet.py -:language: python -::: \ No newline at end of file +| Name | Description | +| -------------------- | ----------------------------------------------------------------------------------------- | +| `depthEstimationVoi` | Volume used in the algorithm to approximate the position of the front face of the pallet. | +| `projectionVoi` | Volume where the detection takes place. The pixels outside this area are discarded. | diff --git a/PDS/GetPallet/resources/EPAL_side_pallet.png b/PDS/GetPallet/resources/EPAL_side_pallet.png new file mode 100644 index 0000000..d4eae91 Binary files /dev/null and b/PDS/GetPallet/resources/EPAL_side_pallet.png differ diff --git a/PDS/GetPallet/resources/block_pallet.png b/PDS/GetPallet/resources/block_pallet.png new file mode 100644 index 0000000..9fa4b52 Binary files /dev/null and b/PDS/GetPallet/resources/block_pallet.png differ diff --git a/PDS/GetPallet/resources/getPallet_flowchart.drawio.svg b/PDS/GetPallet/resources/getPallet_flowchart.drawio.svg index cb849fb..8ee256c 100644 --- a/PDS/GetPallet/resources/getPallet_flowchart.drawio.svg +++ b/PDS/GetPallet/resources/getPallet_flowchart.drawio.svg @@ -1,4 +1,4 @@ - + @@ -187,13 +187,20 @@
- Input the parameters + Provide + + depthHint + + , and + + palletIndex +
- Input the paramete... + Provide depthHint,... @@ -235,12 +242,12 @@ - - + + -
+
NO @@ -248,7 +255,25 @@
- + + NO + + + + + + + + +
+
+
+ NO +
+
+
+ + NO @@ -270,22 +295,83 @@ - - - + + + + + + +
+
+
+ Try another type of pallet ( + + palletIndex + + ), or adjust the + + depthHint + +
+
+
+ + + Try another type o... + + + + + + + + +
+
+
+ YES +
+
+
+ + + YES + + + + + + + +
+
+
+ Exhausted number of attempts +
+
+
+ + + Exhausted numbe... + + + + + + -
+
- Output the depth estimation volume and the projection volume. No pallet pose ouptut. + Pallet not found: call for rescue
- - Output the depth e... + + Pallet not found:... diff --git a/PDS/GetPallet/resources/pallet_params_1.png b/PDS/GetPallet/resources/pallet_params_1.png new file mode 100644 index 0000000..7f7567f Binary files /dev/null and b/PDS/GetPallet/resources/pallet_params_1.png differ diff --git a/PDS/GetPallet/resources/pallet_params_2.png b/PDS/GetPallet/resources/pallet_params_2.png new file mode 100644 index 0000000..99c273a Binary files /dev/null and b/PDS/GetPallet/resources/pallet_params_2.png differ diff --git a/PDS/GetPallet/resources/pallet_params_3.png b/PDS/GetPallet/resources/pallet_params_3.png new file mode 100644 index 0000000..93f279e Binary files /dev/null and b/PDS/GetPallet/resources/pallet_params_3.png differ diff --git a/PDS/GetPallet/resources/pallet_params_4.png b/PDS/GetPallet/resources/pallet_params_4.png new file mode 100644 index 0000000..12d7ecc Binary files /dev/null and b/PDS/GetPallet/resources/pallet_params_4.png differ diff --git a/PDS/GetPallet/resources/pallet_params_5.png b/PDS/GetPallet/resources/pallet_params_5.png new file mode 100644 index 0000000..7ae656e Binary files /dev/null and b/PDS/GetPallet/resources/pallet_params_5.png differ diff --git a/PDS/GetPallet/resources/stringer_pallet.png b/PDS/GetPallet/resources/stringer_pallet.png new file mode 100644 index 0000000..28b607f Binary files /dev/null and b/PDS/GetPallet/resources/stringer_pallet.png differ diff --git a/PDS/GetRack/getRack.md b/PDS/GetRack/getRack.md index 5bcfce9..ea0b411 100644 --- a/PDS/GetRack/getRack.md +++ b/PDS/GetRack/getRack.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # `getRack` The `getRack` functionality of PDS is designed to help an AGV safely place a pallet or load into a standard racking system. The `getRack` function has two phases: @@ -18,74 +14,83 @@ The origin of the rack coordinate system is the intersection of the detected upr If the detected upright and beam do not intersect, the origin of the rack coordinate will be placed at the intersection of the horizontal beam and the line which is orthogonal to both the beam and the upright. If the left upright is segmented then the Y-axis of the established rack coordinate system points to the right and if the right upright is segmented then the Y-axis will point towards the left. -## Input -### Command -The input to the `getRack` command must be provided in JSON. -Below is an example, assuming an instantiated PDS app `app0`: -```json -{"applications": { - "instances": { - "app0": { - "configuration": { - "customization": { - "command": "getRack", - "getRack": { - "depthHint": 1.2, - "horizontalDropPosition": "left", - "verticalDropPosition": "interior", - "zHint": -0.4, - "clearingVolume": { - "xMax": 1.2, - "xMin": -0.1, - "yMax": 1.3, - "yMin": 0.1, - "zMax": 0.4, - "zMin": 0.1 - } - } - } - } - } - } -}} -``` - -### `depthHint` -| Name | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `depthHint` | The depth hint is an approximate distance in meters along the X-axis from the origin of the reference coordinate system (typically the fork tines coordinate system) to the plane corresponding to the front face of the beam. | - -### `horizontalDropPosition` - -The drop operation of the pallet is based on the `horizontalDropPosition` parameter. -There are three configurations available for this parameter: - -| Name | Description | -| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `horizontalDropPosition` | `left` or `right` if the drop operation has to take place on the left or right side of the shelf respectively. The corresponding left or right upright of the rack is considered as a reference.
`center` if the user wishes to drop the pallet in the space between the loads already placed on each sides. In this case, the algorithm can use a detected up-beam on either side or another pallet on the same rack as a reference. The algorithm will sweep the beam left to right and the search stops when enough pixels are detected to form an upright candidate. If no reference is found on either side, the leftmost part of the horizontal beam will be used. | - -### `verticalDropPosition` - -This parameter informs the PDS about the drop location of the pallet. -Depending on the vertical drop position, this parameter can be configured as: - -| Name | Description | -| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `verticalDropPosition` | `interior` (default) if the load has to be placed on a horizontal beam.
`floor`: if the horizontal beam is absent and the load has to be placed directly on the floor. | - -### `zHint` - -| Name | Description | -| ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `zHint` | Approximate distance in meters along the Z-axis from the origin of the reference coordinate system (typically the fork tines coordinate system) to the upper edge of the horizontal beam on which the load is expected to be placed.
`getRack` uses this value to optimize the search volume (and thus performance).
The Z-hint should be within +/- 0.1 m of the true height of the front beam. | - -### `clearingVolume` - -| Name | Description | -| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `clearingVolume` | Volume, defined by its `xMin`, `xMax`, `yMin`, `yMax`, `zMin`, `zMax`, to sweep for obstacles with respect to the established origin of the racking system.
These values will typically be obtained from warehouse management and will correspond to the approximate volume of the load to be placed. | - -## Output +## Configuration + +### Command customization +The parameters listed below can be adjusted when triggering the `getRack` command, to define for example the distance to the rack (`depthHint`), or the drop position (`horizontalDropPosition`). +These parameters are described in the list below: + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :------------------------------- | :----- | :--------------------------------------------------------------------- | :------- | :------ | :------ | :---------------------------- | +| `getRack.clearingVolume.xMax` | number | Bounding box dimension of VOI along X-axis - Maximum | 1.2 | N/A | N/A | N/A | +| `getRack.clearingVolume.xMin` | number | Bounding box dimension of VOI along X-axis - Minimum | -0.1 | N/A | N/A | N/A | +| `getRack.clearingVolume.yMax` | number | Bounding box dimension of VOI along Y-axis - Maximum | 1.3 | N/A | N/A | N/A | +| `getRack.clearingVolume.yMin` | number | Bounding box dimension of VOI along Y-axis - Minimum | 0.1 | N/A | N/A | N/A | +| `getRack.clearingVolume.zMax` | number | Bounding box dimension of VOI along Z-axis - Maximum | 0.4 | N/A | N/A | N/A | +| `getRack.clearingVolume.zMin` | number | Bounding box dimension of VOI along Z-axis - Minimum | 0.1 | N/A | N/A | N/A | +| `getRack.depthHint` | number | Estimated distance between rack and coordinate system center in meters | 1.8 | 0 | N/A | N/A | +| `getRack.horizontalDropPosition` | string | Selection of the horizontal drop setting | left | N/A | N/A | `['left', 'center', 'right']` | +| `getRack.verticalDropPosition` | string | Selection of the vertical drop setting | interior | N/A | N/A | `['interior', 'floor']` | +| `getRack.zHint` | number | Estimated z-coordinate of the rack shelf in meters | -0.4 | N/A | N/A | N/A | + +### Custom rack parameters + +In typical scenarios, using the default parameters at `configuration/parameters/getRack` will be sufficient. +However, in some cases they need to be adjusted, for example to handle a custom type of rack. In this case, specific dimensions can be provided for the beam height, the upright width, etc. +See the list below for the details of the available parameters. + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :------------------------------------------ | :------ | :------------------------------------------------------------------------------------------ | :------ | :------ | :------ | :---------- | +| `getRack.beam.freeSpaceBelow` | number | Minimum free space required for a valid beam below the bottom edge in meters | 0.05 | 0 | N/A | N/A | +| `getRack.beam.histWidthThresh` | number | Width threshold in meters along y-axis for histogram to be considered as beam candidates | 1 | N/A | N/A | N/A | +| `getRack.beam.maxHeight` | number | Maximum height for beam validation in meters | 0.15 | N/A | N/A | N/A | +| `getRack.beam.minHeight` | number | Minimum height for beam validation in meters | 0.06 | N/A | N/A | N/A | +| `getRack.beam.voi.xMax` | number | Bounding box dimension of VOI along X-axis - Maximum | 0.23 | N/A | N/A | N/A | +| `getRack.beam.voi.xMin` | number | Bounding box dimension of VOI along X-axis - Minimum | -0.23 | N/A | N/A | N/A | +| `getRack.beam.voi.yMax` | number | Bounding box dimension of VOI along Y-axis - Maximum | 1.2 | N/A | N/A | N/A | +| `getRack.beam.voi.yMin` | number | Bounding box dimension of VOI along Y-axis - Minimum | -1.2 | N/A | N/A | N/A | +| `getRack.beam.voi.zMax` | number | Bounding box dimension of VOI along Z-axis - Maximum | 0.4 | N/A | N/A | N/A | +| `getRack.beam.voi.zMin` | number | Bounding box dimension of VOI along Z-axis - Minimum | -0.4 | N/A | N/A | N/A | +| `getRack.floorDrop.yRackOriginLeft` | number | Expected `y_val` of the rack origin for the HPOS = left drop | 0.61 | N/A | N/A | N/A | +| `getRack.floorDrop.yRackOriginRight` | number | Expected `y_val` of the rack origin for the HPOS = right drop | -0.61 | N/A | N/A | N/A | +| `getRack.floorDrop.yRackOriginTol` | number | Tolerance for difference in expected and computed rack origin | 0.3 | N/A | N/A | N/A | +| `getRack.rackVolCheck.obstThresh` | integer | Threshold for number of pixels in the rack shelf to be considered for obstacle flag | 10 | N/A | N/A | N/A | +| `getRack.transform.beamOnlyObstThresh` | integer | Number of obstacle pixels threshold during beam-only localization sweep | 5 | 1 | N/A | N/A | +| `getRack.transform.voi.xMax` | number | Bounding box dimension of VOI along X-axis - Maximum | 1.2 | N/A | N/A | N/A | +| `getRack.transform.voi.xMin` | number | Bounding box dimension of VOI along X-axis - Minimum | -0.3 | N/A | N/A | N/A | +| `getRack.transform.voi.yMax` | number | Bounding box dimension of VOI along Y-axis - Maximum | 1.2 | N/A | N/A | N/A | +| `getRack.transform.voi.yMin` | number | Bounding box dimension of VOI along Y-axis - Minimum | -1.2 | N/A | N/A | N/A | +| `getRack.transform.voi.zMax` | number | Bounding box dimension of VOI along Z-axis - Maximum | 0.6 | N/A | N/A | N/A | +| `getRack.transform.voi.zMin` | number | Bounding box dimension of VOI along Z-axis - Minimum | -0.3 | N/A | N/A | N/A | +| `getRack.transform.yRackOriginLeft` | number | Expected `y_val` of the rack origin for the HPOS = left drop | 0.61 | N/A | N/A | N/A | +| `getRack.transform.yRackOriginRight` | number | Expected `y_val` of the rack origin for the HPOS = right drop | -0.61 | N/A | N/A | N/A | +| `getRack.transform.yRackOriginTol` | number | Tolerance for difference in expected and computed rack origin | 0.3 | N/A | N/A | N/A | +| `getRack.upright.histHeightThresh` | number | Height threshold in meters along z-axis for histogram to be considered as upright candidates | 0.7 | N/A | N/A | N/A | +| `getRack.upright.maxWidth` | number | Maximum width for upright validation in meters | 0.12 | N/A | N/A | N/A | +| `getRack.upright.minWidth` | number | Minimum width for upright validation in meters | 0.03 | N/A | N/A | N/A | +| `getRack.upright.protrusionAboveBeamThresh` | number | Threshold for Protrusion above beam edge in meters to be considered for upright candidates | 0.05 | N/A | N/A | N/A | +| `getRack.upright.voi.xMax` | number | Bounding box dimension of VOI along X-axis - Maximum | 0.23 | N/A | N/A | N/A | +| `getRack.upright.voi.xMin` | number | Bounding box dimension of VOI along X-axis - Minimum | -0.23 | N/A | N/A | N/A | +| `getRack.upright.voi.yMax` | number | Bounding box dimension of VOI along Y-axis - Maximum | 1.2 | N/A | N/A | N/A | +| `getRack.upright.voi.yMin` | number | Bounding box dimension of VOI along Y-axis - Minimum | -1.2 | N/A | N/A | N/A | +| `getRack.upright.voi.zMax` | number | Bounding box dimension of VOI along Z-axis - Maximum | 0.6 | N/A | N/A | N/A | +| `getRack.upright.voi.zMin` | number | Bounding box dimension of VOI along Z-axis - Minimum | -0.6 | N/A | N/A | N/A | + +For a visual representation of the available parameters, refer to the images below: + +![`depthHint`, `zHint` and `dropPosition`](./resources/rack_params_1.png) +![`clearingVolume`](./resources/rack_params_2.png) +![center drop](./resources/rack_params_3.png) +![`beam` parameters](./resources/rack_params_4.png) +![`beam` parameters](./resources/rack_params_5.png) +![`floorDrop` parameters](./resources/rack_params_6.png) +![`rackVolCheck`](./resources/rack_params_7.png) +![`transform`](./resources/rack_params_8.png) +![`upright` parameters](./resources/rack_params_9.png) +![`upright` parameters](./resources/rack_params_10.png) + + +## Results The output of a `getRack` command is formatted in JSON. An example JSON result, where the position of the rack was identified is shown below: @@ -111,7 +116,16 @@ An example JSON result, where the position of the rack was identified is shown b "y": 0.636352, "z": 0.491981 }, - "score": 0, + "flagDescription": [ + "", + "", + "", + "", + "", + "", + "", + "" + ] "side": "left", "uprightVoi": { "xMax": 1.63, @@ -123,16 +137,15 @@ An example JSON result, where the position of the rack was identified is shown b } } ``` -### `angles`, `position`, `side` and `score` +### `angles`, `position` and `side` -| Name | Description | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `angles` | Three rotation components, `rotX`, `rotY` and `rotZ` of the rack coordinates system in radians. | -| `position` | The `x`, `y` anx `z` coordinates of the origin of the rack coordinate system. | -| `side` | Type of the rack coordinate system. Either "right" for right-handed or "left" for left-handed. | -| `score` | Detection score of the rack (0 or 1).
Note that the score does *not* correspond to how well the rack is detected, but rather how close to the reference rack it is. | +| Name | Description | +| ---------- | ----------------------------------------------------------------------------------------------- | +| `angles` | Three rotation components, `rotX`, `rotY` and `rotZ` of the rack coordinates system in radians. | +| `position` | The `x`, `y` anx `z` coordinates of the origin of the rack coordinate system. | +| `side` | Type of the rack coordinate system. Either "right" for right-handed or "left" for left-handed. | -### `flags` +### `flags` and `flagDescription` The `flags` provides a bitmask with debugging information for `getRack`. @@ -151,6 +164,8 @@ The `flags` provides a bitmask with debugging information for `getRack`. The resultant flag value is a decimal value representing which flags are activated or not. For example, if the value of the flag is set to 384 then the resultant binary value is `11000000`, that is, bit numbers 7 and 8 were set to 1 (`BAD_TRANSFORM` and `SHELF_OBSTACLE`). +The `flagDescription` parameter is a list of strings that correspond to the active `flags`. For example, if the `flags` value is `384`, the `flagDescription` list will be `["", "", "", "", "", "", "", "BAD_TRANSFORM", "SHELF_OBSTACLE"]` + In the below section, the possible reasons why the flags are set are discussed in detail. #### `NO_BEAM` @@ -236,4 +251,4 @@ An obstacle was detected within the shelf sweeping volume with respect to the es | Name | Description | | ----------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `numPixels` | The number of pixels in the volume of interest. Typically, this is used to verify that the rack is empty before dropping a load. | \ No newline at end of file +| `numPixels` | The number of pixels in the volume of interest. Typically, this is used to verify that the rack is empty before dropping a load. | diff --git a/PDS/GetRack/resources/rack_params_1.png b/PDS/GetRack/resources/rack_params_1.png new file mode 100644 index 0000000..ec85470 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_1.png differ diff --git a/PDS/GetRack/resources/rack_params_10.png b/PDS/GetRack/resources/rack_params_10.png new file mode 100644 index 0000000..439c653 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_10.png differ diff --git a/PDS/GetRack/resources/rack_params_2.png b/PDS/GetRack/resources/rack_params_2.png new file mode 100644 index 0000000..1825a46 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_2.png differ diff --git a/PDS/GetRack/resources/rack_params_3.png b/PDS/GetRack/resources/rack_params_3.png new file mode 100644 index 0000000..6ded7bc Binary files /dev/null and b/PDS/GetRack/resources/rack_params_3.png differ diff --git a/PDS/GetRack/resources/rack_params_4.png b/PDS/GetRack/resources/rack_params_4.png new file mode 100644 index 0000000..04a9adb Binary files /dev/null and b/PDS/GetRack/resources/rack_params_4.png differ diff --git a/PDS/GetRack/resources/rack_params_5.png b/PDS/GetRack/resources/rack_params_5.png new file mode 100644 index 0000000..b56e1b2 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_5.png differ diff --git a/PDS/GetRack/resources/rack_params_6.png b/PDS/GetRack/resources/rack_params_6.png new file mode 100644 index 0000000..277388b Binary files /dev/null and b/PDS/GetRack/resources/rack_params_6.png differ diff --git a/PDS/GetRack/resources/rack_params_7.png b/PDS/GetRack/resources/rack_params_7.png new file mode 100644 index 0000000..a9ab495 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_7.png differ diff --git a/PDS/GetRack/resources/rack_params_8.png b/PDS/GetRack/resources/rack_params_8.png new file mode 100644 index 0000000..e3a5559 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_8.png differ diff --git a/PDS/GetRack/resources/rack_params_9.png b/PDS/GetRack/resources/rack_params_9.png new file mode 100644 index 0000000..2e96429 Binary files /dev/null and b/PDS/GetRack/resources/rack_params_9.png differ diff --git a/PDS/GettingStarted/index_getting_started.md b/PDS/GettingStarted/index_getting_started.md index bcafd2c..e99b90b 100644 --- a/PDS/GettingStarted/index_getting_started.md +++ b/PDS/GettingStarted/index_getting_started.md @@ -1,22 +1,18 @@ ---- -nosearch: true ---- - # Getting started with PDS ## Prerequisites It is expected that a running O3R system (VPU and camera heads) is connected. Please refer to the [unboxing section](../../GettingStarted/Unboxing/hw_unboxing.md). -A typical procedure for getting started would be as follows -- Connect the VPU (M04311) to the O3R222 camera head, -- Connect the Ethernet cable to the VPU (M04311) and the laptop, -- Connect the VPU (M04311) to the power supply and power up the system, -- Start to the GUI - ifmVisionAssistant (iVA), +A typical procedure for getting started would be as follows: +- Connect the VPU (OVP812 or OVP813) to the O3R camera head, +- Connect the Ethernet cable to the VPU and the laptop, +- Connect the VPU to the power supply and power up the system, +- Start the ifmVisionAssistant (iVA) GUI, - Verify that live images are being received. -Before reading this section, make sure you are familiar with [how to get started with the iVA](../../GettingStarted/ifmVisionAssistant/index_iVA.md). +Before reading this section, make sure you are familiar with [how to get started with the iVA](../../SoftwareInterfaces/iVA/index_iVA.md). ## Calibrate the camera diff --git a/PDS/GettingStarted/resources/getPallet_result.png b/PDS/GettingStarted/resources/getPallet_result.png index 9b37859..8cf06f1 100644 Binary files a/PDS/GettingStarted/resources/getPallet_result.png and b/PDS/GettingStarted/resources/getPallet_result.png differ diff --git a/PDS/Integration/flag_array.md b/PDS/Integration/flag_array.md index 0c80851..89980d6 100644 --- a/PDS/Integration/flag_array.md +++ b/PDS/Integration/flag_array.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # PDS pixel flags Additionally to the result of a specific PDS command, the user can access the pixel flags. These flags provide additional details as to what each pixel is used for in the PDS algorithm. @@ -18,12 +14,4 @@ The flags are defined as below: | 6 | `GR_BEAM_EDGE` | Part of beam edge | `getRack` | | 7 | `GR_UPRIGHT_FACE` | Part of upright face | `getRack` | | 8 | `GR_UPRIGHT_EDGE` | Part of upright edge | `getRack` | -| 9 | `GR_CLEARING_VOL` | Pixel inside clearing volume | `getRack` | - -## Example -To retrieve the pixel flag in Python, follow the example below. The example uses a `getPallet` command, but the pixel flags are published for every PDS command and can be accessed in the same way. - -:::{literalinclude} ../Python/get_flags.py -:caption: get_flags.py -:language: python -::: \ No newline at end of file +| 9 | `GR_CLEARING_VOL` | Pixel inside clearing volume | `getRack` | \ No newline at end of file diff --git a/PDS/Integration/index_integration.md b/PDS/Integration/index_integration.md index fd7d6c7..aeddb31 100644 --- a/PDS/Integration/index_integration.md +++ b/PDS/Integration/index_integration.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # PDS integration :::{toctree} diff --git a/PDS/Integration/sw_integration.md b/PDS/Integration/sw_integration.md index e168178..bd5496d 100644 --- a/PDS/Integration/sw_integration.md +++ b/PDS/Integration/sw_integration.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Software integration guide PDS' commands are expected to be triggered from the vehicle's main IPC, through the ifm3d library. This document explains the infrastructure expected to configure and trigger PDS. diff --git a/PDS/Python/getPallet_adjustedParams.py b/PDS/Python/getPallet_adjustedParams.py deleted file mode 100644 index 2e894e8..0000000 --- a/PDS/Python/getPallet_adjustedParams.py +++ /dev/null @@ -1,167 +0,0 @@ -########################################### -# 2024-present ifm electronic, gmbh -# SPDX-License-Identifier: Apache-2.0 -########################################### -""" -Setup: * O3R222 with 3D on port2 - * orientation: camera horizontal (label up, FAKRA cable to the left) - * getPallet: Pallet(s) in FoV @ over 2.0 m distance -""" -import json -import time -import logging -import numpy as np -from ifm3dpy.device import O3R, Error -from ifm3dpy.framegrabber import FrameGrabber, buffer_id - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger(__name__) - -# Edit for the IP address of your OVP8xx and the camera port and extrinsic calibration -IP = "192.168.0.69" -CAMERA_PORT = "port2" -APP_PORT = "app0" -o3r = O3R(IP) - -############################################ -# Setup the application -############################################ -# Ensure a clean slate before running the example -try: - o3r.reset("/applications/instances") -except Error as e: - logger.info(f"Reset failed: {e}") -# Set the extrinsic calibration of the camera -calibration = { - "transX": 0.0, - "transY": 0.0, - "transZ": 0.69, - "rotX": 0.0, - "rotY": 1.8, - "rotZ": 1.57, -} - -logger.info(f"Setting extrinsic calibration for {CAMERA_PORT}") -o3r.set({"ports": {CAMERA_PORT: {"processing": {"extrinsicHeadToUser": calibration}}}}) - -# Creating a PDS instance and setting state to conf to be able to change protected parameters -logger.info(f"Creating a PDS instance with camera in {CAMERA_PORT}") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "CONF"} - } - } - } -) - - -# %% -############################################ -# Setup the framegrabber to receive frames -# when the application is triggered. -############################################ -fg = FrameGrabber(o3r, o3r.port(APP_PORT).pcic_port) -fg.start([buffer_id.O3R_RESULT_JSON]) - - -# Define a callback to be executed when a frame is received -def pallet_callback(frame): - """Callback to be executed for each received pallet frame. - Retrieve the data from the corresponding buffer and - deserialize it into a JSON array. - :param frame: the result of the getPallet command. - """ - if frame.has_buffer(buffer_id.O3R_RESULT_JSON): - json_chunk = frame.get_buffer(buffer_id.O3R_RESULT_JSON) - json_array = np.frombuffer(json_chunk[0], dtype=np.uint8) - json_array = json_array.tobytes() - parsed_json_array = json.loads(json_array.decode()) - logger.info(f"Detected pallet(s): {parsed_json_array['getPallet']['pallet']}") - - -fg.on_new_frame(pallet_callback) -############################################ -# Change the Projection VOI and the minimum -# pixel to validate the pallet -############################################ - -# Projection Volume shift -y_shift = 0.1 -z_shift = 0.1 - -# Configure the 'getPallet' protected parameters only in CONF state -GET_PALLET_PROTECTED_PARAMETERS = { - "orthoProjection": { - "voi": { - "yMin": -0.8 + y_shift, - "yMax": 0.8 + y_shift, - "zMin": -0.4 + z_shift, - "zMax": 0.4 + z_shift, - } - }, - "localizePallets": { - "faceMinPts": 200, #default is 300 - "stringerMinPts": 50, #default is 70 - } -} - - -# Setting the protected parameters -logger.info(f"Creating a PDS instance with camera in {CAMERA_PORT} and setting the changed protected parameters") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: { - "class": "pds", - "state": "CONF", - "ports": [CAMERA_PORT], - "configuration": { - "parameter": {"getPallet": {"0": GET_PALLET_PROTECTED_PARAMETERS}} - }, - } - } - } - } -) -# setting the PDS back to the IDLE state. -o3r.set( - { - "applications": { - "instances": { - APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "IDLE"} - } - } - } -) - -# Provide the estimated distance to the pallet and the pallet type. -GET_PALLET_PARAMETERS = { - "depthHint": 2, # We recommend providing a depth hint for faster detections - "palletIndex": 0, # Block Pallet/EPAL pallet -} -# %% -logger.info("Triggering the getPallet command") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: { - "configuration": { - "customization": { - "command": "getPallet", - "getPallet": GET_PALLET_PARAMETERS, - } - } - } - } - } - } -) -# Sleep to ensure we have time to execute the callback before exiting. -time.sleep(3) - -# %% -fg.stop() diff --git a/PDS/Python/get_flags.py b/PDS/Python/get_flags.py deleted file mode 100644 index d52a1e8..0000000 --- a/PDS/Python/get_flags.py +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env python3 -############################################# -# Copyright 2023-present ifm electronic, gmbh -# SPDX-License-Identifier: Apache-2.0 -############################################# -"""The flags provide additional information for each pixel -in the image, in a similar way as the confidence image for -the O3R camera. The flags are stored in the ifm3d buffer -O3R_RESULT_ARRAY2D. -To retrieve flags, a command must be triggered (we use -the getPallet command in this example). -""" -# %% -import logging -import time -from ifm3dpy.device import O3R, Error -from ifm3dpy.framegrabber import FrameGrabber, buffer_id - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger(__name__) - -# %% Edit for the IP address of your OVP8xx and the camera port -IP = "192.168.0.69" -CAMERA_PORT = "port2" -APP_PORT = "app0" -o3r = O3R(IP) - -# %%######################################### -# Setup the application -############################################ -# Ensure a clean slate before running the example -try: - o3r.reset("/applications/instances") -except Error as e: - logger.error(f"Reset failed: {e}") - -# Set the extrinsic calibration of the camera -calibration = { - "transX": 0.0, - "transY": 0, - "transZ": 0.0, - "rotX": -1.57, - "rotY": 1.57, - "rotZ": 0, -} -logger.info(f"Setting extrinsic calibration for {CAMERA_PORT}") -o3r.set({"ports": {CAMERA_PORT: {"processing": {"extrinsicHeadToUser": calibration}}}}) - -# Create the PDS application and -# choose the camera port -logger.info(f"Creating a PDS instance with camera in {CAMERA_PORT}") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "IDLE"} - } - } - } -) - -# %%######################################### -# Setup the framegrabber to receive frames -# when the application is triggered. -############################################ -fg = FrameGrabber(o3r, o3r.port(APP_PORT).pcic_port) -fg.start([buffer_id.O3R_RESULT_ARRAY2D]) - - -# Define a callback to be executed when a frame is received -def flags_callback(frame): - """Callback to be executed for each received pallet frame. - Retrieve the data from the corresponding buffer and - deserialize it into a JSON array. - :param frame: the result of the getPallet command. - """ - if frame.has_buffer(buffer_id.O3R_RESULT_ARRAY2D): - flags = frame.get_buffer(buffer_id.O3R_RESULT_ARRAY2D) - logger.info(f"Pixel flags: {flags}") - logger.info(f"Flag fox pixel (100, 100): {flags[100, 100]}") - - -fg.on_new_frame(flags_callback) - -# %%######################################### -# Trigger the getPallet command: we need to -# trigger a command to retrieve the corresponding -# flags for each pixel in the image. -############################################ -time.sleep(2) -GET_PALLET_PARAMETERS = { - "depthHint": -1, - "palletIndex": 0, # Block Pallet/EPAL pallet -} - -logger.info("Triggering the getPallet command") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: { - "configuration": { - "customization": { - "command": "getPallet", - "getPallet": GET_PALLET_PARAMETERS, - } - } - } - } - } - } -) -# Sleep to ensure we have time to execute the callback before exiting. -time.sleep(3) - -# %% Stop the framegrabber -fg.stop() diff --git a/PDS/Python/get_pallet.py b/PDS/Python/get_pallet.py deleted file mode 100644 index 245d471..0000000 --- a/PDS/Python/get_pallet.py +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env python3 -########################################### -###2023-present ifm electronic, gmbh -###SPDX-License-Identifier: Apache-2.0 -########################################### -""" -Setup: * Camera: O3R222, 3D on port 2 - * orientation: camera horizontally oriented (label up, Fakra cable to the left) - * Pallet: pallet in FoV @ 1.5m distance from the camera -""" -# %% -import json -import logging -import time -import numpy as np -from ifm3dpy.device import O3R, Error -from ifm3dpy.framegrabber import FrameGrabber, buffer_id - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger(__name__) - -# Device specific configuration -IP = "192.168.0.69" -CAMERA_PORT = "port2" -APP_PORT = "app0" -o3r = O3R(IP) - -############################################ -# Setup the application -############################################ -# Ensure a clean slate before running the example -try: - o3r.reset("/applications/instances") -except Error as e: - logger.info(f"Reset failed: {e}") - -# Set the extrinsic calibration of the camera -calibration = { - "transX": 0.0, - "transY": 0.0, - "transZ": 0.2, - "rotX": 0.0, - "rotY": 1.57, - "rotZ": -1.57, -} -logger.info(f"Setting extrinsic calibration for {CAMERA_PORT}") -o3r.set({"ports": {CAMERA_PORT: {"processing": {"extrinsicHeadToUser": calibration}}}}) - -logger.info(f"Creating a PDS instance with camera in {CAMERA_PORT}") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "IDLE"} - } - } - } -) - -# %% -############################################ -# Setup the framegrabber to receive frames -# when the application is triggered. -############################################ -fg = FrameGrabber(o3r, o3r.port(APP_PORT).pcic_port) -fg.start([buffer_id.O3R_RESULT_JSON]) - - -# Define a callback to be executed when a frame is received -def pallet_callback(frame): - """Callback to be executed for each received pallet frame. - Retrieve the data from the corresponding buffer and - deserialize it into a JSON array. - :param frame: the result of the getPallet command. - """ - if frame.has_buffer(buffer_id.O3R_RESULT_JSON): - json_chunk = frame.get_buffer(buffer_id.O3R_RESULT_JSON) - json_array = np.frombuffer(json_chunk[0], dtype=np.uint8) - json_array = json_array.tobytes() - parsed_json_array = json.loads(json_array.decode()) - logger.info(f"Detected pallet(s): {parsed_json_array['getPallet']['pallet']}") - - -fg.on_new_frame(pallet_callback) - -############################################ -# Trigger the getPallet command -############################################ - -# Provide the estimated distance to the pallet and the pallet type. -GET_PALLET_PARAMETERS = { - "depthHint": 1.2, # We recommend providing a depth hint for faster detections - "palletIndex": 0, # Block Pallet/EPAL pallet -} - -# %% -logger.info("Triggering the getPallet command") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: { - "configuration": { - "customization": { - "command": "getPallet", - "getPallet": GET_PALLET_PARAMETERS, - } - } - } - } - } - } -) -# Sleep to ensure we have time to execute the callback before exiting. -time.sleep(3) -# %% -fg.stop() diff --git a/PDS/Python/get_rack.py b/PDS/Python/get_rack.py deleted file mode 100644 index 2520486..0000000 --- a/PDS/Python/get_rack.py +++ /dev/null @@ -1,119 +0,0 @@ -#!/usr/bin/env python3 -########################################### -###2023-present ifm electronic, gmbh -###SPDX-License-Identifier: Apache-2.0 -########################################### - -""" -Setup: * Camera: O3R222, 3D on port2 - * orientation: camera horizontally (FAKRA cable to the left) - * Rack: rack in FoV @ 1.5m distance -""" -import json -import logging -import time -import numpy as np - -from ifm3dpy.device import O3R, Error -from ifm3dpy.framegrabber import FrameGrabber, buffer_id - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger(__name__) - -# Device specific configuration -IP = "192.168.0.69" -CAMERA_PORT = "port2" -APP_PORT = "app0" -o3r = O3R(IP) - -############################################ -# Setup the application -############################################ -try: - o3r.reset("/applications/instances") -except Error as e: - logger.info(f"Reset failed: {e}") - -# Set the correct extrinsic calibration of the camera. -calibration = { - "transX": 0.0, - "transY": 0.0, - "transZ": 0.2, - "rotX": 0.0, - "rotY": 1.57, - "rotZ": -1.57, -} -logger.info(f"Setting the extrinsic calibration for camera in {CAMERA_PORT}") -o3r.set({"ports": {CAMERA_PORT: {"processing": {"extrinsicHeadToUser": calibration}}}}) - -# Create the application instance and set to IDLE (ready to be triggered) -logger.info(f"Create a PDS instance using camera in {CAMERA_PORT}") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "IDLE"} - } - } - } -) - -############################################ -# Setup the framegrabber to receive frames -# when the application is triggered. -############################################ -fg = FrameGrabber(o3r, o3r.port(APP_PORT).pcic_port) -fg.start([buffer_id.O3R_RESULT_JSON]) - - -# Define a callback function to be executed every time a frame is received -def rack_callback(frame): - """Callback function executed every time a frame is received. - The data is decoded and the result printed. - - :param frame: A frame containing the data for all the buffer ids - requested in the start function of the framegrabber. - """ - if frame.has_buffer(buffer_id.O3R_RESULT_JSON): - json_chunk = frame.get_buffer(buffer_id.O3R_RESULT_JSON) - json_array = np.frombuffer(json_chunk[0], dtype=np.uint8) - json_array = json_array.tobytes() - parsed_json_array = json.loads(json_array.decode()) - logger.info(f"Detected rack: {parsed_json_array['getRack']}") - - -fg.on_new_frame(rack_callback) - -############################################ -# Trigger the getRack command -############################################ -time.sleep(2) # Grace period after the framegrabber starts - -GET_RACK_PARAMETERS = { - "depthHint": 1.2, # Estimated position of the rack (-1 for automatic detection) - "horizontalDropPosition": "left", - "verticalDropPosition": "interior", - "zHint": -0.4, -} - -logger.info("Triggering the getRack command") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: { - "configuration": { - "customization": { - "command": "getRack", - "getRack": GET_RACK_PARAMETERS, - } - } - } - } - } - } -) - -# Allow time for the callback to execute before exiting -time.sleep(3) -fg.stop() diff --git a/PDS/Python/requirements.txt b/PDS/Python/requirements.txt deleted file mode 100644 index d54e53a..0000000 --- a/PDS/Python/requirements.txt +++ /dev/null @@ -1 +0,0 @@ -ifm3dpy>=1.4.3 \ No newline at end of file diff --git a/PDS/Python/vol_check.py b/PDS/Python/vol_check.py deleted file mode 100644 index 895e851..0000000 --- a/PDS/Python/vol_check.py +++ /dev/null @@ -1,118 +0,0 @@ -#!/usr/bin/env python3 -########################################### -###2023-present ifm electronic, gmbh -###SPDX-License-Identifier: Apache-2.0 -########################################### -""" -Setup: * Camera: O3R222, 3D on port2 - * orientation: camera horizontally (Fakra cable to the left, label up) -""" -import json -import logging -import time -import numpy as np - -from ifm3dpy.device import O3R, Error -from ifm3dpy.framegrabber import FrameGrabber, buffer_id - -logging.basicConfig(level=logging.DEBUG) -logger = logging.getLogger(__name__) - -# Device specific configuration -IP = "192.168.0.69" -CAMERA_PORT = "port2" -APP_PORT = "app0" -o3r = O3R(IP) - -############################################ -# Setup the application -############################################ -try: - o3r.reset("/applications/instances") -except Error as e: - logger.info(f"Reset failed: {e}") - -calibration = { - "transX": 0.0, - "transY": 0.0, - "transZ": 0.2, # 20 cm above the ground - "rotX": 0, - "rotY": 1.57, # rotY and rotZ define camera horizontal and looking straight forward - "rotZ": -1.57, -} -logger.info(f"Setting the calibration for camera in {CAMERA_PORT}") -o3r.set({"ports": {CAMERA_PORT: {"processing": {"extrinsicHeadToUser": calibration}}}}) - -logger.info(f"Create a PDS instance using the camera in {CAMERA_PORT}") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "IDLE"} - } - } - } -) - -time.sleep(0.5) -############################################ -# Setup the framegrabber to receive frames -# when the application is triggered. -############################################ -fg = FrameGrabber(o3r, o3r.port(APP_PORT).pcic_port) -fg.start([buffer_id.O3R_RESULT_JSON]) - - -def volume_callback(frame): - """Callback method called every time a frame is received. - Deserialize the data from the result of the volCheck command. - - :param frame: frame containing the results of the volCheck command - """ - if frame.has_buffer(buffer_id.O3R_RESULT_JSON): - json_chunk = frame.get_buffer(buffer_id.O3R_RESULT_JSON) - json_array = np.frombuffer(json_chunk[0], dtype=np.uint8) - json_array = json_array.tobytes() - parsed_json_array = json.loads(json_array.decode()) - logger.info( - f"Number of pixels in the volume: {parsed_json_array['volCheck']['numPixels']}" - ) - - -fg.on_new_frame(volume_callback) - -############################################ -# Trigger the volCheck command -############################################ -time.sleep(2) # Grace period after the framegrabber starts - -# Define the boudaries of the volume to be checked -VOLCHECK_PARAMETERS = { - "xMin": 1, - "xMax": 2, - "yMin": -0.5, - "yMax": 0.5, - "zMin": 0.0, - "zMax": 0.4, -} - -logger.info("Triggering the volCheck command") -o3r.set( - { - "applications": { - "instances": { - APP_PORT: { - "configuration": { - "customization": { - "command": "volCheck", - "volCheck": VOLCHECK_PARAMETERS, - } - } - } - } - } - } -) - -time.sleep(3) -fg.stop() diff --git a/PDS/Recording/recordings_iVA.md b/PDS/Recording/recordings_iVA.md index bdb1450..b15bd9b 100644 --- a/PDS/Recording/recordings_iVA.md +++ b/PDS/Recording/recordings_iVA.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Data recording with ifmVisionAssistant Throughout the stages of integration, development, and testing of the PDS solution, there might be instances where issues arise. In such cases, the first course of action should be to record the relevant data and document the issue for further examination. diff --git a/PDS/Results/results.md b/PDS/Results/results.md new file mode 100644 index 0000000..642a2b7 --- /dev/null +++ b/PDS/Results/results.md @@ -0,0 +1,22 @@ +# Results + +The result of PDS is formatted in JSON. +It contains multiple parts: +- a common part, which provides information such as the timestamps and the last command used; +- a command part, which contains the result of the detection (if any). + +## Common result + +The result of the PDS commands is provided in JSON format. +The following output is always present in the result of a PDS command, whether or not the detection was successful: + +| Property | Type | Description | +| :--------------------- | :----------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `lastCommand` | string | The name of the last command used. | +| `lastCommandInput` | JSON | The values used for the parameters of the last command. Not all the parameters will be displayed there, but only the ones in the `configuration/customization` section. | +| `pdsVersion` | JSON | The current version of the PDS algorithm. | +| `timeStamp` | integer | The timestamps at which the frame was captured. | + +## Command results + +The details of the output of each command can be found in their respective documentation: [result of `getPallet`](../GetPallet/getPallet.md#results), [result of `getRack`](../GetRack/getRack.md#results), [result of `volCheck`](../VolCheck/volCheck.md#results). \ No newline at end of file diff --git a/PDS/Troubleshooting/troubleshooting.md b/PDS/Troubleshooting/troubleshooting.md index 2f47d9f..bc92751 100644 --- a/PDS/Troubleshooting/troubleshooting.md +++ b/PDS/Troubleshooting/troubleshooting.md @@ -1,7 +1,3 @@ ---- -nosearch: true ---- - # Troubleshooting This document outlines some ideas of strategies to apply when encountering issues with PDS. diff --git a/PDS/VisionAssistant/PDS_iVA_release_notes_2.7.13.md b/PDS/VisionAssistant/PDS_iVA_release_notes_2.7.13.md deleted file mode 100644 index 34aa813..0000000 --- a/PDS/VisionAssistant/PDS_iVA_release_notes_2.7.13.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -nosearch: true ---- - -# Vision Assistant 2.7.13 -The following release note provides an overview of the features of the iVA **2.7.13** version. This version is available for field testing the Pose Detection Solution (PDS), and is not available as a general release. For release information of publicly release versions, please refer to [the release notes in ifm3d.com](https://ifm3d.com/latest/SoftwareInterfaces/iVA/index_iVA.html). - -The Vision Assistant is available for Windows and Linux (Ubuntu 20.04 and Ubuntu 22.04). - -## Previous Releases -Previous iVA release is version 2.7.6. All the updates to the Vision Assistant that occurred between 2.7.6 and 2.7.13 are listed below. - -## Compatible Video Processing Platforms (VPUs) -This version of the Vision Assistant is intended to be used with PDS devices M04311. - -## New features -- New application: PDS (Pose Detection System) -- PDS additional 2D overlay visualization. -- New PDS 3D visualizations for the volume of interest, namely `depthEstimationVoi` in green and `projectionVoi` in cyan. -- PDS visualization in the "Monitor" view. diff --git a/PDS/VolCheck/volCheck.md b/PDS/VolCheck/volCheck.md index 8574938..d57a481 100644 --- a/PDS/VolCheck/volCheck.md +++ b/PDS/VolCheck/volCheck.md @@ -1,15 +1,11 @@ ---- -nosearch: true ---- - # `volCheck` The `volCheck` (short for volume check) functionality of PDS offers an easy-to-use possibility to test whether a 3D volume (Volume Of Interest - VOI) is free of obstacles. The number of pixels present in the defined VOI is provided, and the user can decide how many pixels are considered an obstacle. This command is useful to check whether the location is occupied or not before placing the load, or make sanity check for example to verify that the forks are visible where expected. -## Input +## Configuration -### Command +### Command configuration The input to the `volCheck` command should be provided in JSON. Below is an example, assuming an instantiated PDS app `app0`: ```json @@ -36,30 +32,36 @@ Below is an example, assuming an instantiated PDS app `app0`: } ``` -### `volCheck` -| Name | Description | -| ---------- | --------------------------------------------------------------------------------------------------- | -| `volCheck` | The default bounding box parameters for `volCheck`: `xMin`, `xMax`, `yMin`, `yMax`, `zMin`, `zMax`. | + +| Property | Type | Description | Default | Minimum | Maximum | Enumeration | +| :-------------- | :----- | :--------------------------------------------------- | :------ | :------ | :------ | :---------- | +| `volCheck.xMax` | number | Bounding box dimension of VOI along X-axis - Maximum | 2.5 | N/A | N/A | N/A | +| `volCheck.xMin` | number | Bounding box dimension of VOI along X-axis - Minimum | 1 | N/A | N/A | N/A | +| `volCheck.yMax` | number | Bounding box dimension of VOI along Y-axis - Maximum | 0.4 | N/A | N/A | N/A | +| `volCheck.yMin` | number | Bounding box dimension of VOI along Y-axis - Minimum | -0.4 | N/A | N/A | N/A | +| `volCheck.zMax` | number | Bounding box dimension of VOI along Z-axis - Maximum | 0.5 | N/A | N/A | N/A | +| `volCheck.zMin` | number | Bounding box dimension of VOI along Z-axis - Minimum | 0.1 | N/A | N/A | N/A | + +### Fine-tuning parameters + +| Property | Type | Description | Default | Max. | Min. | +| :------------------------ | :----- | :--------------------------- | :------ | :--- | :--- | +| volCheck/nearestXQuantile | number | Quantile [0..1] for nearestX | 0.05 | 0 | 1 | -## Output +## Results The output of a `volCheck` command is formatted in JSON. An example JSON result, where 2087 pixels were detected within the specified volume, is shown below: ```json "volCheck": { - "numPixels": 2087 + "nearestX": 1.3, + "numPixels": 2087, } ``` -### `numPixels` + | Name | Description | | ----------- | ----------------------------------------------------------- | +| `nearestX` | Distance in meters to the closest pixel in the volume. | | `numPixels` | Number of valid pixels inside the given volume of interest. | - -## Example - -:::{literalinclude} ../Python/vol_check.py -:caption: vol_check.py -:language: python -::: diff --git a/PDS/index_pds.md b/PDS/index_pds.md index 6e776a6..50a7c38 100644 --- a/PDS/index_pds.md +++ b/PDS/index_pds.md @@ -1,28 +1,29 @@ ---- -nosearch: true ---- +# PDS (Pick and Drop System) -# PDS (Pose Detection System) - -PDS - `Pose Detection System` - provided by [ifm](https://www.ifm.com), is a software solution building on top of the O3R ecosystem to enable AGVs (Automated Guided Vehicles), fork trucks and other robots to detect the pose of objects within a 3D environment. +PDS - `Pick and Drop System` - provided by [ifm](https://www.ifm.com), is a software solution built on top of the O3R ecosystem to enable AGVs (Automated Guided Vehicles), forklifts, and other robots to detect the pose of objects within a 3D environment for picking them up and detecting racks for dropping them. :::{toctree} :maxdepth: 2 Getting started Calibration Configuration -`getPallet` -`getRack` -`getItem` -`volCheck` +Results +Pallets +Racks +Volume check +Code examples Recording Integration -Vision Assistant release notes ::: -## Features + + +:::{raw} HTML +

Features

+::: -PDS uses the O3R platform as its primary data source: one 3D camera stream can be used at one time. The current firmware version for PDS application is tested only with `O3R222` camera head. +PDS uses the O3R platform as its primary data source: one 3D camera stream can be used at one time. It is recommended to use the `O3R222` camera head. The support for O3R225 wide FoV heads is experimental at the moment. The support for (H)VGA camera heads will be added in the future. @@ -33,10 +34,3 @@ PDS provides four different commands: | `getRack` | Pose of an industrial rack | | `getItem` | Pose of up to 10 custom items (experimental feature) | | `volCheck` | Quantifies the number of valid pixels within a defined volume of interest (VOI) | - - -## Compatibility Matrix - -| Firmware Version | Supported VPU Hardware | Supported Camera Hardware | ifm3d-library | ifmVisionAssistant | Comments | -| ---------------- | ---------------------- | ------------------------- | ------------- | ------------------ | --------------------------------------------------- | -| 1.2.x | `M04311` | O3R222 | >=1.4.3 | >=2.7.2 | Field test only version. Do not use for production. | \ No newline at end of file diff --git a/PLC/driving_scenarios.md b/PLC/driving_scenarios.md new file mode 100644 index 0000000..5c7db96 --- /dev/null +++ b/PLC/driving_scenarios.md @@ -0,0 +1,84 @@ +--- +nosearch: true +--- + + +# Presets configuration based on driving scenarios + +## Forktrucks + +Automated forktrucks/forklifts require different zone configurations to operate safely in the various environments. These zone configurations depend on factors like min/max speed, load and driving scenarios. The following presets configuration serves as an example for a forktruck using two camera facing forward/main driving direcion [`PORT0`, `PORT1`] and another camera[`PORT2`] mounted rear end of vehicle. + +Assuming the length of forktruck as 1.2m and 1m width. + +- Preset 0: When the forktruck is driving at highspeeds +![high_speed_forktruck](plc_resources/high_speed.png) + +- Preset 1: When the forktruck is driving at low speeds +![slow_speed_forktruck](plc_resources/slow_speeds.png) + +- Preset 2: When forktruck is driving in reverse direction +![drive_reverse](plc_resources/drive_reverse.png) + +```json +{ + "applications": { + "instances": { + "app0": { + "class": "ods", + "presets": { + "definitions":{ + "0": { + "description": "Driving Forward - High speeds", + "preset": { + "activePorts": ["port0", "port1"], + "zones": { + "zoneConfigID": 0, + "zoneCoordinates": [ + [[0.6, 0.5], [0.6, -0.5], [2.6, -0.5], [2.6, 0.5]], + [[2.6, 0.5], [2.6, -0.5], [3.6, -0.5], [3.6, 0.5]], + [[3.6, 0.5], [3.6, -0.5], [4, -0.5], [4, 0.5]] + ] + } + } + }, + "1": { + "description": "Driving Forward - Slow speeds", + "preset": { + "activePorts": ["port0","port1"], + "zones": { + "zoneConfigID": 1, + "zoneCoordinates": [ + [[0.6, 0.5], [0.6, -0.5], [1.6, 0.5], [1.6, -0.5]], // Danger/Emergency Stop Zone + [[1.6, 0.5], [1.6, -0.5], [2.6, 0.5], [2.6, -0.5]], // Warning Zone + [[2.6, 0.5], [2.6, -0.5], [4, 0.5], [4, -0.5]] // Notify Zone (Slow Down) + ] + } + } + }, + "2": { + "description": "Driving Backward", + "preset": { + "activePorts": [ + "port2" + ], + "zones": { + "zoneConfigID": 2, + "zoneCoordinates": [ + [[-0.6, 0.5], [-0.6, -0.5], [-1.6, 0.5], [-1.6, -0.5]], // Danger/Emergency Stop Zone + [[-1.6, 0.5], [-1.6, -0.5], [-2.6, 0.5], [-2.6, -0.5]], // Warning Zone + [[-2.6, 0.5], [-2.6, -0.5], [-4, 0.5], [-4, -0.5]] // Notify Zone (Slow Down) + ] + } + } + }, + "3": { + "description": "Power saving / charging mode", + "preset": { + "activePorts": [], + "zones": { + "zoneConfigID": 3 + } + } + }}}}}}} +``` \ No newline at end of file diff --git a/PLC/function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_DE.pdf b/PLC/function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_DE.pdf new file mode 100644 index 0000000..aab728f Binary files /dev/null and b/PLC/function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_DE.pdf differ diff --git a/PLC/function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_EN.pdf b/PLC/function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_EN.pdf new file mode 100644 index 0000000..ca211c3 Binary files /dev/null and b/PLC/function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_EN.pdf differ diff --git a/PLC/function_block/index_fb.md b/PLC/function_block/index_fb.md new file mode 100644 index 0000000..83b7e4d --- /dev/null +++ b/PLC/function_block/index_fb.md @@ -0,0 +1,7 @@ +--- +nosearch: true +--- + +# Function Block + +Please find the detailed documentation of function block and it's functions {download}` here ` \ No newline at end of file diff --git a/PLC/index_plc.md b/PLC/index_plc.md new file mode 100644 index 0000000..9aae003 --- /dev/null +++ b/PLC/index_plc.md @@ -0,0 +1,25 @@ +--- +nosearch: true +--- + +# PLC application + +The PLC application developed by ifm is an embedded additional application included in firmware version `>= 1.5.11`. +The purpose of the PLC application is to simplify the data exchange between and configuration between an ifm embedded application such as ODS on the VPU and a programmable logic controller (PLC). + +The PLC application allows for efficient and fast data transfer of application output and ensures seamless operation within the O3R ecosystem. The PLC application functions as a server with PLC acting as a client. + +The system of embedded solution app (ODS/PDS) and PLC app is designed a multi part system. +The configuration, meaning presets of the ODS application are configured in the respective solution app itself. In addition the PLC app acts as a intermediate communication partner for "reshaping" the output data to be PLC compatible and providing PLC friendly configuration interfaces. + +The user can configure different zones / zone-sets and the active cameras based for different driving scenarios via their respective preset. +Please take a closer look at the figures below for a better understanding of the interoperability between the ifm PLC application (and solution application) with a PLC. + +![PLC app -> PLC](plc_resources/plc_app-plc_image.drawio.svg) + +:::{toctree} + :maxdepth: 2 +Setup guide +TCP/IP communication interface +Function block +::: \ No newline at end of file diff --git a/PLC/plc_resources/FB_layout.jpg b/PLC/plc_resources/FB_layout.jpg new file mode 100644 index 0000000..444346e Binary files /dev/null and b/PLC/plc_resources/FB_layout.jpg differ diff --git a/PLC/plc_resources/drive_reverse.png b/PLC/plc_resources/drive_reverse.png new file mode 100644 index 0000000..35028d2 Binary files /dev/null and b/PLC/plc_resources/drive_reverse.png differ diff --git a/PLC/plc_resources/high_speed.png b/PLC/plc_resources/high_speed.png new file mode 100644 index 0000000..110b0c5 Binary files /dev/null and b/PLC/plc_resources/high_speed.png differ diff --git a/PLC/plc_resources/plc_app-plc_image.drawio.svg b/PLC/plc_resources/plc_app-plc_image.drawio.svg new file mode 100644 index 0000000..39bb1b7 --- /dev/null +++ b/PLC/plc_resources/plc_app-plc_image.drawio.svg @@ -0,0 +1,290 @@ + + + + + + + + + + + + +
+
+
+ + PLC +
+ +
+
+
+ + + PLC + + + + + + + + + +
+
+
+ + + ODSInfo + + +
+
+
+ + + ODSInfo + + + + + + + + + +
+
+
+ + ODS APP + + (app<x>) + + +
+
+
+ + + ODS APP (app<x>) + + + + + + +
+
+
+ + + [ODS parameters] + + +
+
+
+ + + [ODS parameters] + + + + + + +
+
+
+
+ + VPU + +
+
+
+
+ + + VPU + + + + + + + + + +
+
+
+ + PLC APP + + (app<y>) + + +
+
+
+ + + PLC APP (app<y>) + + + + + + +
+
+
+ + + + ODSApp: app<x> + + + + + + + +
+
+
+ + + ODSApp: app<x> + + + + + + + + +
+
+
+ + + SET Preset Id: <i> + + +
+
+
+ + + SET Preset Id: <i> + + + + + + + + + +
+
+
+ + + ETH0 + + +
+
+
+ + + ETH0 + + + + + + + + + +
+
+
+ + + ETH + + +
+
+
+ + + ETH + + + + + + + + + + + +
+
+
+ + + f-command + + +
+
+
+ + + f-command + + + + + + + + +
+
+
+ + + PLC Application Output + + +
+
+
+ + + PLC Application Output + + + + + + + + + Text is not SVG - cannot display + + + + \ No newline at end of file diff --git a/PLC/plc_resources/slow_speeds.png b/PLC/plc_resources/slow_speeds.png new file mode 100644 index 0000000..920c30b Binary files /dev/null and b/PLC/plc_resources/slow_speeds.png differ diff --git a/PLC/plc_resources/state_machine.drawio.svg b/PLC/plc_resources/state_machine.drawio.svg new file mode 100644 index 0000000..171502c --- /dev/null +++ b/PLC/plc_resources/state_machine.drawio.svg @@ -0,0 +1,412 @@ + + + + + + + + + + + +
+
+
+ + Configured PLC Application + +
+
+
+ + + Configured PLC Application + + + + + + + + + +
+
+
+ ERROR RAISED +
+
+
+ + + ERROR RAI... + + + + + + + + + +
+
+
+ + Create ODS Application + +
+
+
+ + + Create ODS Application + + + + + + + + +
+
+
+ + + ODS Application Created + + +
+
+
+ + + ODS Application Created + + + + + + + +
+
+
+ + + PLC Application Created + + +
+
+
+ + + PLC Application Created + + + + + + + + +
+
+
+ + Create PLC Application + +
+
+
+ + + Create PLC Application + + + + + + + +
+
+
+ + Configured ODS Application + +
+
+
+ + + Configured ODS Application + + + + + + + + + +
+
+
+ ERROR RAISED +
+
+
+ + + ERROR RAI... + + + + + + + + +
+
+
+ + Configure ODS Application + +
+
+
+ + + Configure ODS Application + + + + + + + Applications in + + + RUN state + + + + + + + +
+
+
+ + Change ODS Application -> RUN + +
+
+
+ + + Change ODS Application -> RUN + + + + + + + + + +
+
+
+ ERROR RAISED +
+
+
+ + + ERROR RAI... + + + + + + + + +
+
+
+ SET Preset +
+ index <i> +
+
+
+ + + SET Preset... + + + + + + + + +
+
+
+ + Configure PLC Application + +
+
+
+ + + Configure PLC Application + + + + + + + + +
+
+
+

+ + + PLC + + +

+
+
+
+ + + PLC + + + + + + + + + + + +
+
+
+ + IP ADDRESS + +
+
+
+ + + IP ADDRESS + + + + + + + +
+
+
+ PLC APP PORT NO +
+
+
+ + + PLC APP PORT NO + + + + + + + +
+
+
+ PRESET IDX +
+
+
+ + + PRESET IDX + + + + + + + + + + + + +
+
+
+ ERROR RAISED +
+
+
+ + + ERROR RAISED + + + + + + + +
+
+
+
    +
  • + + Review the diagnostics reaction strategy documentation to resolve the errors + +
    • +
+
+
+
+
+ + + Review the diagnostics reaction strategy d... + + + + + + + + + Text is not SVG - cannot display + + + + \ No newline at end of file diff --git a/PLC/setup_guide.md b/PLC/setup_guide.md new file mode 100644 index 0000000..08ae321 --- /dev/null +++ b/PLC/setup_guide.md @@ -0,0 +1,143 @@ +--- +nosearch: true +--- + +# ODS PLC App setup Guide + +## O3R Hardware and Software requirements + +### Hardware requirements + +| Article | Description | +| ----------------- | ----------------------------------------------------------------- | +| OVP811 | Latest VPU generation with pre-installed ODS license | +| O3R222 (AB/AC/AD) |
  • 3D: 38k 224x172, 60°x45°
  • 2D: 1280x800, 127°x80°
    • | +| O3R225 (AB/AC/AD) |
  • 3D: 38k 224x172, 105°x78°
  • 2D: 1280x800, 127°x80°
    • | + +Use 1 Gigabit/s rated hardware: cables, switches only. + +### Software requirements + +| Software | Version | +| -------- | -------------------- | +| Firmware | 1.5.14 | + + + +## PLC Hardware and Software requirements + +### PLC Hardware requirements + +The current function block is developed only for the following PLC's. +| Article | Description | +| -------------- | --------------------------------- | +| SIEMENS S71500 | S7CPU series (cycle time < 20 ms) | + +### PLC Software requirements + +| Software | Version | +| ------------------ | ------- | +| SIEMENS TIA Portal | >= V16 | + +## State Machine overview + +A state machine serves as an abstraction to accomplish the successful interface between ODS, PLC Applications and PLC. +![State Machine](plc_resources/state_machine.drawio.svg) + +:::{note} + If there are any diagnostics raised during the configuration / operation state, please review the [diagnostic reaction strategy documentation](../SoftwareInterfaces/ifmDiagnostic/diagnostic_reaction_strategy.md) to resolve the issues. +::: + +## Prerequisites + +Extrinsic calibration of the cameras and VPU is a prerequisite for setting up an ODS application. Please refer to this [document](https://ifm3d.com/latest/CalibrationRoutines/index_calibrations.html) for extrinsic calibration routines. + +## Setup procedure + +To setup PLC based AGV with O3R ODS (collision avoidance), follow these steps. + +- [ODS PLC App setup Guide](#ods-plc-app-setup-guide) + - [O3R Hardware and Software requirements](#o3r-hardware-and-software-requirements) + - [Hardware requirements](#hardware-requirements) + - [Software requirements](#software-requirements) + - [PLC Hardware and Software requirements](#plc-hardware-and-software-requirements) + - [PLC Hardware requirements](#plc-hardware-requirements) + - [PLC Software requirements](#plc-software-requirements) + - [State Machine overview](#state-machine-overview) + - [Prerequisites](#prerequisites) + - [Setup procedure](#setup-procedure) + - [ODS application setup: creation and configuration](#ods-application-setup-creation-and-configuration) + - [PLC application setup: creation and configuration](#plc-application-setup-creation-and-configuration) + - [Configuration parameters](#configuration-parameters) + - [PLC hardware setup: Network Topology](#plc-hardware-setup-network-topology) + - [SIEMENS Function Block](#siemens-function-block) + +### ODS application setup: creation and configuration + +To create and configure the ODS application instance please refer to the [ODS section](https://ifm3d.com/latest/ODS/index_ods.html). + +:::{note} + +- The PLC function block is only able to configure between already existing presets of the ODS application (via PLC application). Meaning, when a preset index load is requested from the PLC to the PLC application, the PLC application will internally load the given ODS preset. If preset index is not existing, then the system will throw a asynchronous diagnostic: invalid configuration. +- Therefore, the presets of ODS application have to be configured prior to creating the PLC application. +::: + +To configure the presets of an ODS application, please refer to the [presets documentation](../ODS/Presets/presets.md). + +Example preset configurations for different driving scenarios can be found in [driving scenarios document](driving_scenarios.md) + +### PLC application setup: creation and configuration + +In ifmVisionAssistant, create the PLC application from the applications in the `Application` window by clicking on the ![add_app](../CalibrationRoutines/MCC/_resources/add_app.png) icon. + +#### Configuration parameters + +| **Parameter** | **Description** | +| ---------------------- | --------------------------------------------------------------------- | +| `name` | To configure the custom name for the application | +| `state` | The current application state. Per default, the state is set to `RUN` | +| `configuration/odsApp` | Identifier of the ODS app instance. (e.g. `app0`) | + + + +The PLC application is per default in `RUN` state. Therefore the ODS application shall be saved to be in `RUN` state at every bootup session. + +:::{note} + The PLC embedded application sends data over TCP/IP without waiting for acknowledgement. When an additional network load introduced by the user may result in network saturations i.e. communication delays. +::: + +### PLC hardware setup: Network Topology + +To estabilish a successful communication between PLC and VPU, both devices must be in same subnet range. Please refer to the [Ethernet interfaces](https://ifm3d.com/latest/Technology/VPU/ethernet.html) documentation for setting up the static IP address on VPU. + +:::{note} + - Please use only `ETH0` as a main communication interface between VPU and PLC. + - `ETH1` can be used for the debugging purposes via GUI / API. + - Ethernet ring topologies are not supported: `ETH0` and `ETH1` must not be in same subnet range. +::: + +### SIEMENS Function Block + +The SIEMENS function block provided by ifm is intended to provide a interface between PLC and PLC application on VPU. There are four functions included in the function block + +- Establishing a TCP connection to the PLC server of the VPU +- Monitoring the connection to the PLC server +- Receiving the ODS result data and displays at the outputs +- Sending the zone sets to be activated + +![fb_layout](plc_resources/FB_layout.jpg) + +**Activating a preset in the ODS** +The presets are stored in the ODS and labelled with an identifier. When the user inputs the identifier of the preset at the PresetIdx input parameter the Function Block will + + - Sends the command to activate the specified zone set when the value of the PresetId input parameter is changed + - Sends the command to activate the specified zone set each time the connection to the PLC server of the VPU is established + - Repeats the command periodically if it was previously rejected by the PLC server + - Shows in the ActivePresetSuccess output = TRUE if the command was accepted by the PLC server of the VPU + - Shows in the ActivePresetFailed output = TRUE if the command was rejected by the PLC server of the VPU or a communication error occurred during transmission + +:::{note} + A positive acknowledgement of the command does not mean that the zone set is already active in the ODS. The configuration used to determine the zone assignment is always displayed in the ODS.ZoneConfigID output. +::: + +For more details of Function Block implementation and configuration see the {download}`Function Block PDF <./function_block/ifm-OVP81x-ODS-V1-1_S7-1500_TCP_EN.pdf>`. \ No newline at end of file diff --git a/PLC/tcp_ip_interface.md b/PLC/tcp_ip_interface.md new file mode 100644 index 0000000..ca430b6 --- /dev/null +++ b/PLC/tcp_ip_interface.md @@ -0,0 +1,159 @@ +--- +nosearch: true +--- + +# PLC TCP/IP communication interface + +## PLC Application -> PLC + +The PLC application sends the data via TCP Socket(510xx) at the rate of 20 Hz(~ to ODS framerate). If there is no data received from ODS application then PLC application sends the cache data i.e. previous ODS result data but increments the `Result Age Indicator` by **1** for every 50 milliseconds. + +The PLC application is mapped to a TCP/IP communication port (referred also as PCIC Port) and this is assigned based on application instance indicator. + +| Application Instance | TCP/IP Port | +| -------------------- | ----------- | +| `app0` | 51010 | +| `app1` | 51011 | +| `app2` | 51012 | +| `app` | 5101 | + +The PLC application output can be deserialized based on the following schema. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    FieldSize (Bytes)TypeDescription
    Protocol Version2uint8 version[2] +
      +
    • 1 byte major version (not compatible across versions)
      • +
    • 1 byte minor version (compatible within one major version)
      • +
    +
    Size2uint16The size of this frame in bytes (fixed value of 188 for version 1.1)
    ODS Result Data16uint8[16] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    FieldSize (Bytes)TypeDescription
    Result Age Indicator1uint8 +
      +

      Indicates whether ODS data was received from the ODS application (0 if received, otherwise incremented)

      +
    • If no new data is present for the next PLC-App frame the value is increased till 255 and saturates. This is implemented to inform user if there is any broken communication or error in ODS cameras.
      • +
    • The value will stay `255` until it receives data from ODS application.
      • +
    +
    Zone status flags3uint8 + Zone status flags (3 bytes, 0: zone free, 1: zone occupied) +
    ZoneConfigID4uint8 + 32-bit integer representing the Zone Configuration ID +
    Time Stamp8uint64 + Time stamp of ODS algorithm result. VPU time (including NTP if configured) +
    +
    PDS46uint8[46]- TBD -
    Diagnostic Counter2uint8[2]1 byte the current diagnostic slice counting from zero (max. 0 for v1.1) 1 byte the total amount of diagnostics slices (max. 1 for v1.1)
    Diagnostic Data120DiagData data[20]The rolling diagnostic information, in case there is no diagnostic information the Diagnostic slice counter is set to zero and the diagnostic Data is filled with zeros. There is an array of 20 diagnostic data structs available. +

    The Diagnostic data (DiagData) is a struct of:

    +
      +
    • Source: uint8 +
        +
      • 0,...,6: port0, .... port6
        • +
      • 100,...119: app0, .... app19
        • +
      • 255: other
        • +
      +
      • +
    • Active: uint8 contains 1 in case of an active diagnostic
      • +
    • Diagnostic ID: uint32
      • +
    +
    + +:::{note} + The PLC application will send the data at 20Hz (~ODS framerate). If the ODS application is set to `IDLE` state then PLC application will send the cached ODS data i.e. the previously received data. +::: + +## PLC -> PLC Application + +To configure the presets of ODS application, a PCIC command has to be sent to the PLC application on VPU and in ifm O3R ecosystem this command is referred as `f-command`. + +The `f-command` includes the following information: +- `Parameter-ID` : A fixed 5 decimal ASCII padded with `0`. The ID to set presets of ODS application is **`02101`** +- `reserved` : fixed value **`#00000`** +- `Data` : The data must be constructed in the following format. + +| Field | Size(Bytes) | Type | Description | +| ----------------- | ----------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| Version | 2 | uint8 version[2] |
  • 1 byte major version (not compatible across versions).
  • 1 byte minor version (compatible within one major version)
    • | +| Preset identifier Data | 2 | uint16[] | The data in multiples of uint16 in Little-Endian format | + +**For example, to load the preset identifier `3` then the command looks like `f02101#00000\x01\x01\x03\x00`** + +## Switching Delays + +The delays in switching the presets of ODS application and reflecting in the PLC application results are dependent on the preset configuration. Please refer to the [preset documentation under load parameter](../ODS/Presets/presets.md). \ No newline at end of file diff --git a/PLC/test.json b/PLC/test.json new file mode 100644 index 0000000..be7754c --- /dev/null +++ b/PLC/test.json @@ -0,0 +1,69 @@ +{ + "applications": { + "instances": { + "app0": { + "class": "ods", + "ports": [ + "port2", + "port3", + "port6" + ], + "state": "CONF", + "presets": { + "definitions":{ + "0": { + "description": "Forward preset with three zones and one active camera", + "preset": { + "activePorts": [ + "port2" + ], + "zones": { + "zoneConfigID": 0, + "zoneType":"polygon", + "zoneCoordinates": [ + [[0, -0.3], [2.5, -0.3], [2.5, 0.3], [0, 0.3]], + [[2.5, -0.3], [3.5, -0.3], [3.5, 0.3], [2.5, 0.3]], + [[3.5, -0.3], [4.5, -0.3], [4.5, 0.3], [3.5, 0.3]] + ] + } + } + }, + "1": { + "description": "Backward preset with two zones and one active camera", + "preset": { + "activePorts": [ + "port3" + ], + "zones": { + "zoneConfigID": 1, + "zoneType":"polygon", + "zoneCoordinates": [ + [[-0.5, -0.3], [-3.0, -0.3], [-3.0, 0.3], [-0.5, 0.3]], + [[-3.0, -0.3], [-4.0, -0.3], [-4.0, 0.3], [-3.0, 0.3]] + ] + } + } + }, + "2": { + "description": "Left turn preset with three zones and two active cameras", + "preset": { + "activePorts": [ + "port2" + ], + "zones": { + "zoneConfigID": 2, + "zoneType":"polygon", + "zoneCoordinates": [ + [[0.5, 0.3], [1.5, 0.3], [1.5, 0.9], [0.5, 0.9]], + [[1.5, 0.3], [2.5, 0.3], [2.5, 0.9], [1.5, 0.9]], + [[2.5, 0.3], [3.5, 0.3], [3.5, 0.9], [2.5, 0.9]] + ] + } + } + } + } + } + } + } + } + } \ No newline at end of file diff --git a/SoftwareInterfaces/iVA/ReleaseNotes/index_iVA_release_notes.md b/SoftwareInterfaces/iVA/ReleaseNotes/index_iVA_release_notes.md index 00b4357..515536c 100644 --- a/SoftwareInterfaces/iVA/ReleaseNotes/index_iVA_release_notes.md +++ b/SoftwareInterfaces/iVA/ReleaseNotes/index_iVA_release_notes.md @@ -1,6 +1,7 @@ # Release notes :::{toctree} +Version 2.9.9 Version 2.8.7 Version 2.7.6 ::: \ No newline at end of file diff --git a/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.7.6.md b/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.7.6.md index aca74b1..c49308a 100644 --- a/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.7.6.md +++ b/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.7.6.md @@ -1,14 +1,14 @@ # Vision Assistant 2.7.6 -### Added +## Added - Motion Camera Calibration wizard. It is available as an application. -### Changed +## Changed - The ODS occupancy grid is displayed as a binary white/black instead of grayscale. The threshold is set at 127 by default: grid cells with a probability above 127 will be colored white, the others black. The right-hand side detailed view still shows the grayscale (this tab needs to be expanded by clicking on the two arrows on the right edge of the window). -### Fixed +## Fixed - Ports can now be smoothly added and removed in the application configuration. - The ports can be properly switched on/off in the "View options" in "Replay" mode. - The VPU temperature display is no longer obstructed by long port names. -### Known issues +## Known issues - Importing settings from file with the "Save and Reboot" enabled triggers error messages. ## Older versions diff --git a/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.9.9.md b/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.9.9.md new file mode 100644 index 0000000..ca76ca6 --- /dev/null +++ b/SoftwareInterfaces/iVA/ReleaseNotes/release_notes_2.9.9.md @@ -0,0 +1,45 @@ +# ifm Vision Assistant 2.9.9 release notes + +The following release note provides an overview of the features of the iVA **2.9.9** version, related to the O3R perception platform. +Refer to [ifm3d.com](http://www.ifm3d.com) for additional documentation. + +The software is available for windows and Linux (Ubuntu 20.04 and Ubuntu 22.04). It can be downloaded from [ifm.com](https://www.ifm.com/us/en/product/OVP810?tab=documents). + +## Previous Releases +Previous iVA release is version 2.8.7. + +## Compatible Video Processing Platforms (VPUs) +This firmware release can be applied to the following ifm video processing platforms: + +| Article | Description | +| ------- | ----------- | +| OVP800 | Series product | +| OVP801 | Series product including ODS license | +| OVP810 | Series product | +| OVP811 | Series product including ODS license | +| OVP812 | Series product including PDS license | +| OVP813 | Series product including ODS and PDS licenses | + +## New features +- Support for PDS applications. +- The distance noise and reflectivity images are available. +- The CAN network can be configured. +- The service report can be downloaded in the Device status tab. +- The IMU data can be visualized under Port 6. + +## Changes +- Improvements in ODS zone visualization: + - Possibility to set up to 16 points, + - Ability to visualize convex and concave shapes, depending on if the `convexHull` or the `polygon` `zoneType` is selected, + - Possibility to set zone coordinates up to +/- 15 meters, + - The zones are labeled in the application visualization. +- The port synchronization wizard was removed. Now, when calibrating the ports using the manual calibration wizard, both ports corresponding to a single camera are calibrated together. + +## Bugfixes +- The save init function is no longer called automatically when changing the network settings. The network settings are always saved persistently on the device, but this avoids persistently saving other settings at the same time. +- It is now possible to only import application instances (previously, other options had to be checked additionally). +- The maximum number of simultaneously active cameras for ODS can be up to three. +- The gateway field was missing for Ethernet 0 after deactivating DHCP. This is fixed. + +## Known Issues +- The firmware update using the iVA Linux AppImage may fail on the first attempt, causing the VPU to remain in recovery mode. \ No newline at end of file diff --git a/GettingStarted/ifmVisionAssistant/connecting_ifmVA_to_O3R.md b/SoftwareInterfaces/iVA/connecting_ifmVA_to_O3R.md similarity index 100% rename from GettingStarted/ifmVisionAssistant/connecting_ifmVA_to_O3R.md rename to SoftwareInterfaces/iVA/connecting_ifmVA_to_O3R.md diff --git a/GettingStarted/ifmVisionAssistant/first_steps_ifmVA.md b/SoftwareInterfaces/iVA/first_steps_ifmVA.md similarity index 100% rename from GettingStarted/ifmVisionAssistant/first_steps_ifmVA.md rename to SoftwareInterfaces/iVA/first_steps_ifmVA.md diff --git a/SoftwareInterfaces/iVA/iVA_with_linux.md b/SoftwareInterfaces/iVA/iVA_with_linux.md new file mode 100644 index 0000000..d215ea7 --- /dev/null +++ b/SoftwareInterfaces/iVA/iVA_with_linux.md @@ -0,0 +1,15 @@ +# ifmVisionAssistant on Linux + +To run the Vision Assistant on Linux, you can use the provided AppImage. + +1. Download the latest ifmVisionAssistant AppImage from the [official ifm website](https://www.ifm.com): navigate to the product page (OVP810 for example), and go to the Downloads tab: + ![Download the Vision Assistant from ifm.com](./resources/download_linux.png) +2. Unzip the downloaded archive. +3. You might need to change the permissions to be able to execute the file: + ```bash + chmod +x ifmVisionAssistant-REL_2.8.7-x86_64.AppImage + ``` +4. Launch the program: + ```bash + ./ifmVisionAssistant-REL_2.8.7-x86_64.AppImage + ``` diff --git a/SoftwareInterfaces/iVA/index_iVA.md b/SoftwareInterfaces/iVA/index_iVA.md index c025baf..cf47ab3 100644 --- a/SoftwareInterfaces/iVA/index_iVA.md +++ b/SoftwareInterfaces/iVA/index_iVA.md @@ -2,8 +2,12 @@ :::{toctree} Release notes +Introduction and installation +ifmVisionAssistant on Linux +Connect to the O3R +First steps Changing parameters Configuring applications Device and diagnostics information Manual calibration of ports for vehicle algorithms -::: \ No newline at end of file +::: diff --git a/GettingStarted/ifmVisionAssistant/introduction_and_installation.md b/SoftwareInterfaces/iVA/introduction_and_installation.md similarity index 100% rename from GettingStarted/ifmVisionAssistant/introduction_and_installation.md rename to SoftwareInterfaces/iVA/introduction_and_installation.md diff --git a/GettingStarted/ifmVisionAssistant/resources/blank_monitor.png b/SoftwareInterfaces/iVA/resources/blank_monitor.png similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/blank_monitor.png rename to SoftwareInterfaces/iVA/resources/blank_monitor.png diff --git a/GettingStarted/ifmVisionAssistant/resources/display_2D_in_3D.gif b/SoftwareInterfaces/iVA/resources/display_2D_in_3D.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/display_2D_in_3D.gif rename to SoftwareInterfaces/iVA/resources/display_2D_in_3D.gif diff --git a/SoftwareInterfaces/iVA/resources/download_linux.png b/SoftwareInterfaces/iVA/resources/download_linux.png new file mode 100644 index 0000000..4e41792 Binary files /dev/null and b/SoftwareInterfaces/iVA/resources/download_linux.png differ diff --git a/GettingStarted/ifmVisionAssistant/resources/main_view_2D.gif b/SoftwareInterfaces/iVA/resources/main_view_2D.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/main_view_2D.gif rename to SoftwareInterfaces/iVA/resources/main_view_2D.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/main_view_3D.gif b/SoftwareInterfaces/iVA/resources/main_view_3D.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/main_view_3D.gif rename to SoftwareInterfaces/iVA/resources/main_view_3D.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/manual_connect.gif b/SoftwareInterfaces/iVA/resources/manual_connect.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/manual_connect.gif rename to SoftwareInterfaces/iVA/resources/manual_connect.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/no_device_found.gif b/SoftwareInterfaces/iVA/resources/no_device_found.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/no_device_found.gif rename to SoftwareInterfaces/iVA/resources/no_device_found.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/old_display_2D_in_3D.gif b/SoftwareInterfaces/iVA/resources/old_display_2D_in_3D.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/old_display_2D_in_3D.gif rename to SoftwareInterfaces/iVA/resources/old_display_2D_in_3D.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/old_main_view_2D.gif b/SoftwareInterfaces/iVA/resources/old_main_view_2D.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/old_main_view_2D.gif rename to SoftwareInterfaces/iVA/resources/old_main_view_2D.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/old_main_view_3D.gif b/SoftwareInterfaces/iVA/resources/old_main_view_3D.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/old_main_view_3D.gif rename to SoftwareInterfaces/iVA/resources/old_main_view_3D.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/preview_2d_3D.png b/SoftwareInterfaces/iVA/resources/preview_2d_3D.png similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/preview_2d_3D.png rename to SoftwareInterfaces/iVA/resources/preview_2d_3D.png diff --git a/GettingStarted/ifmVisionAssistant/resources/recent_connections.gif b/SoftwareInterfaces/iVA/resources/recent_connections.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/recent_connections.gif rename to SoftwareInterfaces/iVA/resources/recent_connections.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/set_all_ports_to_run.gif b/SoftwareInterfaces/iVA/resources/set_all_ports_to_run.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/set_all_ports_to_run.gif rename to SoftwareInterfaces/iVA/resources/set_all_ports_to_run.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/start_screen.png b/SoftwareInterfaces/iVA/resources/start_screen.png similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/start_screen.png rename to SoftwareInterfaces/iVA/resources/start_screen.png diff --git a/GettingStarted/ifmVisionAssistant/resources/starting_ifmVA.gif b/SoftwareInterfaces/iVA/resources/starting_ifmVA.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/starting_ifmVA.gif rename to SoftwareInterfaces/iVA/resources/starting_ifmVA.gif diff --git a/GettingStarted/ifmVisionAssistant/resources/wine_iVA.gif b/SoftwareInterfaces/iVA/resources/wine_iVA.gif similarity index 100% rename from GettingStarted/ifmVisionAssistant/resources/wine_iVA.gif rename to SoftwareInterfaces/iVA/resources/wine_iVA.gif diff --git a/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_odsapp.md b/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_odsapp.md index 839958b..6ea94a8 100644 --- a/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_odsapp.md +++ b/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_odsapp.md @@ -15,3 +15,4 @@ | 105011 | ERROR_ODSAPP_EXTR_DI_CALIB_IMPLAUSIBLE | Implausible extrinsic head calibration | | 105012 | ERROR_ODSAPP_TEST_MODE_WARNING | A protected parameter was set to enter a test mode | | 105013 | ERROR_ODSAPP_ERROR_ODSAPP_FOV_INSUFFICIENT_FOR_NEGATIVE_OBSTACLES | FOV is insufficient for negative obstacle detection | +| 105019 | ERROR_ODSAPP_CAMERA_DECALIBRATED | Camera orientation deviates from expected value| \ No newline at end of file diff --git a/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_vpu.md b/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_vpu.md index 8f609e9..6300fa3 100644 --- a/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_vpu.md +++ b/SoftwareInterfaces/ifmDiagnostic/diagnosis_error_codes/FW1.1/section_vpu.md @@ -1,7 +1,3 @@ - - -### VPU - | ID | Name | Description | |----|------|-------------| | 103002 | ERROR_VPU_OVERVOLTAGE_24V | Overvoltage was detected for the 24V line | @@ -16,10 +12,5 @@ | 103011 | ERROR_VPU_NO_FREE_IMAGE_BUFFERS | More image buffers are needed than are available | | 103012 | ERROR_VPU_WATCHDOG_TIMEOUT | A watchdog timeout occurred; the component was restarted | | 103013 | ERROR_VPU_INVALID_CONFIGURATION | Unable to read a configuration path | - -### Communication - -| ID | Name | Description | -|----|------|-------------| | 103000 | ERROR_VPU_COMM_TCU | Unable to communicate with the TCU | | 103001 | ERROR_VPU_COMM_TEMPSENSOR | Unable to communicate with the temperature sensor | diff --git a/SoftwareInterfaces/ifmDiagnostic/diagnostic_reaction_strategy.md b/SoftwareInterfaces/ifmDiagnostic/diagnostic_reaction_strategy.md index 0b8ca8f..d94a05c 100644 --- a/SoftwareInterfaces/ifmDiagnostic/diagnostic_reaction_strategy.md +++ b/SoftwareInterfaces/ifmDiagnostic/diagnostic_reaction_strategy.md @@ -4,12 +4,12 @@ This reaction strategy document is motivated by the fact that different error co ## Severity levels definitions -| Severity | Meaning | -| --------- | ------- | -| Information | This level is used to convey information to the user. No persistent error has occurred. No action needs to be taken by the user. The system remains in full operational state.| -| Warning | This level indicates that an error was detected but is not serious enough to interfere with the running of the system and potential embedded applications.
    Performance of the system may be reduced. Please see the [handling strategies section](#handling-strategies-description) for more details on how fix the specific issue. | -| Severe Error | This level indicates that a serious error was detected and is persistent.
    This inhibits the full operability of the system and needs to be fixed by the user to ensure the full feature level and system performance.
    In comparison to `Unrecoverable` error cases, the system doesn't shut off the respective components automatically . The error may be fixed in live operation of the system by the user. Please see the [handling strategies section](#handling-strategies-description) for more details on how fix the specific issue.| -| Unrecoverable | This level usually indicates an error that forces termination of respective embedded process.
    In consequence in most cases at least a reboot is required to reset the respective components. Additional preliminary solutions steps preceding the reboot may be required. Please see the [handling strategies section](#handling-strategies-description) for more details on how fix the specific issue. | +| Severity | Meaning | +| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Information | This level is used to convey information to the user. No persistent error has occurred. No action needs to be taken by the user. The system remains in full operational state. | +| Warning | This level indicates that an error was detected but is not serious enough to interfere with the running of the system and potential embedded applications.
    Performance of the system may be reduced. Please see the [handling strategies section](#handling-strategies-description) for more details on how fix the specific issue. | +| Severe Error | This level indicates that a serious error was detected and is persistent.
    This inhibits the full operability of the system and needs to be fixed by the user to ensure the full feature level and system performance.
    In comparison to `Unrecoverable` error cases, the system doesn't shut off the respective components automatically . The error may be fixed in live operation of the system by the user. Please see the [handling strategies section](#handling-strategies-description) for more details on how fix the specific issue. | +| Unrecoverable | This level usually indicates an error that forces termination of respective embedded process.
    In consequence in most cases at least a reboot is required to reset the respective components. Additional preliminary solutions steps preceding the reboot may be required. Please see the [handling strategies section](#handling-strategies-description) for more details on how fix the specific issue. | ## Severity level dependent actions @@ -19,147 +19,143 @@ Please verify by the error code severity level if this error inhibits the operat ### Boot-Sequence -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 101014 | ERROR_BOOT_SEQUENCE_VPU_EEPROM | VPU EEPROM content invalid | Severe | VPU specific EEPROM content can not be read. [reboot](#reboot--power-cycle) if persistent [factory reset](#factory-reset)| -| 101018 | ERROR_BOOT_SEQUENCE_INVALID_CONFIGURATION | Unable to construct a valid device configuration | Unrecoverable | Bring-up failed and unrecoverable. Manual reconfiguration required. [reboot](#reboot--power-cycle) if persistent [factory reset](#factory-reset)| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ----------------------------------------- | ------------------------------------------------ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| 101014 | `ERROR_BOOT_SEQUENCE_VPU_EEPROM` | VPU EEPROM content invalid | Severe | VPU specific EEPROM content can not be read. [reboot](#reboot--power-cycle) if persistent [factory reset](#factory-reset) | +| 101018 | `ERROR_BOOT_SEQUENCE_INVALID_CONFIGURATION` | Unable to construct a valid device configuration | Unrecoverable | Bring-up failed and unrecoverable. Manual reconfiguration required. [reboot](#reboot--power-cycle) if persistent [factory reset](#factory-reset) | -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 101006 | ERROR_BOOT_SEQUENCE_PORT_CALIBRATION | The port calibration is invalid | Severe / Unrecoverable | [reboot](#reboot--power-cycle) if persistent [factory reset](#factory-reset) | -| 101007 | ERROR_BOOT_SEQUENCE_PORT_INVALID_CONFIGURATION | The init configuration does not match the port | Severe | The configuration saved on the device (`saveInit`) does not match the connected hard. A [factory reset](#factory-reset) to erase the persistent configuration is required. | -| 101015 | ERROR_BOOT_SEQUENCE_PORT_DUMMY_CALIBRATION | A dummy calibration is used for the port | Severe | A dummy calibration is used during runtime. Measurement performance and accuracy is reduced. [reboot](#reboot--power-cycle) if persistent after [factory reset](#factory-reset) the hardware needs to be replaced.| -| 101017 | ERROR_BOOT_SEQUENCE_PORT_CONFIGURATION_TIMEOUT | The imager did not respond to the frame software | Severe | Camera head did not respond within 5 sec after boot-up. [reboot](#reboot--power-cycle) if persistent over runtime and [re-flash FW](#flash-firmware-anew). A hardware replacement of the head might be required. | -| 101019 | ERROR_BOOT_SEQUENCE_PORT_EEPROM_OVERRIDE | A user-provided EEPROM file is used instead of the actual EEPROM | Severe | User specified EEPROM content used.
    Measurement performance and accuracy is reduced. [reboot](#reboot--power-cycle). If persistent after [factory reset](#factory-reset) the hardware needs to be replaced. | -| 101020 | ERROR_BOOT_SEQUENCE_PORT_CALIBRATION_OVERRIDE | A user-provided calibration file is used | Severe | User specified calibration used.
    Measurement performance and accuracy may be reduced. [reboot](#reboot--power-cycle). If persistent after [factory reset](#factory-reset) the hardware needs to be replaced. | -| 101021 | ERROR_BOOT_SEQUENCE_PORT_IDENTIFICATION | Port identification data is invalid | Severe | Camera head can not be identified. [reboot](#reboot--power-cycle). | +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ---------------------------------------------- | ---------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 101006 | `ERROR_BOOT_SEQUENCE_PORT_CALIBRATION` | The port calibration is invalid | Severe / Unrecoverable | [reboot](#reboot--power-cycle) if persistent [factory reset](#factory-reset) | +| 101007 | `ERROR_BOOT_SEQUENCE_PORT_INVALID_CONFIGURATION` | The init configuration does not match the port | Severe | The configuration saved on the device (`saveInit`) does not match the connected hard. A [factory reset](#factory-reset) to erase the persistent configuration is required. | +| 101015 | `ERROR_BOOT_SEQUENCE_PORT_DUMMY_CALIBRATION` | A dummy calibration is used for the port | Severe | A dummy calibration is used during runtime. Measurement performance and accuracy is reduced. [reboot](#reboot--power-cycle) if persistent after [factory reset](#factory-reset) the hardware needs to be replaced. | +| 101017 | `ERROR_BOOT_SEQUENCE_PORT_CONFIGURATION_TIMEOUT` | The imager did not respond to the frame software | Severe | Camera head did not respond within 5 sec after boot-up. [reboot](#reboot--power-cycle) if persistent over runtime and [re-flash FW](#flash-firmware-anew). A hardware replacement of the head might be required. | +| 101019 | `ERROR_BOOT_SEQUENCE_PORT_EEPROM_OVERRIDE` | A user-provided EEPROM file is used instead of the actual EEPROM | Severe | User specified EEPROM content used.
    Measurement performance and accuracy is reduced. [reboot](#reboot--power-cycle). If persistent after [factory reset](#factory-reset) the hardware needs to be replaced. | +| 101020 | `ERROR_BOOT_SEQUENCE_PORT_CALIBRATION_OVERRIDE` | A user-provided calibration file is used | Severe | User specified calibration used.
    Measurement performance and accuracy may be reduced. [reboot](#reboot--power-cycle). If persistent after [factory reset](#factory-reset) the hardware needs to be replaced. | +| 101021 | `ERROR_BOOT_SEQUENCE_PORT_IDENTIFICATION` | Port identification data is invalid | Severe | Camera head can not be identified. [reboot](#reboot--power-cycle). | -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 101000 | ERROR_BOOT_SEQUENCE_TCU_INVALID_FW | The installed TCU firmware is incompatible | Severe / Unrecoverable | TCU firmware is not compatible / was not updated automatically to the correct FW dependent version
    [reboot](#reboot--power-cycle) if persistent [re-flash FW](#flash-firmware-anew).| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ---------------------------------- | ------------------------------------------ | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 101000 | `ERROR_BOOT_SEQUENCE_TCU_INVALID_FW` | The installed TCU firmware is incompatible | Severe / Unrecoverable | TCU firmware is not compatible / was not updated automatically to the correct FW dependent version
    [reboot](#reboot--power-cycle) if persistent [re-flash FW](#flash-firmware-anew). | -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 101004 | ERROR_BOOT_SEQUENCE_HEAD_INVALID_DRIVER | The appropriate driver for the connected head is not available | Unrecoverable | The camera head hardware may not be supported by this VPU and firmware. Disconnect all camera heads
    [reboot](#reboot--power-cycle) and connect one camera head at a time to debug the problem.| -| 101008 | ERROR_BOOT_SEQUENCE_HEAD_INVALID_SERIALNUMBER | The serial number of an extrinsically calibrated head does not match; please recalibrate after changing heads | Unrecoverable | The extrinsic calibration saved on the device (`saveInit`) does not match the connected hard at this specific port. [Factory reset](#factory-reset) to remove all the persistently saved settings or debug the configuration manually for each port. | -| 101016 | ERROR_BOOT_SEQUENCE_HEAD_INVALID_COMBINATION | Only 2D/2D pairing or 3D/3D pairing is allowed | Unrecoverable | Different camera streams connected to the same deserializer. Update hardware connectivity and [reboot](#reboot--power-cycle) | +| ID | Name | Description | Severity level | Handling strategy | +| ------ | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 101004 | `ERROR_BOOT_SEQUENCE_HEAD_INVALID_DRIVER` | The appropriate driver for the connected head is not available | Unrecoverable | The camera head hardware may not be supported by this VPU and firmware. Disconnect all camera heads
    [reboot](#reboot--power-cycle) and connect one camera head at a time to debug the problem. | +| 101008 | `ERROR_BOOT_SEQUENCE_HEAD_INVALID_SERIALNUMBER` | The serial number of an extrinsically calibrated head does not match; please recalibrate after changing heads | Unrecoverable | The extrinsic calibration saved on the device (`saveInit`) does not match the connected hard at this specific port. [Factory reset](#factory-reset) to remove all the persistently saved settings or debug the configuration manually for each port. | +| 101016 | `ERROR_BOOT_SEQUENCE_HEAD_INVALID_COMBINATION` | Only 2D/2D pairing or 3D/3D pairing is allowed | Unrecoverable | Different camera streams connected to the same deserializer. Update hardware connectivity and [reboot](#reboot--power-cycle) | -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 101010 | ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_CHECK | Due to incorrect ICC firmware data an ICC update is attempted | Warning | The illumination controller FW does not match the one required by the device FW. Wait for the ICC FW update to complete. If persistent over > 5 min [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | -| 101011 | ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_UPDATE | The ICC firmware data is still incorrect after an ICC update | -| 101012 | ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_INCOMPATIBLE | Due to an incompatible ICC firmware version an ICC update is attempted | Warning | The illumination controller FW does not match the one required by the device FW. Wait for the ICC FW update to complete. If persistent over > 5 min [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | -| 101011 | ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_UPDATE | The ICC firmware data is still incorrect after an ICC update | -| 101013 | ERROR_BOOT_SEQUENCE_HEAD_ICC_FLASH | Unable to read the ICC flash | Unrecoverable | The illumination controller FW does not match the one required by the device FW. [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | -| 101011 | ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_UPDATE | The ICC firmware data is still incorrect after an ICC update | Unrecoverable | The illumination controller FW does not match the one required by the device FW. [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | +| ID | Name | Description | Severity level | Handling strategy | +| ------ | -------------------------------------------- | ---------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 101010 | `ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_CHECK` | Due to incorrect ICC firmware data an ICC update is attempted | Warning | The illumination controller FW does not match the one required by the device FW. Wait for the ICC FW update to complete. If persistent over > 5 min [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | +| 101011 | `ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_UPDATE` | The ICC firmware data is still incorrect after an ICC update | +| 101012 | `ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_INCOMPATIBLE` | Due to an incompatible ICC firmware version an ICC update is attempted | Warning | The illumination controller FW does not match the one required by the device FW. Wait for the ICC FW update to complete. If persistent over > 5 min [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | +| 101011 | `ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_UPDATE` | The ICC firmware data is still incorrect after an ICC update | +| 101013 | `ERROR_BOOT_SEQUENCE_HEAD_ICC_FLASH` | Unable to read the ICC flash | Unrecoverable | The illumination controller FW does not match the one required by the device FW. [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | +| 101011 | `ERROR_BOOT_SEQUENCE_HEAD_ICC_FW_UPDATE` | The ICC firmware data is still incorrect after an ICC update | Unrecoverable | The illumination controller FW does not match the one required by the device FW. [reboot](#reboot--power-cycle). If persistent over reboots [re-flash FW](#flash-firmware-anew). | -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 101002 | ERROR_BOOT_SEQUENCE_IMU_INVALID_HW | The installed IMU (if any) was not recognised | Unrecoverable | Detected IMU hardware is not compatible.
    [reboot](#reboot--power-cycle).| -| 101003 | ERROR_BOOT_SEQUENCE_IMU_INVALID_CALIBRATION | The IMU calibration is either missing or incorrect | Unrecoverable | Invalid IMU calibration used at boot-up - [re-flash FW](#flash-firmware-anew) | +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ------------------------------------------- | -------------------------------------------------- | -------------- | ----------------------------------------------------------------------------- | +| 101002 | `ERROR_BOOT_SEQUENCE_IMU_INVALID_HW` | The installed IMU (if any) was not recognised | Unrecoverable | Detected IMU hardware is not compatible.
    [reboot](#reboot--power-cycle). | +| 101003 | `ERROR_BOOT_SEQUENCE_IMU_INVALID_CALIBRATION` | The IMU calibration is either missing or incorrect | Unrecoverable | Invalid IMU calibration used at boot-up - [re-flash FW](#flash-firmware-anew) | ### VPU -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 103002 | ERROR_VPU_OVERVOLTAGE_24V | Overvoltage was detected for the 24V line | Warning | Fix power supply | -| 103003 | ERROR_VPU_UNDERVOLTAGE_24V | Undervoltage was detected for the 24V line | Unrecoverable | This error may cause undefined system states. Fix power supply and [reboot](#reboot--power-cycle) | -| 103004 | ERROR_VPU_OVERVOLTAGE_5V | Overvoltage was detected for the 5V line | Warning | Fix power supply | -| 103005 | ERROR_VPU_UNDERVOLTAGE_5V | Undervoltage was detected for the 5V line | Unrecoverable | This error may cause undefined system states. Fix power supply and [reboot](#reboot--power-cycle) | -| 103006 | ERROR_VPU_OVERVOLTAGE_1_8V | Overvoltage was detected for the 1.8V line | Warning | Fix power supply | -| 103007 | ERROR_VPU_UNDERVOLTAGE_1_8V | Undervoltage was detected for the 1.8V line | Unrecoverable | This error may cause undefined system states. Fix power supply and [reboot](#reboot--power-cycle) | -| 103008 | ERROR_VPU_OVERTEMPERATURE | Detected overtemperature for VPU main board | Severe |[Provide adequate (passive) cooling via heat conduction and convection](../../ODS/DeviceVerification/HeatDissipation/heat_dissipation_guidelines.md/). The systems CPU and GPU performance will be throttled until adequate temperatures are reached. | -| 103009 | ERROR_VPU_NTP_NOT_SYNCHRONIZED | Unable to synchronise time with NTP server | Warning | Check NTP server availability and correct configuration.
    A reboot may be required to avoid time jumps in live operation due to syncing of clocks. The NTP time will be directly applied at next start-up. | -| 103010 | ERROR_VPU_TRIGGER_OVERRUN | A trigger overrun was detected | Severe | The trigger signal are not consistent: data may be lost due to lost trigger signals.
    Verify software trigger sending units on your IPC. | -| 103011 | ERROR_VPU_NO_FREE_IMAGEBUFFERS | More image buffers are needed than are available | Warning (sporadic) /Severe (permanent) | The data is not send fast enough. In consequence the internal queues have filled up. This may be because of internal application streams switching or because of a slow receiver.
    If persistent please verify network transmission speeds: GBit/s is required. Verify the receiver is adequate to receive the data frames fast enough via TCP/IP. | -| 103012 | ERROR_VPU_WATCHDOG_TIMEOUT | A watchdog timeout occurred; the component was restarted | Severe | Internal data evaluation processes have not responded in time and are restarted.
    The processes have been restarted but data frames may have been lost in the meanwhile. This diagnostic element is persistent (dormant) on the log after detection. To reset the logs completely a reboot is required.
    If the user experiences this problem more than once, please send a trace of the systems logs via email to support.efector.object-ident@ifm.com. The system trace can be retrieved via the [ifm3d API](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.device.Device.html#ifm3dpy.device.Device.trace_logs) and [ifm3d API's CLI](https://api.ifm3d.com/stable/content/cmdline_overview.html).| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ------------------------------ | -------------------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 103002 | `ERROR_VPU_OVERVOLTAGE_24V` | Overvoltage was detected for the 24V line | Warning | Fix power supply | +| 103003 | `ERROR_VPU_UNDERVOLTAGE_24V` | Undervoltage was detected for the 24V line | Unrecoverable | This error may cause undefined system states. Fix power supply and [reboot](#reboot--power-cycle) | +| 103004 | `ERROR_VPU_OVERVOLTAGE_5V` | Overvoltage was detected for the 5V line | Warning | Fix power supply | +| 103005 | `ERROR_VPU_UNDERVOLTAGE_5V` | Undervoltage was detected for the 5V line | Unrecoverable | This error may cause undefined system states. Fix power supply and [reboot](#reboot--power-cycle) | +| 103006 | `ERROR_VPU_OVERVOLTAGE_1_8V` | Overvoltage was detected for the 1.8V line | Warning | Fix power supply | +| 103007 | `ERROR_VPU_UNDERVOLTAGE_1_8V` | Undervoltage was detected for the 1.8V line | Unrecoverable | This error may cause undefined system states. Fix power supply and [reboot](#reboot--power-cycle) | +| 103008 | `ERROR_VPU_OVERTEMPERATURE` | Detected overtemperature for VPU main board | Severe | [Provide adequate (passive) cooling via heat conduction and convection](../../Technology/Hardware/Mounting/heat_dissipation_guidelines.md). The systems CPU and GPU performance will be throttled until adequate temperatures are reached. | +| 103009 | `ERROR_VPU_NTP_NOT_SYNCHRONIZED` | Unable to synchronise time with NTP server | Warning | Check NTP server availability and correct configuration.
    A reboot may be required to avoid time jumps in live operation due to syncing of clocks. The NTP time will be directly applied at next start-up. | +| 103010 | `ERROR_VPU_TRIGGER_OVERRUN` | A trigger overrun was detected | Severe | The trigger signal are not consistent: data may be lost due to lost trigger signals.
    Verify software trigger sending units on your IPC. | +| 103011 | `ERROR_VPU_NO_FREE_IMAGEBUFFERS` | More image buffers are needed than are available | Warning (sporadic) /Severe (permanent) | The data is not send fast enough. In consequence the internal queues have filled up. This may be because of internal application streams switching or because of a slow receiver.
    If persistent please verify network transmission speeds: GBit/s is required. Verify the receiver is adequate to receive the data frames fast enough via TCP/IP. | +| 103012 | `ERROR_VPU_WATCHDOG_TIMEOUT` | A watchdog timeout occurred; the component was restarted | Severe | Internal data evaluation processes have not responded in time and are restarted.
    The processes have been restarted but data frames may have been lost in the meanwhile. This diagnostic element is persistent (dormant) on the log after detection. To reset the logs completely a reboot is required.
    If the user experiences this problem more than once, please send a trace of the systems logs via email to support.efector.object-ident@ifm.com. The system trace can be retrieved via the [ifm3d API](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.device.Device.html#ifm3dpy.device.Device.trace_logs) and [ifm3d API's CLI](https://api.ifm3d.com/stable/content/cmdline_overview.html). | -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 103000 | ERROR_VPU_COMM_TCU | Unable to communicate to the TCU | Unrecoverable | The internal trigger mechanism is disturbed and image acquisition is not triggered reliably. [Reboot](#reboot--power-cycle) | -| 103001 | ERROR_VPU_COMM_TEMPSENSOR | Unable to communicate to the temperature sensor | Warning | The internal communication to the VPUs temperature sensor is disturbed. Temperature values might not be accurate. If persistent [reboot](#reboot--power-cycle) | +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ------------------------- | ----------------------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 103000 | `ERROR_VPU_COMM_TCU` | Unable to communicate to the TCU | Unrecoverable | The internal trigger mechanism is disturbed and image acquisition is not triggered reliably. [Reboot](#reboot--power-cycle) | +| 103001 | `ERROR_VPU_COMM_TEMPSENSOR` | Unable to communicate to the temperature sensor | Warning | The internal communication to the VPUs temperature sensor is disturbed. Temperature values might not be accurate. If persistent [reboot](#reboot--power-cycle) | ### Distance image processing -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 102015 | ERROR_DI_UNEXPECTED_CALIB_CONTENT | The port calibration contains unexpected values | Unrecoverable | Sanity checks of the 3D calibration file failed. The data acquisition of the respective camera is blocked. This is most likely due to old / incompatible hardware.
    If error is persistent [factory reset](#factory-reset) and [re-flash FW](#flash-firmware-anew) or replace hardware.| -| 102016 | ERROR_DI_MOTION_COMP_NO_EGO_DATA | Expected motion compensation data, but none was received | Warning | If motion compensation is activated but can not be performed due to missing ego data.
    The motion compensation feature is available only in combination with a live ODS application. Reset the port parameters to default to disable motion compensation. | -| 102017 | ERROR_DI_MOTION_COMP_EGA_DATA_TIMESTAMP_MISMATCH | Received motion compensation data is too old or contains implausible timestamps | Warning | If motion compensation is activated but can not be performed due to ego data unavailability or outdated timestamps.
    The motion compensation feature is available only in combination with a live ODS application. Reset the port parameters to default to disable motion compensation.| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ------------------------------------------------ | ------------------------------------------------------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 102015 | `ERROR_DI_UNEXPECTED_CALIB_CONTENT` | The port calibration contains unexpected values | Unrecoverable | Sanity checks of the 3D calibration file failed. The data acquisition of the respective camera is blocked. This is most likely due to old / incompatible hardware.
    If error is persistent [factory reset](#factory-reset) and [re-flash FW](#flash-firmware-anew) or replace hardware. | +| 102016 | `ERROR_DI_MOTION_COMP_NO_EGO_DATA` | Expected motion compensation data, but none was received | Warning | If motion compensation is activated but can not be performed due to missing ego data.
    The motion compensation feature is available only in combination with a live ODS application. Reset the port parameters to default to disable motion compensation. | +| 102017 | `ERROR_DI_MOTION_COMP_EGA_DATA_TIMESTAMP_MISMATCH` | Received motion compensation data is too old or contains implausible timestamps | Warning | If motion compensation is activated but can not be performed due to ego data unavailability or outdated timestamps.
    The motion compensation feature is available only in combination with a live ODS application. Reset the port parameters to default to disable motion compensation. | ### Camera head -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 102000 | ERROR_HEAD_OVERTEMPERATURE_ICC | Head ICC has overtemperature | Severe | [Provide adequate (passive) cooling via heat conduction and convection](../../ODS/DeviceVerification/HeatDissipation/heat_dissipation_guidelines.md/).
    The camera head will not be useable until it has cooled down to operating temperature levels again. | -| 102001 | ERROR_HEAD_BROWNOUT | Detected head brownout | Severe | Fix power supply - if active / persistent [reboot](#reboot--power-cycle) to reset / reboot heads. | -| 102002 | ERROR_HEAD_ENVELOPE_PAUSE_VIOLATION | Detected violation of the envelope pause ratio | Severe | If detected the system will enter an error state and block the image acquisition - to reset, [reboot](#reboot--power-cycle).| -| 102003 | ERROR_HEAD_OVERVOLTAGE | Head has overvoltage | Severe | The respective head will be blocked from acquiring images until no overvoltage is present - fix the power supply and cables.| -| 102004 | ERROR_HEAD_UNDERVOLTAGE | Head has undervoltage | Severe | The respective head will be blocked from acquiring images until no undervoltage is present - fix power supply and cable hardware. A reboot may be required to exit the error state.| -| 102005 | ERROR_HEAD_VCSEL_OVERCURRENT | Head VCSEL has overcurrent | Unrecoverable | The active illumination has shut off. The respective head will be blocked from acquiring images until the system is [rebooted](#reboot--power-cycle)| -| 102006 | ERROR_HEAD_EYESAFETY_SHUTDOWN | Head was shut down due to eye-safety concerns | Unrecoverable | The active illumination has shut off due to a eye safety violation. The respective head (including its illumination) will be blocked until system is [rebooted](#reboot--power-cycle)| -| 102007 | ERROR_HEAD_INVALID_CALIBRATION | Head current calibration is invalid | Unrecoverable | The respective head has detected a check sum error for its calibration - will be blocked until system is [rebooted](#reboot--power-cycle)
    if persistent [factory reset](#factory-reset), finally [re-flash FW](#flash-firmware-anew) and/or replace hardware. | -| 102008 | ERROR_HEAD_ENVELOPE_PULSE_ERROR | Detected pulse error for envelope signal | Unrecoverable | The active illumination has shut off due to a eye safety violation. The respective head will be blocked until system is [rebooted](#reboot--power-cycle)| -| 102009 | ERROR_HEAD_VCSEL_OVERTEMPERATURE | Head VCSEL has overtemperature | Unrecoverable | The active illumination has shut off. The respective head will be blocked until system is [rebooted](#reboot--power-cycle)| -| 102010 | ERROR_HEAD_ENVELOPE_MAX_DURATION | Maximum length of envelope signal was detected | Unrecoverable | The active illumination has shut off due to a eye safety violation. The active illumination has shut off. The respective head will be blocked until system is [rebooted](#reboot--power-cycle)| -| 102012 | ERROR_HEAD_IMAGER_RESET | The 3D imager was unexpectedly reset | Severe | The respective imager process was reset. Data frames may be lost in the meanwhile. | -| 102019 | ERROR_HEAD_VCSEL_OVERVOLTAGE | Head VCSEL has overvoltage | Unrecoverable | The respective head will be blocked from acquiring images until no overvoltage is present - fix the power supply and cables.| -| 102020 | ERROR_HEAD_ENVELOPE_PAUSE_CURRENT | Detected a violation of the envelope pause current | Unrecoverable | The active illumination has shut off. The respective head will be blocked from acquiring images until the system is [rebooted](#reboot--power-cycle)| -| 102021 | ERROR_HEAD_VCSEL_SHUTDOWN | Head VCSEL was shut down; VPU must be rebooted to recover | Unrecoverable |The active illumination has shut off. The respective head will be blocked from acquiring images until the system is [rebooted](#reboot--power-cycle)| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | ----------------------------------- | --------------------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 102000 | `ERROR_HEAD_OVERTEMPERATURE_ICC` | Head ICC has overtemperature | Severe | [Provide adequate (passive) cooling via heat conduction and convection](../../Technology/Hardware/Mounting/heat_dissipation_guidelines.md).
    The camera head will not be useable until it has cooled down to operating temperature levels again. | +| 102001 | `ERROR_HEAD_BROWNOUT` | Detected head brownout | Severe | Fix power supply - if active / persistent [reboot](#reboot--power-cycle) to reset / reboot heads. | +| 102002 | `ERROR_HEAD_ENVELOPE_PAUSE_VIOLATION` | Detected violation of the envelope pause ratio | Severe | If detected the system will enter an error state and block the image acquisition - to reset, [reboot](#reboot--power-cycle). | +| 102003 | `ERROR_HEAD_OVERVOLTAGE` | Head has overvoltage | Severe | The respective head will be blocked from acquiring images until no overvoltage is present - fix the power supply and cables. | +| 102004 | `ERROR_HEAD_UNDERVOLTAGE` | Head has undervoltage | Severe | The respective head will be blocked from acquiring images until no undervoltage is present - fix power supply and cable hardware. A reboot may be required to exit the error state. | +| 102005 | `ERROR_HEAD_VCSEL_OVERCURRENT` | Head VCSEL has overcurrent | Unrecoverable | The active illumination has shut off. The respective head will be blocked from acquiring images until the system is [rebooted](#reboot--power-cycle) | +| 102006 | `ERROR_HEAD_EYESAFETY_SHUTDOWN` | Head was shut down due to eye-safety concerns | Unrecoverable | The active illumination has shut off due to a eye safety violation. The respective head (including its illumination) will be blocked until system is [rebooted](#reboot--power-cycle) | +| 102007 | `ERROR_HEAD_INVALID_CALIBRATION` | Head current calibration is invalid | Unrecoverable | The respective head has detected a check sum error for its calibration - will be blocked until system is [rebooted](#reboot--power-cycle)
    if persistent [factory reset](#factory-reset), finally [re-flash FW](#flash-firmware-anew) and/or replace hardware. | +| 102008 | `ERROR_HEAD_ENVELOPE_PULSE_ERROR` | Detected pulse error for envelope signal | Unrecoverable | The active illumination has shut off due to a eye safety violation. The respective head will be blocked until system is [rebooted](#reboot--power-cycle) | +| 102009 | `ERROR_HEAD_VCSEL_OVERTEMPERATURE` | Head VCSEL has overtemperature | Unrecoverable | The active illumination has shut off. The respective head will be blocked until system is [rebooted](#reboot--power-cycle) | +| 102010 | `ERROR_HEAD_ENVELOPE_MAX_DURATION` | Maximum length of envelope signal was detected | Unrecoverable | The active illumination has shut off due to a eye safety violation. The active illumination has shut off. The respective head will be blocked until system is [rebooted](#reboot--power-cycle) | +| 102012 | `ERROR_HEAD_IMAGER_RESET` | The 3D imager was unexpectedly reset | Severe | The respective imager process was reset. Data frames may be lost in the meanwhile. | +| 102019 | `ERROR_HEAD_VCSEL_OVERVOLTAGE` | Head VCSEL has overvoltage | Unrecoverable | The respective head will be blocked from acquiring images until no overvoltage is present - fix the power supply and cables. | +| 102020 | `ERROR_HEAD_ENVELOPE_PAUSE_CURRENT` | Detected a violation of the envelope pause current | Unrecoverable | The active illumination has shut off. The respective head will be blocked from acquiring images until the system is [rebooted](#reboot--power-cycle) | +| 102021 | `ERROR_HEAD_VCSEL_SHUTDOWN` | Head VCSEL was shut down; VPU must be rebooted to recover | Unrecoverable | The active illumination has shut off. The respective head will be blocked from acquiring images until the system is [rebooted](#reboot--power-cycle) | ### IMU -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 104000 | ERROR_IMU_DATA_IMPLAUSIBLE | Received implausible IMU data | Severe | The system checks for IMU timestamp and temperature value ar implausible. This warning is active until new plausible data is available.
    If error is persistent over longer time durations [reboot](#reboot--power-cycle). If error persists over reboots in long-term system operation the hardware may be corrupted. A VPU hardware replacement may be required.| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | -------------------------- | ----------------------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 104000 | `ERROR_IMU_DATA_IMPLAUSIBLE` | Received implausible IMU data | Severe | The system checks for IMU timestamp and temperature value ar implausible. This warning is active until new plausible data is available.
    If error is persistent over longer time durations [reboot](#reboot--power-cycle). If error persists over reboots in long-term system operation the hardware may be corrupted. A VPU hardware replacement may be required. | ### Hardware port -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 102011 | ERROR_PORT_FRAME_TIMEOUT | The 3D imager detected a frame timeout | Warning (sporadic) / Severe (permanent) | Data may have been lost, the imager process has recovered. If persistently active [reboot](#reboot--power-cycle) | -| 102013 | ERROR_PORT_FPDLINK | An unrecoverable FPD-Link error occurred | Severe / Unrecoverable | [reboot](#reboot--power-cycle) if persistent check for [hardware defects](#fpd-link-errors) | -| 102014 | ERROR_PORT_ALGO_INTERNAL | Internal error in the port algorithm | Warning (sporadic) / Severe (permanent) | If persistent [reboot](#reboot--power-cycle) for non-series hardware check [internal Algorithmic errors](#internal-error-in-the-port-algorithm).
    If the user experiences this problem more than once, please send a trace of the systems logs via email to support.efector.object-ident@ifm.com. The system trace can be retrieved via the [ifm3d API](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.device.Device.html#ifm3dpy.device.Device.trace_logs) and [ifm3d API's CLI](https://api.ifm3d.com/stable/content/cmdline_overview.html).| -| 102018 | ERROR_PORT_MODE_ALGO_INCOMPATIBLE | The appropriate algorithm for the selected mode is not available | Unrecoverable | If persistent [reboot](#reboot--power-cycle). If persistent [re-flash FW](#flash-firmware-anew)| +| ID | Name | Description | Severity level | Handling strategy | +| ------ | --------------------------------- | ---------------------------------------------------------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 102011 | `ERROR_PORT_FRAME_TIMEOUT` | The 3D imager detected a frame timeout | Warning (sporadic) / Severe (permanent) | Data may have been lost, the imager process has recovered. If persistently active [reboot](#reboot--power-cycle) | +| 102013 | `ERROR_PORT_FPDLINK` | An unrecoverable FPD-Link error occurred | Severe / Unrecoverable | [reboot](#reboot--power-cycle) if persistent check for [hardware defects](#fpd-link-errors) | +| 102014 | `ERROR_PORT_ALGO_INTERNAL` | Internal error in the port algorithm | Warning (sporadic) / Severe (permanent) | If persistent [reboot](#reboot--power-cycle) for non-series hardware check [internal Algorithmic errors](#internal-error-in-the-port-algorithm).
    If the user experiences this problem more than once, please send a trace of the systems logs via email to support.efector.object-ident@ifm.com. The system trace can be retrieved via the [ifm3d API](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.device.Device.html#ifm3dpy.device.Device.trace_logs) and [ifm3d API's CLI](https://api.ifm3d.com/stable/content/cmdline_overview.html). | +| 102018 | `ERROR_PORT_MODE_ALGO_INCOMPATIBLE` | The appropriate algorithm for the selected mode is not available | Unrecoverable | If persistent [reboot](#reboot--power-cycle). If persistent [re-flash FW](#flash-firmware-anew) | ### ODS application -| ID | Name | Description | Severity level | Handling strategy | -|----|------|-------------| -------------- | ----------------- | -| 105000 | ERROR_ODSAPP_EXTR_VPU_CALIB_IMPLAUSIBLE | Implausible extrinsic VPU calibration | Severe | The app checks plausible IMU / VPU extrinsic calibration. Perform a IMU / VPU calibration and sanity check [link](../../ODS/ExtrinsicCalibration/index_extrinsic_calibration.md).
    For more details see the chapter [Implausible extrinsic calibrations](#implausible-extrinsic-calibrations)| -| 105001 | ERROR_ODSAPP_IMAGE_IMU_DELAY_IMPLAUSIBLE | Detected an implausible delay between IMU and image data | Warning (sporadic) / Severe (persistent) | The app checks plausible 3D data timestamps. No internal escalation is performed if active.
    If persistent reconfigure the ODS application / reboot.
    If the user experiences this problem more than once, please send a trace of the systems logs via email to support.efector.object-ident@ifm.com. The system trace can be retrieved via the [ifm3d API](https://api.ifm3d.com/stable/_autosummary/ifm3dpy.device.Device.html#ifm3dpy.device.Device.trace_logs) and [ifm3d API's CLI](https://api.ifm3d.com/stable/content/cmdline_overview.html).| -| 105002 | ERROR_ODSAPP_VO_EXTR_DI_CALIB_IMPLAUSIBLE | Implausible extrinsic calibration of visual odometry head | Warning | The app checks camera specific calibration for plausibility. If this error is active the respective camera can not be used in this app for estimating the systems odometry.
    If the camera is setup to see the floor and the diagnostic is persistent verify the mounting setup and extrinsic calibration values. See the additional comments in [camera head calibrations](#camera-head-extrinsic-calibrations) and [implausible extrinsic calibrations](#implausible-extrinsic-calibrations).| -| 105003 | ERROR_ODSAPP_PARAMETER_PLAUSIBILITY_CHECK_FAILED | Implausible ODS configuration parameters | Severe | The app checks its configuration for plausibility. If implausible the app is set to error state and has to be reconfigured to exit error state.
    [Check the app specific JSON schema for configuration verification](../../ODS/Configuration/configuration.md).| -| 105004 | ERROR_ODSAPP_VO_IMAGE_FRAMERATE | Insufficient framerate of ODS input images | Warning (sporadic) / Severe (permanent) | The app checks plausible 3D frame timestamps. No internal escalation is performed if active.
    Sporadic diagnostics are expected. If this error is permanently active then system performance may be severely reduced. To fix the issue, [reboot](#reboot--power-cycle) / [re-flash](#flash-firmware-anew). Depending on the additional load from OEM Docker containers, these errors may be activated. If the error persists, please reduce your system load (especially CPU and GPU) to ensure uninterrupted ODS output. | -| 105005 | ERROR_ODSAPP_INTR_IMU_CALIB_IMPLAUSIBLE | Implausible intrinsic IMU calibration | Warning | The app checks performs repeated IMU standstill calibration during live operation. If the estimated calibration values differ more than the internal threshold from their expected values this error will be activated.
    This is an internal error - no internal escalation strategy will be applied. It will be automatically be reset by the application at the next possible instance. Please check the suggestions [Mounting setup](#mounting-setup). If the error persists and has only occurred during operation, the IMU may be defective: please verify IMU functionality using the provided [IMU extrinsic calibration verification strategy](../../CalibrationRoutines/OVPCalibration/README.md).| -| 105006 | ERROR_ODSAPP_UNSTABLE_FRAMERATES | Unstable framerate of at least one input stream | Warning (sporadic) / Severe (permanent) |At least one sub-component of the app is unstable due to unstable framerate of the input data stream. See [ODS unstable framerate](#ods-unstable-framerate-implications) | -| 105007 | ERROR_ODSAPP_VELOCITY_UNAVAILABLE | Unable to determine velocity | Warning |The app performs ego-motion estimations. No internal escalation is performed if active. [Mounting setup](#mounting-setup).| -| 105008 | ERROR_ODSAPP_DEFAULT_ZONE_USED | Using the default ODS zones | Info | The app checks for default zone configuration. No internal escalation is performed if active. To silence this warning change the setting of `/applications/instances/appX/configuration/zones/zoneConfigID` | -| 105009 | ERROR_ODSAPP_STANDSTILL_CHECKS_NOT_EXECUTED | Standstill condition not detected | Warning / Severe | The app failed to perform ego-motion estimations. If the camera is not setup to see the floor, this information can be disregarded.
    An initial standstill period of about five seconds after the O3R ODS application is mandatory, since plausibility checks must be executed and IMU biases have to be estimated before ego motion can become available.
    This error is set on initialization of the O3R ODS application. It is reset after plausibility checks have been executed once. If the error persists even though the vehicle is at standstill and a large portion of the floor is visible for the camera assigned for visual odometry, this is an indication that there is an issue either with the IMU or with the extrinsic camera calibration. | -| 105011 | ERROR_ODSAPP_EXTR_DI_CALIB_IMPLAUSIBLE | Implausible extrinsic head calibration | Severe | The app checks plausible extrinsic head calibration. The O3R ODS app is functional in this state. The extrinsic calibration of the ODS ports must be fixed before new application data will be send.
    A `RUN`-`CONF`-`RUN` state change of the application may be required to reinitialize the application.
    [Check your mounting setup](#mounting-setup).| -| 105012 | ERROR_ODSAPP_TEST_MODE_WARNING | A protected parameter was set to enter a test mode | Severe | The app is configured with development features active, for example the overwriting of velocity information status. Such behavior is not expected on devices in operation. To fix this issue please [reboot](#reboot--power-cycle), [re-flash](#flash-firmware-anew). If the error persists after a re-flash please reach out to ifm at support.efector.object-ident@ifm.com.| -| 105013 | ERROR_ODSAPP_ERROR_ODSAPP_FOV_INSUFFICIENT_FOR_NEGATIVE_OBSTACLES | FOV is insufficient for negative obstacle detection | Severe | The app detects insufficient amount of floor. No negative obstacles can be detected - the ODS performance is reduced.
    Please verify an active negative obstacle detection configuration under `/applications/instances/appX/configuration/portX/negObst/enableNegativeObstacles`.
    A [different mounting](../../ODS/Mounting/mounting.md) setup may be required to enable the negative obstacle detection feature.| - - - +| ID | Name | Description | Severity Level | Handling Strategy | +| ------ | ------------------------------------------ | -------------------------------------------------------------- | -------------- | ----------------- | +| 105014 | `ERROR_PDSAPP_EXTR_DI_CALIB_IMPLAUSIBLE` | Implausible extrinsic head calibration | Severe | check the camera calibration | +| 105015 | `ERROR_PDSAPP_DEPTH_HINT_OUT_OF_RANGE` | Raised if the algorithm detects an implausible depth hint in the input | Severe | check the given `depthHint` if it is in the FOV | diff --git a/SoftwareInterfaces/ifmDiagnostic/diagnostic_sources.md b/SoftwareInterfaces/ifmDiagnostic/diagnostic_sources.md index 02c7fbf..48460a5 100644 --- a/SoftwareInterfaces/ifmDiagnostic/diagnostic_sources.md +++ b/SoftwareInterfaces/ifmDiagnostic/diagnostic_sources.md @@ -7,8 +7,8 @@ The O3R system provides diagnosis information for different root causes. We prov **WHEN:** -+ This list of error code may be present from start-up of the system. -+ This list of error codes should not persist long-term, that is, the error state has to switch to dormant for the boot-up to be successful. +- This list of error code may be present from start-up of the system. +- This list of error codes should not persist long-term, that is, the error state has to switch to dormant for the boot-up to be successful. **CONTENT** @@ -25,13 +25,13 @@ For camera head specific error codes on boot-up see the table head and its subca **WHEN:** -+ This list of error codes may occur at any time during the runtime of the system. -+ Depending on the severity of these error codes, that is the combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided, see [error escalation strategy](diagnostic_reaction_strategy). +- This list of error codes may occur at any time during the runtime of the system. +- Depending on the severity of these error codes, that is the combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided, see [error escalation strategy](diagnostic_reaction_strategy). **CONTENT:** -+ VPU error codes hold information about undervoltage scenarios at certain voltage levels, VPU over-temperature, NTP sync status, trigger overruns, image buffer overflows, and watchdog timeouts for the imager retrieval processes. -+ COMM error codes hold information about communication faults to the TCU (trigger control unit) and temperature sensors. +- VPU error codes hold information about undervoltage scenarios at certain voltage levels, VPU over-temperature, NTP sync status, trigger overruns, image buffer overflows, and watchdog timeouts for the imager retrieval processes. +- COMM error codes hold information about communication faults to the TCU (trigger control unit) and temperature sensors. ```{include} diagnosis_error_codes/FW1.1/section_vpu.md ``` @@ -40,15 +40,15 @@ For camera head specific error codes on boot-up see the table head and its subca **WHEN:** -+ This list of error codes that may occur at any time during the runtime of the system. -+ Depending on the severity of these error codes, that is the combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided, see [error escalation strategy](diagnostic_reaction_strategy). +- This list of error codes that may occur at any time during the runtime of the system. +- Depending on the severity of these error codes, that is the combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided, see [error escalation strategy](diagnostic_reaction_strategy). **CONTENT:** Port error codes hold information about data stream drops due to: -+ 3D imager timeouts -+ FPD-Link internal communication errors, for example EMV shocks. -+ Internal algorithmic errors while processing the time of flight data. +- 3D imager timeouts +- FPD-Link internal communication errors, for example EMV shocks. +- Internal algorithmic errors while processing the time of flight data. ```{include} diagnosis_error_codes/FW1.1/section_port.md ``` @@ -56,15 +56,19 @@ Port error codes hold information about data stream drops due to: ## Camera Head **WHEN:** -+ This list of error codes may occur at any time during the runtime of the system. -+ Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). +- This list of error codes may occur at any time during the runtime of the system. +- Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). **CONTENT:** -+ Head error codes hold information about overtemperature scenarios, brown outs, etc. detected at the temperature sensor per camera head. -+ Head error codes hold information about overvoltage and undervoltage scenarios at certain voltage levels. -+ Camera shut-off due to eye-safety violations or overtemperature at the VCSEL driver. -+ Camera imager resets, etc. +- Head error codes hold information about overtemperature scenarios, brown outs, etc. detected at the temperature sensor per camera head. +- Head error codes hold information about overvoltage and undervoltage scenarios at certain voltage levels. +- Camera shut-off due to eye-safety violations or overtemperature at the VCSEL driver. +- Camera imager resets, etc. + +:::{note} +- Please contact [support](support.efector.object-ident@ifm.com) if diagnostic codes related to camera heads occur repeatedly. +::: :::{note} - Please contact [support](support.efector.object-ident@ifm.com) if diagnostic codes related to camera heads occur repeatedly. @@ -75,8 +79,8 @@ Port error codes hold information about data stream drops due to: ## IMU **WHEN:** -+ This list of error codes may occur at any time during the runtime of the system. -+ Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). +- This list of error codes may occur at any time during the runtime of the system. +- Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). **CONTENT:** IMU specific errors such as implausible IMU data returned from the sensor. @@ -88,32 +92,43 @@ IMU specific errors such as implausible IMU data returned from the sensor. **WHEN:** -+ This list of error codes may occur at any time during the runtime of the system. -+ Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). +- This list of error codes may occur at any time during the runtime of the system. +- Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). **CONTENT:** -+ Unexpected content encountered inside the (intrinsic) calibration file. -+ No ego-motion data (ODS) was received or ego-motion data is corrupted. +- Unexpected content encountered inside the (intrinsic) calibration file. +- No ego-motion data (ODS) was received or ego-motion data is corrupted. ```{include} diagnosis_error_codes/FW1.1/section_di.md ``` -## ODS app +## ODS application **WHEN:** -+ This list of error codes may occur at any time during the runtime of the ODS application. -+ Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). +- This list of error codes may occur at any time during the runtime of the ODS application. +- Depending on the severity of these error codes: that is combination of error code and number of activations in a certain amount of time, the error escalation strategy has to be decided see [error escalation strategy](diagnostic_reaction_strategy). **CONTENT:** -ODS app error codes hold information about: -+ implausible extrinsic calibration of the camera heads. -+ mismatched timestamp between IMU data and image data. -+ implausible extrinsic calibration, that is default values used for `extrinsic_head_to_user` and implausible extrinsic calibration (camera heads) values. -+ ODS app parameterization plausibility checks, i.e default zones used. -+ insufficient / unstable framerates during runtime. -+ missing stand still (3-5 sec) before starting each ODS run. +ODS application error codes hold information about: +- Implausible extrinsic calibration of the camera heads. +- Mismatched timestamp between IMU data and image data. +- Implausible extrinsic calibration, that is default values used for `extrinsic_head_to_user` and implausible extrinsic calibration (camera heads) values. +- ODS application parameterization plausibility checks, i.e default zones used. +- Insufficient / unstable framerates during runtime. +- Missing stand still (3-5 sec) before starting each ODS run. ```{include} diagnosis_error_codes/FW1.1/section_odsapp.md ``` + +## PDS application +**WHEN:** + +This list of errors may occur at any time during the runtime of the PDS application. + +**CONTENT:** + +The PDS application is a triggered application and may raise the following XMLRPC exceptions: +- ERROR_PDSAPP_EXTR_DI_CALIB_IMPLAUSIBLE: Raised, if the algorithm detects an implausible Distance Image extrinsic calibration. +- ERROR_PDSAPP_DEPTH_HINT_OUT_OF_RANGE: Raised, if the algorithm detects an implausible depth hint in the input. diff --git a/Technology/3D/IntrinsicCalib/intrinsic_calib_3d.md b/Technology/3D/IntrinsicCalib/intrinsic_calib_3d.md index 989a4be..72d47ad 100644 --- a/Technology/3D/IntrinsicCalib/intrinsic_calib_3d.md +++ b/Technology/3D/IntrinsicCalib/intrinsic_calib_3d.md @@ -6,7 +6,7 @@ Every camera is individually calibrated in production. We use two different mode This applies to the wide opening angle (O3R225) cameras. For these cameras, we use the fisheye distortion model as described below. -### For unprojection: intrinisc calibration model +### For unprojection: intrinsic calibration model This model is used for converting pixel positions to 3D vectors. It corresponds to `modelID=2`. The following formulas show how to apply the model: ![Fish eye model for unprojection](resources/fisheye_unprojection.png) @@ -28,7 +28,7 @@ Where ## Bouguet model -### For unprojection: intrinisc calibration model +### For unprojection: intrinsic calibration model This model is used for converting pixel positions to 3D vectors. It corresponds to `modelID=0`. The following formulas show how to apply the model: ![Bouguet model for unprojection](resources/bouguet_unprojection.png) diff --git a/Technology/3D/index_3d.md b/Technology/3D/index_3d.md index 077584d..9b70ff9 100644 --- a/Technology/3D/index_3d.md +++ b/Technology/3D/index_3d.md @@ -14,5 +14,4 @@ ProcessingParams/index_processing_params triggering ImagesDescription/index_images_description IntrinsicCalib/intrinsic_calib_3d - ::: \ No newline at end of file diff --git a/Technology/Hardware/index_hardware.md b/Technology/Hardware/index_hardware.md index dff7312..e90e1af 100644 --- a/Technology/Hardware/index_hardware.md +++ b/Technology/Hardware/index_hardware.md @@ -3,7 +3,7 @@ :::{toctree} :maxdepth: 2 Available hardware -Wiring +Wiring Mounting Maintenance ::: \ No newline at end of file diff --git a/Technology/PortsOverview/ports_overview.md b/Technology/PortsOverview/ports_overview.md index c210475..722c387 100644 --- a/Technology/PortsOverview/ports_overview.md +++ b/Technology/PortsOverview/ports_overview.md @@ -20,7 +20,8 @@ While it is good practice to check the PCIC port directly for the requested hard ## IMU port -Besides the hardware ports mentioned above, there is an additional non-configurable hardware port Port 6 which is specific to the IMU on board the VPU. The PCIC port mapped to Port 6 is 50016 (only used when [configuring ODS](../../ODS/Configuration/configuration.md)). +Besides the hardware ports mentioned above, there is an additional non-configurable hardware port, Port 6, which is specific to the IMU on board the VPU. +The PCIC port mapped to Port 6 is 50016 (only used when [configuring ODS](../../ODS/Configuration/configuration.md)). :::{note} It is not possible to receive any data from the IMU at the moment. diff --git a/Technology/SystemVerification/stress_tests.md b/Technology/SystemVerification/stress_tests.md index 5fc2421..ec32194 100644 --- a/Technology/SystemVerification/stress_tests.md +++ b/Technology/SystemVerification/stress_tests.md @@ -67,7 +67,7 @@ This may either be done on a recorded dataset in the post or on data retrieved i ### 3. Temperature Testing A first set of temperature tests is typically run during a proof of concept phase on a component level, that is the O3R system and its intended (prototype) mounting setup are tested in a smaller temperature chamber. This is a good indication of whether the passive cooling capabilities of the camera and VPU mounting are sufficient for the full ambient temperature range. -Please set all 3D camera heads to their maximum allowed framerate: `20 Hz`. Additionally you can change the "exposureShort" from 400 to 1000. +Please set all 3D camera heads to their maximum allowed framerate: `20 Hz`. Additionally you can change the `exposureShort` from 400 to 1000. Load on the VPUs CPU and GPU can be introduced by running a functional ODS application instance, or by artificially introducing CPU and GPU load via tools such as [stress](https://packages.ubuntu.com/search?keywords=stress) and [gpu-burn](https://github.com/anseeto/jetson-gpu-burn). diff --git a/Technology/VPU/ssh.md b/Technology/VPU/ssh.md index 7a4d1b7..7c21ea3 100644 --- a/Technology/VPU/ssh.md +++ b/Technology/VPU/ssh.md @@ -1,14 +1,14 @@ # SSH usage -To connect to the VPU through SSH or deploy software to the VPU with SCP, it is necessary 1. have an ssh key pair, 2. Have the public key copied to the VPU configuration. +To connect to the VPU through SSH or deploy software to the VPU with SCP, it is necessary 1. have an SSH key pair, 2. Have the public key copied to the VPU configuration. -Follow instructions below to configure ssh on the VPU via either the CLI or via python3 +Follow instructions below to configure SSH on the VPU via either the CLI or via python3 ## Setup via CLI :::{note} -The following instructions are tailored towards a bash (Unix shell). When deploying on a Windows based architecture, please modify the instructions sets for your shell accordingly, or try the setup instructions for python instead +The following instructions are tailored towards a bash (Unix shell). When deploying on a Windows based architecture, please modify the instructions sets for your shell accordingly, or try the setup instructions for Python instead ::: ### Generate an SSH key-pair @@ -89,15 +89,15 @@ $ ifm3d dump | jq .device.network.authorized_keys Note that the `authorized_keys` is a persistent parameter: it does not require a call to [`save_init`](../configuration.md#persistent-settings-without-save_init) to be persistent over reboots. -## Setup via python script +## Setup via Python script -This method will require a python 3 environment with ifm3dpy and paramiko installed via pip +This method will require a Python 3 environment with ifm3dpy and Paramiko installed via pip ``` pip install ifm3dpy paramiko ``` -The python script [ssh_key_gen.py](https://github.com/ifm/ifm3d-examples/blob/main/ovp8xx/python/ovp8xxexamples/core/ssh_key_gen.py) can be used on both windows and linux +The Python script [ssh_key_gen.py](https://github.com/ifm/ifm3d-examples/blob/main/ovp8xx/python/ovp8xxexamples/core/ssh_key_gen.py) can be used on both Windows and Linux Run the script with "--help" for optional arguments @@ -118,7 +118,7 @@ options: --log-file LOG_FILE The file to save relevant output ``` -Running the file will generate the key-pair as specified, if unavailable, copy the public key to the VPU, and run a test command via ssh. +Running the file will generate the key-pair as specified, if unavailable, copy the public key to the VPU, and run a test command via SSH. ```bash 2024-10-30 11:49:25,348 [MainThread ] [INFO ] Connecting to 192.168.0.69 to verify the keys are set correctly. @@ -140,6 +140,6 @@ o3r-vpu-c0:~$ The -i "identity" argument is required on some shells for the private key to be used. -There will be a prompt for the passphrase, if this was configured when running ssh-keygen +There will be a prompt for the passphrase, if this was configured when running `ssh-keygen`. -If successful with the setup, the user will be logged into a shell on the VPU as the oem user. \ No newline at end of file +If successful with the setup, the user will be logged into a shell on the VPU as the OEM user. \ No newline at end of file diff --git a/index.md b/index.md index 251213b..9a3f8e1 100644 --- a/index.md +++ b/index.md @@ -11,6 +11,7 @@ Technology Software Interfaces Calibration Routines ODS +PDS FAQ History ::: @@ -65,7 +66,7 @@ Learn how to quickly setup the Obstacle Detection Solution :class-item: homepage-icons :width: 20 :img-top: ./resources/ifm3d_landing_point-cloud.svg - :link: GettingStarted/ifmVisionAssistant/first_steps_ifmVA + :link: SoftwareInterfaces/iVA/first_steps_ifmVA :link-type: doc :link-alt: Link to the first steps with the Vision Assistant +++ diff --git a/news_fw.md b/news_fw.md index 1f049b1..3f3ba51 100644 --- a/news_fw.md +++ b/news_fw.md @@ -1,2 +1,2 @@ A new firmware version is available for the O3R platform. -Find it here: [FW 1.4.30](https://www.ifm.com/us/en/product/OVP810?tab=documents) \ No newline at end of file +Find it here: [FW 1.10.13](https://www.ifm.com/us/en/product/OVP810?tab=documents) \ No newline at end of file