Ability to identify conflicts in backed-up resources

[GowthamShanmugam]

Describe the problem/challenge you have

As a disaster recovery team, it is crucial to back up application workload resources using label selectors. Our scenario involves deploying multiple application workloads within the same namespace. The main challenge arises from the risk of resource conflicts due to overlapping labels, where a single resource might be selected by multiple application label selectors for backup.

This behavior needs to be avoided because, during relocation of an application workload, the original resource is removed from the primary cluster. If other application workloads depend on the same resource, their functionality could be disrupted, potentially leading to crashes.

For instance:

ConfigMap: C1(key1: V1), C2(key1: V1, key2: V2)
Creating backup1 with label key1 = V1
Creating backup2 with label key2 = V2
In this case, C2 is selected by both backups, leading to a conflict.
We must implement a mechanism to detect such conflicts and alert users accordingly.

Note:
We are using --include-resources, exclude is too much when there are more workloads under the same namespace.

Describe the solution you'd like

1. Is it possible to provide any Velero API to provide information about the conflicts?
2. (Or) A new API that accepts the backup name as input and returns a list of resource types and resource names in the response would be highly beneficial.
3. (Or) Is possible to access the Velero backup using any API? so we can access the collected resources and check for the conflicts from our plugin?

Or any other suggested way?

Anything else you would like to add:

Environment:

oadp-velero-rhel 1.4.1

[blackpiglet]

Could you give more information about why resources selected by multiple label selectors are an issue?
If a k8s resource is selected by multiple labels, IMO, it's included in multiple backups in your scenario, and it can be restored by all of the backups.
It's not an obvious conflict scenario in data restores per my understanding.

[kaovilai]

It was said that the conflicting resources have large volumes which is not ideal. I think the user wants ability to define two different backups where

• backupA covers labelA
• backupB covers labelB and excluding any labelB items that also has labelA.

[kaovilai]

I had suggested that a velero plugin could be used perhaps where it would

• detect during Execute if an item was already in backupA
• call WithoutBackup() to skip including this item from backupB.
  • note that WithoutBackup function isn't yet in a BackupItemAction plugin, but should be similar to existing WithoutRestore for RestoreItemAction.

[sseago]

@kaovilai "detect during Execute if an item was already in backupA"
That would be pretty complicated. The plugin would need to download the resource list from backupA and parse it -- it would also mean that reordering backupA and backupB would change what was in each.

[GowthamShanmugam]

if BackupA is taking a backup of ResourceA, ResourceA is labeled as being owned by BackupA? This way, if BackupB attempts to back up ResourceA, it will skip the task because the resource is already marked as owned by BackupA.

[kaovilai]

IIUC velero label resources into backup stored at storage. so you will only see label an item came from which backup at restore time.

[kaovilai]

The simplest workaround in this example

ConfigMap: C1(key1: V1), C2(key1: V1, key2: V2)

is add labels to both c1 and c2 which backup they would belong. so C1(key1: V1, key2: backup1), C2(key1: V1, key2: V2, key3: backup2)

otherwise making velero label things as they are backed up you would end up with additional label/annotation anyway, just added by velero rather than user control.