You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently the processor uses the entire log message as the deduplication key, allowing the user to remove certain fields from being considered during the hashing part via the exclude_fields option.
One scenario that is not covered is that the deduplication key could be already present as a body or attribute in the log message. You could exclude the other body and attributes via exclude_fields, but this negative match is very error prone and does not scale well.
Describe the solution you'd like
I propose we add a new option to use a certain LogRecord field as deduplication key. This option accepts ottl style path variables:
processors:
logdedup:
key: body["my_id_field"]
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
@djaglowski Do you have any resources on how to evaluate the OTTL string from the field? I'm struggling to follow up on this part. For example, I have a LogRecord and an OTTL string like body["my_field"] and I need to extract the actual value from the LogRecord. I’d appreciate any help!
Component(s)
processor/logdedup
Is your feature request related to a problem? Please describe.
Currently the processor uses the entire log message as the deduplication key, allowing the user to remove certain fields from being considered during the hashing part via the
exclude_fields
option.One scenario that is not covered is that the deduplication key could be already present as a body or attribute in the log message. You could exclude the other body and attributes via exclude_fields, but this negative match is very error prone and does not scale well.
Describe the solution you'd like
I propose we add a new option to use a certain LogRecord field as deduplication key. This option accepts ottl style path variables:
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: