Add processing metrics to identify potential lag

[vietk]

Hello

In the case of using reactive messaging application to implement streaming application à la kafkastreams, we are missing some metrics that cannot (or hardly) be deduced by the raw consumer/producer metrics already present.

When a topic/partition start to lag we need to identify if it's because

• more messages are coming in (causes an input lag)
• or the reactive messaging application latency start to increase (because performance issues).

The idea would be to replicate the metrics already present in kafkastreams for the streams thread such as :

• process-rate: The average number of processed operations per second across all tasks.
• process-latency-avg: The average execution time in ms, for the respective across all running tasks of

Or at node/task level (the equivalent of @incoming+@ougtoing method) level :

• record-e2e-latency-avg: The average end-to-end latency of a record, measured by comparing the record timestamp with the system time when it has been fully processed by the node.

Moreover, I think these metrics can conceptually have meaning for other queuing system and could be implemented for all connectors type.

Waiting for your inputs

Regards