Instrument interceptors with OpenCensus

[bogdandrutu]

Trace

For trace, we start a Span every time we start a new batch (will end when we call the ReceiveSpans). Because the RPC is a bidirectional and can live long we need to decouple this span from the RPC span, but because they are related we should add the RPC span as parent link (here is a good example where links as useful).

Stats

Measure:

• oc.io/interceptor/received_spans - long value

Tags:

• opencensus_interceptor

Measurement:
For oc.io/interceptor/received_spans we record a measurement every time we receive a new message from the client even if the number is 0.

Views:
We should install a distribution view with buckets {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 16, 18, 20, 25, 30, 35, 40, 45, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 450, 500, 600, 700, 800, 900, 1000, 1200, 1400, 1600, 1800, 2000}

[bogdandrutu]

Please also install the ocgrpc plugin for the interceptor gRPC server.

[odeke-em]

Great suggestions @bogdandrutu! Also I think in the future(after we've finished rolling out this instrumentation), would it make sense for us to extract information from the node? For example:

• process_identifier --> "opencensus_task" for stats and perhaps service name for metrics?

[odeke-em]

Same thoughts, I actually have a TODO in the code as per

opencensus-service/interceptor/opencensus/trace_interceptor.go

Lines 89 to 91 in d91241e

	// TODO: (@odeke-em) in the future, also add OpenCensus
	// stats handlers to start this gPRC server.
	srv := grpc.NewServer()

but I think the even better way would be to create a helper method say in internal that creates a gRPC server for us since every interceptor that needs it will have to remember to install the ocgrpc plugin which could become error-prone.

[bogdandrutu]

@odeke-em just having a todo to also add tags from the Node is fine for the moment.

[odeke-em]

@odeke-em just having a todo to also add tags from the Node is fine for the moment.

@bogdandrutu perfect. I've done that!

Because the RPC is a bidirectional and can live long we need to decouple this span from the RPC span, but because they are related we should add the RPC span as parent link (here is a good example where links as useful).

EDIT: I'll take a look at this right now.

[odeke-em]

Alright, so with PRs:

PR	Functionality
#65	Metrics for the interceptor
#76	Tracing the OpenCensus interceptor

I believe this issue is solved. However, we'll need to keep this information and ideas in when we implement other interceptors. PTAL and feel free to reopen.

[bogdandrutu]

Maybe add a md file in the interceptor about how to instrument any interceptor.

[odeke-em]

In deed, great suggestion @bogdandrutu, I filed #79.