OTEP: Logger.Enabled

[pellared]

The purpose of this OTEP is to have an agreement that the SDK needs both:

• a "hook" for Logger.Enabled for customization and flexibility e.g. via extending LogRecordProcessor with Enabled method, Add Enabled to LogRecordProcessor #4363
• new LoggerConfig fields to conveniently address the most popular use cases, especially configuring the minimum severity level, Add min_severity to LoggerConfig #4364

The OTEP tries to elaborate how/why it should work well with (trace) sampling as well as Events API.

Fixes #4207

Side note: I can create issues for each "open question" for tracking purposes.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[cijothomas]

One thing I am not so sure about.. Do we expect OTel SDK to provide such control, mimicking what is often provided by the Logging libraries themselves? They often provide sophisticated mechanisms, and it may not be worth replicating everything in OTel...
We can explicitly state that our desire is to provide some control, but not necessarily matching/competing with existing logging libraries?

[pellared]

I do not expect the Logs API to be as user-friendly as the logging libraries popular for given language.

However, I think that the SDK (the logging backend) should be able handle different logging processing scenarios. E.g. sending verbose logs via user_events and warning logs via OTLP (probably I could come up with better examples). I heard that someone wanted to send non-event log records and event records to different destinations. Moreover, there is the following question in the Events Basics OTEP:

How to support routing logs from the Logs API to a language-specific logging library while simultaneously routing logs from the language-specific logging library to an OpenTelemetry Logging Exporter?

I see logging libraries as logs frontend and Logs SDK as logs backend. Logs API is the thing which bridges between the frontend and backend.

Side note: I think that even without this use case we would get the same design.

[pellared]

I moved it as the last use-case given that it may be seen as the least important. a4257ef

[pellared]

Let me know if the current way it is described looks good overall.

[pellared]

@cijothomas, PTAL

I want to also add that some users may want to directly use Logs API without using any bridged library. See: #4352

[lmolkova]

Left a few minor comments, LGTM otherwise!

[pellared]

During the Logs SIG meeting we agreed that LogRecordFilterer should be the "main" proposal (it is more loosely coupled, processors are not "chained" by design) and move the existing that adds Enabled to LogRecordProcessor as an alternative. It is also something which most people are familiar with.

EDIT: After a demo of the OTel Go Logs design it looks like an acceptable solution, but the OTEP should call out explicitly the optimizations we have done. E.g. that we are not looping through processors when none of them implements Enabled and that we are only looping thrrough the processors which implement enabled.

// Enabled returns true if at least one Processor held by the LoggerProvider
// that created the logger will process param for the provided context and param.
//
// If it is not possible to definitively determine the param will be
// processed, true will be returned by default. A value of false will only be
// returned if it can be positively verified that no Processor will process.
func (l *logger) Enabled(ctx context.Context, param log.EnabledParameters) bool {
	// If there are more Processors than FilterProcessors we cannot be sure
	// that all Processors will drop the record. Therefore, return true.
	//
	// If all Processors are FilterProcessors, check if any is enabled.
	return len(l.provider.processors) > len(l.provider.fltrs) || anyEnabled(ctx, param, l.provider.fltrs)
}

I also plan creating a PR for #4364 as this does not seem controversial. This should simplify this OTEP (assuming the PR will be merged).

[MrAlias]

Ideally, this is configurable on a per-processor level, not just SDK wide, so you can have alternate destinations for different log severity.

[pellared]

I agree. I will add this use case. Also given later I already described:

MinimumSeverityLevelProcessor is for configuring log processing pipelines.
It is the only choice when one would like to set the minimum severity level
for a certain exporting pipeline.
For example, one batching processor would be exporting logs only above info level
via OTLP and a second simple processor would be exporting all logs to stdout.

[MrAlias]

These number references are confusing as the seem to be referencing something from another section.

[MrAlias]

This seems to be doing too much. Why not just focus on adding logger enabled like the title indicates?