Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply double check pattern to avoid race condition in LoggingService #11284

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

YuliiaKovalova
Copy link
Member

@YuliiaKovalova YuliiaKovalova commented Jan 14, 2025

Fixes: https://dev.azure.com/devdiv/DevDiv/_workitems/edit/2320135/

Context

Reset() method invocation sometimes causes exception System.IO.IOException: 'The handle is invalid.'.
Very likely it happens on application shutdown.
The assumption is:

  • ShutdownComponent starts and gets the _lockObject
  • It cleans up/disposes the events
  • Meanwhile our logging thread is trying to use those same events without checking _lockObject in context of StartLoggingEventProcessing()

Changes Made

To resolve this issue, the double-check pattern was implemented for the event fields.
This double-check approach prevents the 'invalid handle' exception from occurring during the race condition between component shutdown and the execution of the logging thread.

Testing

N/A

@YuliiaKovalova YuliiaKovalova marked this pull request as ready for review January 15, 2025 10:27
Copy link
Member

@rainersigwald rainersigwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I understand how this helps. Isn't there still a race between thread 1 seeing that it's non-null and calling into it and thread 2 calling Dispose()?

@YuliiaKovalova YuliiaKovalova marked this pull request as draft January 24, 2025 09:36
@YuliiaKovalova YuliiaKovalova force-pushed the dev/ykovalova/better_handle_event_reset branch from 95adccd to 3611f9a Compare January 24, 2025 10:08
@YuliiaKovalova YuliiaKovalova force-pushed the dev/ykovalova/better_handle_event_reset branch from 50d0c8c to 74a1a74 Compare January 27, 2025 09:18

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Copy link
Member

@JanKrivanek JanKrivanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss this one

_emptyQueueEvent.Set();
_emptyQueueEvent?.Set();
}
catch (Exception)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try to avoid this - we'd have no indications of unexpected behavior from labs or watsons

// Long-lived event handles that never get disposed to avoid race conditions.
private readonly AutoResetEvent _longLivedDequeueEvent = new AutoResetEvent(false);
private readonly ManualResetEvent _longLivedEmptyQueueEvent = new ManualResetEvent(true);
private readonly AutoResetEvent _longLivedEnqueueEvent = new AutoResetEvent(false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are those needed?

{
_emptyQueueEvent.Set();
// Check if instance fields are nulled (cleanup was called)
if (_eventQueue == null || _dequeueEvent == null || _emptyQueueEvent == null || _enqueueEvent == null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These can still get nullified right after checking.
We should store them in local variables and check and use those here. Then nullyfying in the close method could be kept

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants