Ohhh damn - I did have this some time ago.

The reason was something here (or better: this code is the fix - as some tasks where null):
var tasks = workerTasks.Values.WhereNotNull();

While cancelling (aka in dispose) a lot of things are happening async - so it is not as trivial to reproduce (especially not with an attached debugger).
Originally I saw that some tasks in our collection are null probably because they can't cancelled while they were still in the queue.

I think this is similar. There might be a still running "something" that waits for something (while in dispose) and tries to access a disposed resource after. Basically, we need a few more safeguards after we awaited some process. I try to have a look on Friday.

Sadly I can't seem to repro it locally. Fails the build server around once every ten runs.

Well - your hardware is just too good :D

Hello,

I'm evaluating NCronJob and suffer from this behavior as well. Same kind of issue. Transient and CI only

By any chance - is there some kind of setup you can share? For example:

• NCronJob runs via an WebApplicationFactory
• There is a job that runs way longer and doesn't get passed in the CancellationToken
• ...

There is a job that runs way longer and doesn't get passed in the CancellationToken

@linkdotnet Hmmm. Could you please elaborate on that scenario?

• App is launched
• Long cron job is started (but doesn't properly deal with the cancellation token)
• App is disposed before the job is finished

Could that trigger a double dispose call at the queue worker level?

Yeah something like that - I tried to reproduce this with a small test - but no luck so far.

Could that trigger a double dispose call at the queue worker level?

Well, we do have disposables that call other disposables. There might be room for chaos.

Given the source of your code @nulltoken: QueueWorker:

public override async Task StopAsync(CancellationToken cancellationToken)
{
    if (shutdown != null)
    {
        await shutdown.CancelAsync();
    }

    ...

If shutdown is already disposed, we will run into that issue. The question is why is it already disposed?
The "easy" fix is:

public override async Task StopAsync(CancellationToken cancellationToken)
{
    if (shutdown is not null && !isDisposed)
    {
        await shutdown.CancelAsync();
    }
    ...

Maybe we could push a preview with the given fix above and you (@nulltoken) and @kikisaeba can check?

Also JobWorker seems to suffer from such an issue:

This can happen if the Queue manager has been disposed and we waited too long and go into the finally block

Maybe we could push a preview with the given fix above and you (@nulltoken) and @kikisaeba can check?

I'd be all in for this

Also JobWorker seems to suffer from such an issue:

Oh! Interesting.

• There is a job that runs way longer and doesn't get passed in the CancellationToken

On my side, I've double checked. All tokens are propagated to an eventual HttpClient SendAsync() method.

Hupps - that should have not been closed. In any case, there will be a preview version out in a second.
Maybe you can have a look at this and report back if it still occurs.

Maybe you can have a look at this and report back if it still occurs.

Will do

https://www.nuget.org/packages/NCronJob/4.3.0-preview

https://www.nuget.org/packages/NCronJob/4.3.0-preview

Woot! I've run four of five builds against a branch that had a higher ratio of failure (1 every 3 builds on average) and no issue.
That looks promising!

However, I suggest that we wait until tomorrow before releasing a new stable version

• I'll run a more intensive test session
• Maybe @kikisaeba will give it a test and report back by then

@linkdotnet Great job!

BTW, you forgot to attribute the fix to its rightful author in the changelog.

You deserve all the possible kudos <3

https://www.nuget.org/packages/NCronJob/4.3.0-preview

Woot! I've run four of five builds against a branch that had a higher ratio of failure (1 every 3 builds on average) and no issue. That looks promising!

However, I suggest that we wait until tomorrow before releasing a new stable version

• I'll run a more intensive test session
• Maybe @kikisaeba will give it a test and report back by then

Yeah - I also will wait a bit longer. And will also check tomorrow morning for some more spots that might be impacted.

BTW, you forgot to attribute the fix to its rightful author in the changelog.
You deserve all the possible kudos <3

Thanks <3 - will add this once we are certain it fixes the issue. In any case, a new release might be nice given all your hard work and features should be shipped to the public.

I'll ran a four additional builds, seems stable on my side.
Will provide more feedback tomorrow.