Handling unconnected AudioWorkletNode's output

[hoch]

Currently the spec says: https://webaudio.github.io/web-audio-api/#audioworkletprocess-callback-parameters

If there are no actively processing AudioNodes connected to the 𝑛
th input of the AudioWorkletNode for the current render quantum, then the content of inputs[n] is an empty array, indicating that zero channels of input are available. This is the only circumstance under which the number of elements of inputs[n] can be zero.

However, the spec text doesn't cover the case of unconnected output. I believe we can apply the same principal here - the output with no outgoing connection will return zero length array.

[hoch]

Until most recently, the implementation of both Chromium and FireFox did provide an empty array (zero channels) for the unconnected output. We might want to add the spec text to explicitly say what the current implementation does.

[hoch]

@padenot I just verify that FireFox actually returns an array of 128 elements for any outputs regardless of its connection status.

Repro:
https://codepen.io/fr0stbyter/pen/xxBaOxj?editors=1010

I thought we're supposed to return an empty array, not an array of 128 zero values. We need an agreement on how to handle this.

[greggman]

IMO you should not return an empty array. It makes it so that code that doesn't check, breaks. Returning an array makes this most robust.

To put it another way, it seems better to have an API that can't fail than one that can. Or, it seems better to have an API that does not require a check than one that does.

[hoch]

@greggman So you prefer having an array of 128 zeros for unconnected outputs?

[greggman]

I'm not an expert in this API. I only know a recent change in Chrome broke a site I work on.

It doesn't make any sense to me to call the process function with no arrays. The entire point is to use the data in the arrays or fill them with data. If there are no arrays to use/fill then maybe there should be no function call?

[greggman]

@mjwilson-google

[hoch]

It doesn't make any sense to me to call the process function with no arrays. The entire point is to use the data in the arrays or fill them with data. If there are no arrays to use/fill then maybe there should be no function call?

In this case, no array signals the connection status. This is useful because the user code can use this signal to skip the processing code entirely.

Keep in mind that an AudioWorkletNode can have multiple outputs. output_1 can be connected while output_2 is unconnected (or disconnected). In this case, the implementation should still invoke process() callback. In the callback, the first output array will be an empty one with 128 slots, but the other array will be null in the current stable Chrome.

Now I am revisiting this issue again and trying to find old discussion threads - if there's no clear specification on the output array behavior, I think we need to have that discussion again.

[mjwilson-google]

This is a bit of a philosophical question with no right answer when designing an audio API. Where should the samples "live", is the mental model of the nodes pure software constructs where having disappearing outputs might make sense versus modeling a physical system, etc.

That said, since Web Audio is a mature API I think we should prioritize not breaking existing usage. @greggman, as far as Chromium goes we have a patch coming that should fix the site breakage. What I have seen before in Web Audio is we tend to codify existing behavior in the spec even if it's not ideal. I think we should discuss this further in the working group.

[padenot]

We can't change the behavior here now, and we need to encode the historical behaviour in the spec unfortunately, as @mjwilson-google says. If we feel we want to signal to authors that the output isn't connected we can do so differently.

[hoch]

Yes. I agree. Glad we found this now and let's work on describing the current behavior into the spec.

[hoch]

Teleconference 3/7:

• Perhaps we can add a non-normative text on how to handle unconnected AudioWorklet's output. Using an OscillatorNode as a LFO of another OscillatorNode is a valid use case, so the AudioWorkletNode should be able to support the case.

[hoch]

TPAC 2024:

Let's clarify the unconnected output array's behavior - something similar to "it returns an array of 128 elements for any outputs regardless of its connection status."