Skip to content

Readable._read is called multiple times synchronously #24919

@Rantanen

Description

@Rantanen
  • Version: v10.10.0, v11.4.0
  • Platform: Linux 4.4.0-45-generic deprecate domains #66-Ubuntu SMP Wed Oct 19 14:12:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • Subsystem: stream

The documentation for readable._read(size) states the following:

readable._read() is guaranteed to be called only once within a synchronous execution, i.e. a microtick.

However the following example demonstrates starvation if that method is able to push all of its data synchronously. The stream implementation ends up calling the read-function repeatedly and won't yield execution as documented.

var Readable = require('stream').Readable;

let c = 10;
let r = new Readable({
    read: () => {
        process.nextTick(console.log, 'next-tick');
        r.push('a');
    }
});

r.on('data', () => {});
process.nextTick(console.log, 'Got food');  // Never printed.
console.log('Requested food');

I believe the issue is caused by the flow function calling stream.read repeatedly within a while-loop until the stream fails to return data:

function flow(stream) {
const state = stream._readableState;
debug('flow', state.flowing);
while (state.flowing && stream.read() !== null);
}

One option for fixing this might be to make flow recurse with next tick:

function flow(stream) {
  const state = stream._readableState;
  debug('flow', state.flowing);
  flow_(stream, state);
}

function flow_(stream, state) {
  if(state.flowing && stream.read() !== null)
    process.nextTick(flow_, stream, state);
}

Alternatively just updating the documentation might also work. I guess most streams that would encounter the issue would just delay r.push() into the next tick as a workaround.

Even in my case this was just something I spotted while looking into #24918. I don't personally have a use case for such a stream.

Metadata

Metadata

Assignees

No one assigned

    Labels

    streamIssues and PRs related to the stream subsystem.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions