Details
Assignee
UnassignedUnassignedReporter
Alex MillerAlex MillerApproval
ScreenedPriority
Critical
Details
Details
Assignee
Unassigned
UnassignedReporter
Alex Miller
Alex MillerApproval
Screened
Priority
Created May 17, 2018 at 5:26 PM
Updated November 22, 2022 at 4:22 AM
The docs for
pipeline
say "Will stop consuming thefrom
channel if theto
channel closes." however the following code will print 0..49 (and would continue consumingfrom
forever).Cause: Currently, when the output go loop puts to the
to
channel and gets a closed result indicator (logical false), it only stops the output go block, but lets the ingest and compute go loops continue, so the pipeline keeps consuming thefrom
channel and computing values it can't use in the output. In other words, it does not do what it says it will do.Alternatives:
Change the docstring to not promise to stop consuming
from
(this seems wrong)When
to
is closed, signal the ingest and compute go blocks to stop, either via state (atom), or a control channelPush the signal back through the compute and ingest blocks somehow, maybe by closing the jobs and results channels and watching for that?
Proposed:
Create an atom indicating ingest processing should stop.
In the output loop, if the
to
channel is found to be closed, reset the stop atom to true.In the ingest loop, check the stop atom - if set, stop ingesting and close the jobs chan (as we already do when the
from
channel is closed)Note that some work may be in flight (1 may be in-process in the output go block, N may be in the results channel, and N may be parked in flight in process go blocks)
Patch: async-217-2.patch (same change in clj and cljs)
The example above usually prints 0…2 with the patch.
Screened by: Ghadi