Page MenuHomePhabricator

close multiprocess executor event queue in parent process rather than child process
ClosedPublic

Authored by dgibson on Mon, Nov 16, 2:26 AM.

Details

Summary

Some sporadic failures during multiprocess execution tests point to an issue with the queue still writing events after it has been closed. Try closing the queue in the process that created it (which is probably better anyways?) and wait to close it until after the child process has been joined.

Test Plan

BK + Azure (no more BrokenPipeError in windows test output)

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.