HomePhabricator

switch posix mirroring implementation from dagster cli to bare unix tail

Authored by prha on Wed, Nov 13, 4:52 PM.

Description

switch posix mirroring implementation from dagster cli to bare unix tail

Summary:
There is a race condition between the tail process and the compute function execution.
Switching to use the dagster cli machinery (in order to have the same execution path for both
posix and windows) slowed down the tail process enough that a lot of quickly executing solids
were completing and killing the tail process before their output was mirrored.

The threshold on my machine to consistently mirror output was a couple hundred milliseconds.

This diff bypasses the python cli machinery in favor of the bare UNIX tail process. This is
the approach used in 0.6.1, and that we moved away from in https://dagster.phacility.com/D1258.

We maintain the polling behavior for windows, incurring a latency cost

Tracking issue for replacing this with a less-bespoke solution for process management: https://github.com/dagster-io/dagster/issues/1906

Test Plan:
Checked dagit and dagster CLI. Added a cli test making sure that simple logging
solids have their output captured

Reviewers: #ft, themissinghlink

Reviewed By: #ft, themissinghlink

Subscribers: alangenfeld, themissinghlink

Differential Revision: https://dagster.phacility.com/D1411

Details

Committed
prhaWed, Nov 13, 7:06 PM
Reviewer
Restricted Project
Differential Revision
D1411: switch posix mirroring implementation from dagster cli to bare unix tail
Parents
R1:e42caa59efea: pin google-cloud-storage until they fix deps
Branches
Unknown
Tags
Unknown