HomePhabricator

switch spark compute to pipe to stdout/stderr instead of structured logs

Authored by prha on Tue, Aug 27, 11:10 PM.

Description

switch spark compute to pipe to stdout/stderr instead of structured logs

Summary:
Now that we capture the compute IO, we should switch the spark solid
to dumping IO to stdout/stderr instead of using the python logging system

Depends on D897, D895, D893

Test Plan: Ran the event_ingest_pipeline

Reviewers: #ft, alangenfeld

Reviewed By: #ft, alangenfeld

Subscribers: max, schrockn, alangenfeld

Differential Revision: https://dagster.phacility.com/D898

Details