Page MenuHomePhabricator

Don't load pyspark when importing dagster_aws.emr
AbandonedPublic

Authored by sandyryza on Oct 9 2020, 5:52 PM.

Details

Summary

This is a breaking change for anyone relying on the emr_pyspark_step_launcher. Now they need to do
from dagster_aws.emr.pyspark import emr_pyspark_step_launcher instead of
from dagster_aws.emr import emr_pyspark_step_launcher.

Test Plan

bk

Diff Detail

Repository
R1 dagster
Branch
pyspark_step_launcher (branched from master)
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 9 2020, 6:10 PM
Harbormaster failed remote builds in B19373: Diff 23543!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 9 2020, 6:33 PM
Harbormaster failed remote builds in B19374: Diff 23544!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 9 2020, 7:12 PM
Harbormaster failed remote builds in B19376: Diff 23547!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 9 2020, 9:07 PM
Harbormaster failed remote builds in B19384: Diff 23556!
schrockn added a subscriber: schrockn.

As long we are doing a proper breaking change, why not move into its own module and actually model the dependencies properly?

This revision now requires changes to proceed.Oct 13 2020, 3:12 PM