Page MenuHomePhabricator

step execution cli command
ClosedPublic

Authored by johann on Jul 13 2020, 10:22 PM.

Details

Summary

new execute_step_with_structured_logs command to replace graphql executePlan

Test Plan

unit - execute_step command and integration for run launcher, coordinator, + step jobs

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 13 2020, 10:34 PM
Harbormaster failed remote builds in B15250: Diff 18671!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 14 2020, 6:50 PM
Harbormaster failed remote builds in B15335: Diff 18770!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 17 2020, 7:44 PM
Harbormaster failed remote builds in B15587: Diff 19071!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 20 2020, 8:20 PM
Harbormaster failed remote builds in B15626: Diff 19115!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 21 2020, 3:08 PM
Harbormaster failed remote builds in B15655: Diff 19152!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 21 2020, 3:35 PM
Harbormaster failed remote builds in B15658: Diff 19156!

fix retries serialization

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 21 2020, 4:30 PM
Harbormaster failed remote builds in B15669: Diff 19169!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 21 2020, 5:58 PM
Harbormaster failed remote builds in B15675: Diff 19177!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 28 2020, 10:00 PM
Harbormaster failed remote builds in B16115: Diff 19688!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 28 2020, 10:51 PM
Harbormaster failed remote builds in B16122: Diff 19695!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 28 2020, 11:05 PM
Harbormaster failed remote builds in B16125: Diff 19698!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 28 2020, 11:53 PM
Harbormaster failed remote builds in B16126: Diff 19699!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 29 2020, 2:39 AM
Harbormaster failed remote builds in B16130: Diff 19703!
Harbormaster returned this revision to the author for changes because remote builds failed.Jul 29 2020, 3:05 AM
Harbormaster failed remote builds in B16131: Diff 19704!

add signal remap to step job

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 29 2020, 2:00 PM
Harbormaster failed remote builds in B16136: Diff 19709!
johann edited the test plan for this revision. (Show Details)
johann added reviewers: alangenfeld, catherinewu.

add graphql replacement to raw k8s run launcher

alangenfeld added inline comments.
python_modules/libraries/dagster-celery-k8s/dagster_celery_k8s/executor.py
239

nit: this name isnt accurate - its a dict if we are using pack/unpack value

270

might be better to throw a more formal DagsterInvariantViolation or something that clearly calls out un-expected origin type since we are going to be introducing those new types soon here

322

I'm not sure about this todo - we'll just execute in whatever container we are launching

This revision is now accepted and ready to land.Jul 29 2020, 3:59 PM
johann marked 2 inline comments as done.

diff review

python_modules/libraries/dagster-celery-k8s/dagster_celery_k8s/executor.py
270

might be better to throw a more formal DagsterInvariantViolation or something that clearly calls out un-expected origin type since we are going to be introducing those new types soon here

270

Good catch, I think that can actually just be relaxed to PipelineOrigin

This revision was landed with ongoing or failed builds.Jul 31 2020, 3:52 PM
This revision was automatically updated to reflect the committed changes.