Page MenuHomePhabricator

[core] StepHandle
ClosedPublic

Authored by alangenfeld on Dec 8 2020, 11:32 PM.

Details

Summary

Born from the ashes of D5360, extracted from the prototype D4259 comes StepHandle

This new object for interacting with steps in an ExecutionPlan sets us up to be able to handle more complex "key" structures which is needed for dynamic orchestration.

In addition to introducing this handle for keying in to step_dict, this diff attempts to clean up ExecutionPlan and ExecutionStep a bit.

Test Plan

buildkite
integration

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

alangenfeld edited the test plan for this revision. (Show Details)

test fixees

Macro looksgood:

python_modules/dagster/dagster/core/execution/plan/handle.py
9–14

This is docblock worthy given that it is a bit odd that it only container a solid_handle atm

16

remember to rebase :-)

21

can we do anything to future proof this for non-solid-compute steps in the future?

This revision is now accepted and ready to land.Dec 10 2020, 4:43 PM
python_modules/dagster/dagster/core/execution/plan/handle.py
9–14

might just catch this in the upcoming diff that fleshes this out

21

as long as we are smart about how we encode them i think its fine - ie having a special delimiter or use a reserved keyword

This revision was automatically updated to reflect the committed changes.