Page MenuHomeElementl

[1/2] Fix re-execution if execution plan is delegated
ClosedPublic

Authored by rexledesma on Apr 8 2021, 5:34 PM.

Details

Summary

A simpler version of https://dagster.phacility.com/D7348

Rather than looking into the execution plan if the step key has/will be run, we instead look into
the pipeline run's step keys to execute.

Fixes the case if a pipeline (A -> B[?]), or (A -> B) fails and tries to reexecute, in 2/2 will fix
if (A -> B[?] -> C[?]) fails

Test Plan

pytest

Diff Detail

Repository
R1 dagster
Branch
rl/fix-multiprocess-load-input-execution-take-2 (branched from master)
Lint
Lint Passed
Unit
No Test Coverage

Event Timeline

rexledesma added inline comments.
python_modules/dagster/dagster/core/execution/context/system.py
573

probably should live under PipelineRun.executes_step_key, but can refactor in [2/2]

This revision is now accepted and ready to land.Apr 8 2021, 5:55 PM

comment

python_modules/dagster/dagster_tests/core_tests/execution_plan_tests/test_execution_plan_reexecution.py
109 ↗(On Diff #35096)

@sandyryza @alangenfeld would like to get eyes on this just to sanity check myself

python_modules/dagster/dagster_tests/core_tests/execution_plan_tests/test_execution_plan_reexecution.py
109 ↗(On Diff #35096)

looking at * above seemslike we want add_two to execute

93 ↗(On Diff #35098)

*

alangenfeld added inline comments.
python_modules/dagster/dagster_tests/core_tests/execution_plan_tests/test_execution_plan_reexecution.py
90 ↗(On Diff #35116)

yeahfuckyeah