Page MenuHomeElementl

Load execution plan from the snapshot on the run when resume retrying
ClosedPublic

Authored by dgibson on Jul 23 2021, 4:05 PM.

Details

Summary

Right now when you re-execute a pipeline from failure, we load an execution plan over gRPC not once, not twice, but thrice - once from the parent run to figure out the steps, once from the child run without any steps to validate that the steps are correct, and then a third time with the steps.

This diff takes the first call out of the equation by loading the execution plan directly from the snapshot.

Test Plan

BK, re-execute a pipeline from failure in dagit, verify it works and log to verify that the number of gRPC calls goes from 3 to 2

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 23 2021, 4:28 PM
Harbormaster failed remote builds in B34143: Diff 42204!
dgibson published this revision for review.Jul 23 2021, 5:17 PM

coolbeans

python_modules/dagster/dagster/core/execution/plan/resume_retry.py
46

nit: a dagster exception maybe

This revision is now accepted and ready to land.Jul 23 2021, 6:52 PM
This revision was landed with ongoing or failed builds.Jul 23 2021, 7:15 PM
This revision was automatically updated to reflect the committed changes.