Page MenuHomeElementl

retry from failure fan-in bug
ClosedPublic

Authored by yuhan on Jun 30 2021, 4:52 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Mar 27, 7:23 PM
Unknown Object (File)
Mon, Mar 27, 7:23 PM
Unknown Object (File)
Sun, Mar 19, 5:58 PM
Unknown Object (File)
Tue, Mar 14, 10:20 PM
Unknown Object (File)
Tue, Mar 14, 6:21 PM
Unknown Object (File)
Fri, Mar 10, 1:53 PM
Unknown Object (File)
Tue, Mar 7, 12:45 AM
Unknown Object (File)
Mon, Mar 6, 3:42 AM
Subscribers
None

Details

Summary

https://github.com/dagster-io/dagster/issues/4325

the bug was that FromMultipleSources didn't check logs of transitive parent runs. this diff moves the similar logics into one place, i.e. StepExecutionContext._get_source_run_id_from_logs

I think the ideal fix is to record that kind of info (e.g. source run id) in places like KnownExecutionState, but this isn't a user-facing improvement, so will explore that after the release. tracking: https://github.com/dagster-io/dagster/issues/3511

Test Plan

added test

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

python_modules/dagster/dagster/core/execution/plan/inputs.py
441–442

wellthereitis

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 30 2021, 5:17 PM
Harbormaster failed remote builds in B32903: Diff 40553!
yuhan edited reviewers, added: alangenfeld; removed: yuhan.
This revision is now accepted and ready to land.Jul 7 2021, 6:28 PM
This revision was automatically updated to reflect the committed changes.