Page MenuHomeElementl

retry from failure fan-in bug
ClosedPublic

Authored by yuhan on Jun 30 2021, 4:52 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Jun 26, 4:49 PM
Unknown Object (File)
Sun, Jun 26, 7:00 AM
Unknown Object (File)
Fri, Jun 24, 10:06 AM
Unknown Object (File)
Tue, Jun 21, 3:52 PM
Unknown Object (File)
Mon, Jun 20, 8:32 AM
Unknown Object (File)
Sat, Jun 18, 1:19 AM
Unknown Object (File)
Fri, Jun 17, 5:47 PM
Unknown Object (File)
Thu, Jun 16, 10:32 AM
Subscribers
None

Details

Summary

https://github.com/dagster-io/dagster/issues/4325

the bug was that FromMultipleSources didn't check logs of transitive parent runs. this diff moves the similar logics into one place, i.e. StepExecutionContext._get_source_run_id_from_logs

I think the ideal fix is to record that kind of info (e.g. source run id) in places like KnownExecutionState, but this isn't a user-facing improvement, so will explore that after the release. tracking: https://github.com/dagster-io/dagster/issues/3511

Test Plan

added test

Diff Detail

Repository
R1 dagster
Branch
arcpatch-D8619
Lint
Lint Passed
Unit
No Test Coverage

Event Timeline

python_modules/dagster/dagster/core/execution/plan/inputs.py
441–442

wellthereitis

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 30 2021, 5:17 PM
Harbormaster failed remote builds in B32903: Diff 40553!
yuhan edited reviewers, added: alangenfeld; removed: yuhan.
This revision is now accepted and ready to land.Jul 7 2021, 6:28 PM
This revision was automatically updated to reflect the committed changes.