HomeElementl

fix get_source_run_id

Description

fix get_source_run_id

Summary:
https://github.com/dagster-io/dagster/issues/4087
current: _get_source_run_id only looks parent_run_id vs current run_id, which fails the re-execution of a re-executed subset, such as:

pipeline: a->b->c

run#1 step_selection: *
run#2 step_selection: b,c
run#3 step_selection: b,c - this will fails bc `get_source_run_id` for step a will return run#2' id which doesn't have an output for the step a

fix: walk through event logs to find the right run_id based on the run lineage (run_group info)

Test Plan:

  • unit
  • dagit: multiple "from selection" works:
    image.png (622×1 px, 104 KB)

Reviewers: alangenfeld, sandyryza, max

Reviewed By: sandyryza

Differential Revision: https://dagster.phacility.com/D7620

Details

Provenance
yuhanAuthored on Apr 27 2021, 6:58 AM
Reviewer
sandyryza
Differential Revision
D7620: fix get_source_run_id
Parents
R1:a8cd7d887f2b: Use "dev" as version string for master
Branches
Unknown
Tags
Unknown