HomeElementl

add run group tags to backfill jobs

Description

add run group tags to backfill jobs

Summary:
this fixes a bug where partial re-execution via backfills fails because it's unable to find the source run id using run group info
the root cause was instance.get_run_group fetches the info based on tags but the pipeline runs created via the backfill re-execution flow didn't have the corespondng tags (i.e. PARENT_RUN_ID and ROOT_RUN_ID)

the better solution is to not let get_run_group depend on the tags info but that may require a change in the runs table, so im not going with that for now - wanted to get the fix in today's release

Test Plan:
local dagit - this only happens in sql storage bc of how the get_run_group was implemented, so it's a bit tricky to mock everything in the situation in a unit test

repro the bug: launch backfill -> fail the job -> reexecute from failure. the run didn't have the parent/root tags

Screen Shot 2021-05-06 at 10.25.27 AM.png (2×4 px, 875 KB)

the fix fixed the bug: re-execution succeeded. the runs have the tags

Screen Shot 2021-05-06 at 10.25.04 AM.png (718×4 px, 304 KB)

Reviewers: prha, alangenfeld

Reviewed By: alangenfeld

Differential Revision: https://dagster.phacility.com/D7767

Details

Provenance
yuhanAuthored on May 6 2021, 5:27 PM
Reviewer
alangenfeld
Differential Revision
D7767: add run group tags to backfill jobs
Parents
R1:08e1760a18af: [dagstermill] fix aliased bug
Branches
Unknown
Tags
Unknown