HomePhabricator

[dagster] fix composition bug

Authored by alangenfeld on Aug 20 2019, 6:37 PM.

Description

[dagster] fix composition bug

Summary:
This terminal_step_output jonx did not get handled correctly for the entirety of composites existence. We never tested the case where you have fully parallel subdags within a composite which is where it breaks down.

To handle correctly, we build up a dict of compute steps as we recurse and then use the mapping path to resolve through any output mappings to the actual compute step it comes from.

Fixes https://github.com/dagster-io/dagster/issues/1674

Test Plan: newly added test case

Reviewers: #ft, schrockn

Reviewed By: #ft, schrockn

Subscribers: schrockn

Differential Revision: https://dagster.phacility.com/D847

Details

Committed
alangenfeldAug 21 2019, 2:46 PM
Reviewer
Restricted Project
Differential Revision
D847: [dagster] fix composition bug
Parents
R1:a2e28cc8c793: Fix SFTP solid test
Branches
Unknown
Tags
Unknown