HomePhabricator

[dagster] error on unresolved inputs that cant be hydrated

Authored by alangenfeld on Jul 31 2019, 10:53 PM.

Description

[dagster] error on unresolved inputs that cant be hydrated

Summary:
This diff resolves an issue where you could create unexecutable pipelines by selecting subsets where the dangling inputs can not be materialized from config. To solve this we:

  • introduce validation at pipeline instantiation time that all unresolved inputs must have a hydration config
  • create a single InvalidSubsetError type that incapsulates both bad solid names as well as the issue described above
  • ensure that the UI communicates these problems effectively when selecting invalid subsets

Test Plan:
new test case

load up the airline demo ingest pipeline and select subsets that are invalid

Reviewers: #ft, max

Reviewed By: #ft, max

Subscribers: max

Differential Revision: https://dagster.phacility.com/D751

Details

Committed
alangenfeldAug 6 2019, 8:55 PM
Reviewer
Restricted Project
Differential Revision
D751: [dagster] error on unresolved inputs that cant be hydrated
Parents
R1:28c3b215a164: Convert SolidDefinitions and PipelineDefinitions to @solid and @pipeline
Branches
Unknown
Tags
Unknown