Page MenuHomePhabricator

Fix errors with GCP pipeline
ClosedPublic

Authored by sashank on Tue, Jan 14, 12:58 AM.

Details

Summary

Some of the breaking changes, to the config system left the gcp pipeline in a broken state. This diff fixes the config errors. Specifically, the input defs of composite solids have to match the first solid's input defs, and the output defs of a composite solid have to match the last solid's output_defs.

Test Plan

Add test

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

sashank created this revision.Tue, Jan 14, 12:58 AM

why wasn't this caught by tests?

sashank updated this revision to Diff 8695.Tue, Jan 14, 8:49 PM

Add dagster-gcp to setup.py

sashank updated this revision to Diff 8696.Tue, Jan 14, 8:57 PM

Add dagster-gcp install to tox.ini

schrockn accepted this revision.Tue, Jan 14, 10:38 PM

Seems good. Not sure with _START is better than 'start'

examples/dagster_examples/gcp_data_platform/final_pipeline.py
143

why not just 'start'? I don't see why the constant helps

This revision is now accepted and ready to land.Tue, Jan 14, 10:38 PM
sashank added inline comments.Tue, Jan 14, 10:55 PM
examples/dagster_examples/gcp_data_platform/final_pipeline.py
143

Just copied the pattern from other places we do this. Removed it here.

sashank updated this revision to Diff 8714.Tue, Jan 14, 10:55 PM

Remove _START constant

This revision was automatically updated to reflect the committed changes.