[crag] Add validation step to default config buildup for to_job.
ClosedPublic
Actions

Authored by • cdecarolis on Aug 2 2021, 11:10 PM.

Details

Reviewers

• sandyryza
alangenfeld

Commits

R1:b4ddfb97ba7b: [crag] Add validation step to default config buildup for to_job.

Summary

We previously weren't doing any validation of config in to_job. This adds validation, and also carries over all fields from old config document.

Test Plan

added unit test to ensure config mistakes are caught.

Diff Detail

Repository

R1 dagster

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

• cdecarolis created this revision.Aug 2 2021, 11:10 PM

Harbormaster completed remote builds in B34570: Diff 42751.Aug 3 2021, 12:11 AM

• cdecarolis requested review of this revision.Aug 3 2021, 12:11 AM

• cdecarolis retitled this revision from Add validation step to default config buildup for to_job. to [crag] Add validation step to default config buildup for to_job..Aug 3 2021, 12:12 AM

Harbormaster failed remote builds in B34568: Diff 42748!Aug 3 2021, 12:16 AM

alangenfeld accepted this revision.Aug 3 2021, 2:56 PM

alangenfeld added inline comments.

python_modules/dagster/dagster/core/definitions/graph.py
795–797	I think we can do better with this pre-amble make sure to align with the name of the arg - drop "default" i think mention the job name / graph name, maybe both since is the a failure in the process of creating the job from the graph