Page MenuHomePhabricator

Add Scheduler to dagit and DagsterGraphQLContext
ClosedPublic

Authored by sashank on Aug 12 2019, 6:45 PM.

Details

Reviewers
alangenfeld
Group Reviewers
Restricted Project
Commits
R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext
Summary

Add Scheduler to the DagsterGraphQLContext and the option to specify the storage directory for schedules in the dagit cli. By default, the storage directory is $DAGSTER_HOME/schedules

Test Plan

run dagit with SCHEDULER feature enabled, unit

dagster.cgf:

[FEATURES]
scheduler=

Diff Detail

Repository
R1 dagster
Branch
schedule-storage-dagit
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

sashank created this revision.Aug 12 2019, 6:45 PM
sashank updated this revision to Diff 3604.Aug 12 2019, 7:00 PM

Fix typo

alangenfeld requested changes to this revision.Aug 12 2019, 9:23 PM
alangenfeld added a subscriber: alangenfeld.
alangenfeld added inline comments.
python_modules/dagster/dagster/utils/__init__.py
272–315

lets set up something a little better here - we should at bare minimum use an enum. Could be nice to set up something cute
being able to do Features.SCHEDULER.is_enabled() for example just to spitball an api

This revision now requires changes to proceed.Aug 12 2019, 9:23 PM
sashank updated this revision to Diff 3631.Aug 13 2019, 5:04 AM

Update to use new Scheduler

Create Features class:

  • Check if feature is enabled using Features.FEATURENAME.is_enabled
sashank retitled this revision from Add schedule storage to dagit to Add Scheduler to dagit and DagsterGraphQLContext.Aug 13 2019, 5:10 AM
sashank edited the summary of this revision. (Show Details)
sashank edited the test plan for this revision. (Show Details)Aug 13 2019, 5:11 AM
alangenfeld accepted this revision.Aug 13 2019, 3:13 PM
alangenfeld added inline comments.
python_modules/dagit/dagit/cli.py
73

hm wonder if this will be useful at all

python_modules/dagster/dagster/utils/__init__.py
272–292

_ prefix these since they shouldnt be used outside this file

This revision is now accepted and ready to land.Aug 13 2019, 3:13 PM
sashank updated this revision to Diff 3651.Aug 13 2019, 6:14 PM

Make get_enabled_features() private

sashank updated this revision to Diff 3653.Aug 13 2019, 6:22 PM

Use six.raise_from in SystemCronScheduler (from last diff)

sashank updated this revision to Diff 3654.Aug 13 2019, 6:26 PM

make graphql

sashank updated this revision to Diff 3658.Aug 13 2019, 6:54 PM

Rerun tests