Details

Reviewers

Group Reviewers

Restricted Project

Commits

R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext

Summary

Add Scheduler to the DagsterGraphQLContext and the option to specify the storage directory for schedules in the dagit cli. By default, the storage directory is $DAGSTER_HOME/schedules

Test Plan

run dagit with SCHEDULER feature enabled, unit

dagster.cgf:

[FEATURES]
scheduler=

Diff Detail

Repository

R1 dagster

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

• sashank created this revision.Aug 12 2019, 6:45 PM

• sashank added a parent revision: D791: Create Scheduler API.Aug 12 2019, 6:46 PM

Harbormaster failed remote builds in B2881: Diff 3602!Aug 12 2019, 6:52 PM

Fix typo

Harbormaster failed remote builds in B2882: Diff 3604!Aug 12 2019, 7:07 PM

Rebase

Harbormaster completed remote builds in B2883: Diff 3605.Aug 12 2019, 8:45 PM

alangenfeld requested changes to this revision.Aug 12 2019, 9:23 PM

alangenfeld added a subscriber: alangenfeld.

alangenfeld added inline comments.

python_modules/dagster/dagster/utils/__init__.py
272–310	lets set up something a little better here - we should at bare minimum use an enum. Could be nice to set up something cute being able to do `Features.SCHEDULER.is_enabled()` for example just to spitball an api

This revision now requires changes to proceed.Aug 12 2019, 9:23 PM

Update to use new Scheduler

Create Features class:

Check if feature is enabled using Features.FEATURENAME.is_enabled

• sashank retitled this revision from Add schedule storage to dagit to Add Scheduler to dagit and DagsterGraphQLContext.Aug 13 2019, 5:10 AM

• sashank edited the summary of this revision. (Show Details)

Harbormaster failed remote builds in B2904: Diff 3631!Aug 13 2019, 5:11 AM

• sashank edited the test plan for this revision. (Show Details)Aug 13 2019, 5:11 AM

• sashank added a child revision: D804: Create Scheduler GraphQL interface.Aug 13 2019, 9:23 AM

alangenfeld accepted this revision.Aug 13 2019, 3:13 PM

alangenfeld added inline comments.

python_modules/dagit/dagit/cli.py
73	hm wonder if this will be useful at all
python_modules/dagster/dagster/utils/__init__.py
272–310	_ prefix these since they shouldnt be used outside this file

This revision is now accepted and ready to land.Aug 13 2019, 3:13 PM

Make get_enabled_features() private

Harbormaster failed remote builds in B2921: Diff 3651!Aug 13 2019, 6:21 PM

Use six.raise_from in SystemCronScheduler (from last diff)

make graphql

Harbormaster failed remote builds in B2922: Diff 3653!Aug 13 2019, 6:27 PM

Harbormaster completed remote builds in B2923: Diff 3654.Aug 13 2019, 6:32 PM

Rerun tests

Harbormaster completed remote builds in B2927: Diff 3658.Aug 13 2019, 7:01 PM

Closed by commit R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext (authored by • sashank). · Explain WhyAug 13 2019, 8:34 PM

This revision was automatically updated to reflect the committed changes.

Sashank Thupukari <s@thupukari.com> added a commit: R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext.

Add Scheduler to dagit and DagsterGraphQLContext
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 3663

js_modules/dagit/src/filteredSchema.generated.json

python_modules/dagit/dagit/app.py

python_modules/dagit/dagit/cli.py

python_modules/dagit/dagit_tests/test_app.py

python_modules/dagster-graphql/dagster_graphql/implementation/context.py

python_modules/dagster-graphql/dagster_graphql/implementation/scheduler/system_cron_scheduler.py

python_modules/dagster/dagster/utils/init.py

Add Scheduler to dagit and DagsterGraphQLContextClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 3663

js_modules/dagit/src/filteredSchema.generated.json

python_modules/dagit/dagit/app.py

python_modules/dagit/dagit/cli.py

python_modules/dagit/dagit_tests/test_app.py

python_modules/dagster-graphql/dagster_graphql/implementation/context.py

python_modules/dagster-graphql/dagster_graphql/implementation/scheduler/system_cron_scheduler.py

python_modules/dagster/dagster/utils/__init__.py

Add Scheduler to dagit and DagsterGraphQLContext
ClosedPublic
Actions

Revision Contents
Changeset List

python_modules/dagster/dagster/utils/init.py