Details

Reviewers

Group Reviewers

Restricted Project

Commits

R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext

Summary

Add Scheduler to the DagsterGraphQLContext and the option to specify the storage directory for schedules in the dagit cli. By default, the storage directory is $DAGSTER_HOME/schedules

Test Plan

run dagit with SCHEDULER feature enabled, unit

dagster.cgf:

[FEATURES]
scheduler=

Diff Detail

Repository

R1 dagster

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

sashank created this revision.Aug 12 2019, 6:45 PM

sashank added a parent revision: D791: Create Scheduler API.Aug 12 2019, 6:46 PM

Harbormaster failed remote builds in B2881: Diff 3602!Aug 12 2019, 6:52 PM

Fix typo

Harbormaster failed remote builds in B2882: Diff 3604!Aug 12 2019, 7:07 PM

Rebase

Harbormaster completed remote builds in B2883: Diff 3605.Aug 12 2019, 8:45 PM

alangenfeld requested changes to this revision.Aug 12 2019, 9:23 PM

alangenfeld added a subscriber: alangenfeld.

alangenfeld added inline comments.

python_modules/dagster/dagster/utils/__init__.py
272–310	lets set up something a little better here - we should at bare minimum use an enum. Could be nice to set up something cute being able to do `Features.SCHEDULER.is_enabled()` for example just to spitball an api

This revision now requires changes to proceed.Aug 12 2019, 9:23 PM

Update to use new Scheduler

Create Features class:

Check if feature is enabled using Features.FEATURENAME.is_enabled

sashank retitled this revision from Add schedule storage to dagit to Add Scheduler to dagit and DagsterGraphQLContext.Aug 13 2019, 5:10 AM

sashank edited the summary of this revision. (Show Details)

Harbormaster failed remote builds in B2904: Diff 3631!Aug 13 2019, 5:11 AM

sashank edited the test plan for this revision. (Show Details)Aug 13 2019, 5:11 AM

sashank added a child revision: D804: Create Scheduler GraphQL interface.Aug 13 2019, 9:23 AM

alangenfeld accepted this revision.Aug 13 2019, 3:13 PM

alangenfeld added inline comments.

python_modules/dagit/dagit/cli.py
73	hm wonder if this will be useful at all
python_modules/dagster/dagster/utils/__init__.py
272–310	_ prefix these since they shouldnt be used outside this file

This revision is now accepted and ready to land.Aug 13 2019, 3:13 PM

Make get_enabled_features() private

Harbormaster failed remote builds in B2921: Diff 3651!Aug 13 2019, 6:21 PM

Use six.raise_from in SystemCronScheduler (from last diff)

make graphql

Harbormaster failed remote builds in B2922: Diff 3653!Aug 13 2019, 6:27 PM

Harbormaster completed remote builds in B2923: Diff 3654.Aug 13 2019, 6:32 PM

Rerun tests

Harbormaster completed remote builds in B2927: Diff 3658.Aug 13 2019, 7:01 PM

Closed by commit R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext (authored by sashank). · Explain WhyAug 13 2019, 8:34 PM

This revision was automatically updated to reflect the committed changes.

Sashank Thupukari <s@thupukari.com> added a commit: R1:a8adcc7d7e40: Add Scheduler to dagit and DagsterGraphQLContext.

			Path	Packages
M			js_modules/dagit/src/filteredSchema.generated.json (2 lines)
M			python_modules/dagit/dagit/app.py (30 lines)
M			python_modules/dagit/dagit/cli.py (25 lines)
M			python_modules/dagit/dagit_tests/test_app.py (27 lines)
M			python_modules/dagster-graphql/dagster_graphql/implementation/context.py (15 lines)
M			python_modules/dagster-graphql/dagster_graphql/implementation/scheduler/system_cron_scheduler.py (17 lines)
M			python_modules/dagster/dagster/utils/__init__.py (32 lines)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	3602	988a71a		Aug 12 2019, 6:45 PM	★	★
Diff 2	3604	988a71a	Fix typo	Aug 12 2019, 7:00 PM	★	★
Diff 3	3605	fd42a65	Rebase	Aug 12 2019, 8:31 PM	★	★
Diff 4	3631	4d407fb	Update to use new Scheduler	Aug 13 2019, 5:04 AM	★	★
Diff 5	3651	b0b886a	Make get_enabled_features() private	Aug 13 2019, 6:14 PM	★	★
Diff 6	3653	b0b886a	Use `six.raise_from` in `SystemCronScheduler` (from last diff)	Aug 13 2019, 6:22 PM	★	★
Diff 7	3654	b0b886a	`make graphql`	Aug 13 2019, 6:26 PM	★	★
Diff 8	3658	46421ee	Rerun tests	Aug 13 2019, 6:54 PM	★	★
Diff 9	3663	46421ee	R1:a8adcc7d7e40cc1da32d8aa13bd7b95fb0189624	Aug 13 2019, 8:34 PM	★	★

Add Scheduler to dagit and DagsterGraphQLContext
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 3663

js_modules/dagit/src/filteredSchema.generated.json

python_modules/dagit/dagit/app.py

python_modules/dagit/dagit/cli.py

python_modules/dagit/dagit_tests/test_app.py

python_modules/dagster-graphql/dagster_graphql/implementation/context.py

python_modules/dagster-graphql/dagster_graphql/implementation/scheduler/system_cron_scheduler.py

python_modules/dagster/dagster/utils/init.py

Add Scheduler to dagit and DagsterGraphQLContextClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 3663

js_modules/dagit/src/filteredSchema.generated.json

python_modules/dagit/dagit/app.py

python_modules/dagit/dagit/cli.py

python_modules/dagit/dagit_tests/test_app.py

python_modules/dagster-graphql/dagster_graphql/implementation/context.py

python_modules/dagster-graphql/dagster_graphql/implementation/scheduler/system_cron_scheduler.py

python_modules/dagster/dagster/utils/__init__.py

Add Scheduler to dagit and DagsterGraphQLContext
ClosedPublic
Actions

Revision Contents
Changeset List

python_modules/dagster/dagster/utils/init.py