HomePhabricator

Create ScheduleDefinition and dagster CLI scheduler commands

Authored by Sashank Thupukari <s@thupukari.com> on Aug 19 2019, 8:09 PM.

Description

Create ScheduleDefinition and dagster CLI scheduler commands

Summary:
In this v1 implementation, we attach ScheduleDefinitions to a RepositoryDefinition.

many_events_daily_schedule = ScheduleDefinition(
    name="many_events_midnight_daily",
    cron_schedule="0 0 * * *",
    execution_params={
        "environmentConfigData": {"storage": {"filesystem": None}},
        "selector": {"name": "many_events", "solidSubset": None},
        "mode": "default",
    },
)

many_events_every_minute_schedule = ScheduleDefinition(
    name="many_events_midnight_every_minute",
    cron_schedule="* * * * *",
    execution_params={
        "environmentConfigData": {"storage": {"filesystem": None}},
        "selector": {"name": "many_events", "solidSubset": None},
        "mode": "default",
    },
)

return RepositoryDefinition(
    name='toys_repository',
    pipeline_defs=[
        many_events,
    ],
    schedule_defs=[many_events_daily_schedule, many_events_every_minute_schedule],
)

We can then interact with the ScheduleDefinitions using the dagster cli.

$ dagster schedules list
Repository toys_repository
**************************
Schedule: many_events_midnight_daily
Cron Schedule: 0 0 * * *
********************************************
Schedule: many_events_midnight_every_minute
Cron Schedule: * * * * *
$ dagster schedules list --verbose
Repository toys_repository
**************************
Schedule: many_events_midnight_daily
Cron Schedule: 0 0 * * *
Execution Params: {'environmentConfigData': {'storage': {'filesystem': None}}, 'selector': {'name': 'many_events', 'solidSubset': None}, 'mode': 'default'}
********************************************
Schedule: many_events_midnight_every_minute
Cron Schedule: * * * * *
Execution Params: {'environmentConfigData': {'storage': {'filesystem': None}}, 'selector': {'name': 'many_events', 'solidSubset': None}, 'mode': 'default'}
$ dagster schedules start many_events_midnight_daily
Started schedule many_events_midnight_daily with ID 2a0ebd26-da90-4a71-96df-f249f50a6498
$ dagster schedules list --running
Repository toys_repository
**************************
Schedule: many_events_midnight_daily [Running]
Cron Schedule: 0 0 * * *
$ dagster schedules end many_events_midnight_daily
Ended schedule many_events_midnight_daily with ID 2a0ebd26-da90-4a71-96df-f249f50a6498
$ dagster schedules list --running
Repository toys_repository
**************************

Test Plan: unit

Reviewers: #ft, alangenfeld, schrockn, natekupp

Reviewed By: #ft, schrockn, natekupp

Subscribers: schrockn, alangenfeld

Differential Revision: https://dagster.phacility.com/D842

Details

Committed
Sashank Thupukari <s@thupukari.com>Aug 20 2019, 10:20 PM
Reviewer
Restricted Project
Differential Revision
D842: Create ScheduleDefinition and dagster CLI scheduler commands
Parents
R1:df1f097aa9a4: Add systemd service for dagit
Branches
Unknown
Tags
Unknown