HomeElementl

Combine scheduler and queuer CLIs into a single dagster_daemon run cli call

Description

Combine scheduler and queuer CLIs into a single dagster_daemon run cli call

Summary:
This is a look at what it might look like if we used a single dagster-daemon CLI call for each of the scheduler/run-coordinator/sensors/insert future daemons here>. It likely does not actually work yet, just getting the proposal out for any initial feedback. The big question in my mind is how/where they should be configured.

The outline is:

  • Daemon-specific stuff configured on the instance (the alternative is adding a bunch of daemon-specific CLI args, I would also be ok with that - but I also like that there's a single source of truth for whether you're using a daemon and how that daemon should be configured)
  • on startup, the daemon command loads up any daemons based on the instance and runs each of their loops.

Test Plan: BK

Reviewers: alangenfeld, johann, schrockn, prha

Reviewed By: johann

Differential Revision: https://dagster.phacility.com/D5009

Details

Provenance
dgibsonAuthored on Nov 3 2020, 2:55 PM
Reviewer
johann
Differential Revision
D5009: Combine scheduler and queuer CLIs into a single dagster_daemon run cli call
Parents
R1:2be6c502ccd8: [dagit] Fix Run page overscrolling
Branches
Unknown
Tags
Unknown