HomeElementl

[RFC] Cancel queued runs

Description

[RFC] Cancel queued runs

Summary:

  • If queued, send a failed event
  • otherwise, pass on to the run launcher

Currently this is lumping together cancel (remove from queue) and terminate (kill a launched run). I think this is ok for now (how often does the use case come up that you only want to cancel a certain run if it is queued?), but it doesn't seem like a difficult decision to walk back if we need to make a distinction later.

An oddity now is that pipelines go from QUEUED (can click terminate) to NOT_STARTED (can't click terminate) to STARTED (can click terminate). A possible solution is to guarantee that every run worker checks pipeline state before starting a run, so a NOT_STARTED run could be safely marked as FAILED.

Not in this diff:

  • Override can_terminate
  • Bulk deletion mutation

Test Plan:
Integration

manually tried in dagit

TODO: add cases

Reviewers: alangenfeld, dgibson, prha

Reviewed By: dgibson

Differential Revision: https://dagster.phacility.com/D5291

Details

Provenance
johannAuthored on Nov 25 2020, 2:12 PM
Reviewer
dgibson
Differential Revision
D5291: [RFC] Cancel queued runs
Parents
R1:e88ad6815224: bye-system-storage-6 remove @system_storage and SystemStorageDefinition
Branches
Unknown
Tags
Unknown