HomeElementl

Daemon heartbeats backend

Description

Daemon heartbeats backend

Summary:
Add instance methods for checking on dagster-daemon health.

As part of it's loop the daemon updates the timestamp of it's heartbeat in a new db table. Other system components can use daemon_healthy() to check if heartbeats have been arriving.

Why run storage and not event log? Decision is somewhat arbitrary. Event log gets partitioned by run, so run storage was more convenient

TODO:

  • do an update rather than insert on the heartbeats table, or continually truncate it (Is there any reason we'd want to see past heartbeats?)

Questions:

  • use of time libraries is messy, how should I improve mocking, conversions, etc

Test Plan:
integration

Unit tests on run storage and an integration test with the daemon

Reviewers: dgibson, alangenfeld, dish, prha

Reviewed By: dgibson

Subscribers: dish

Differential Revision: https://dagster.phacility.com/D5120

Details

Provenance
johannAuthored on Nov 12 2020, 8:14 PM
Reviewer
dgibson
Differential Revision
D5120: Daemon heartbeats backend
Parents
R1:d1f9f58cd679: fix fan-in with asset stores
Branches
Unknown
Tags
Unknown
Reverted By
R1:8f4d961aa673: Revert "Daemon heartbeats backend"