HomeElementl

Ignore errors for daemon liveness check

Description

Ignore errors for daemon liveness check

Summary:
Previously, the we used the dagster-daemon health-check command as a liveness probe on k8s. This command would fail when one of the daemon heartbeats contained an error, thus the daemon would go into a crashloop. The daemon already has error handling, so we shouldn't kill it whenever it reports an error.

The new dagster-daemon liveness-check command only asserts that heartbeats have been posted, and ignores any errors they contain.

Test Plan:
Integration

new units

Reviewers: dgibson, prha

Reviewed By: dgibson

Differential Revision: https://dagster.phacility.com/D6212

Details

Provenance
johannAuthored on Jan 29 2021, 4:47 PM
Reviewer
dgibson
Differential Revision
D6212: Ignore errors for daemon liveness check
Parents
R1:d4c33e93db7c: fix bad args for schedule job state
Branches
Unknown
Tags
Unknown