Page MenuHomePhabricator

Add liveness probe to extra celery workers
ClosedPublic

Authored by rexledesma on Oct 6 2020, 12:55 AM.

Details

Summary

As the title.

Test Plan

bk, add fields to helm config

Diff Detail

Repository
R1 dagster
Branch
arcpatch-D4674
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 6 2020, 1:39 AM
Harbormaster failed remote builds in B19167: Diff 23294!

skip everything except integration tests

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 6 2020, 7:47 PM
Harbormaster failed remote builds in B19208: Diff 23341!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 6 2020, 8:44 PM
Harbormaster failed remote builds in B19213: Diff 23347!
johann added inline comments.
integration_tests/python_modules/dagster-k8s-test-infra/dagster_k8s_test_infra/helm.py
335

@alangenfeld - you mentioned you only wanted to have the extra worker for a single test. The cluster is session scoped though, so I'm not sure how we'd do that

Since the extraWorkerQueues part of this test has caused flakiness in the past, can we make sure to run this through bk a few times (5?) times before merging? last time i tried to add extraWorkerQueues back, it passed the first three runs and then failed twice (https://dagster.phacility.com/D4535)

integration_tests/python_modules/dagster-k8s-test-infra/dagster_k8s_test_infra/helm.py
335

The cluster is session scoped

Ah i see I didn't realize this - not sure whats best. You could do a helm upgrade with new chunk of values but thats pretty complicated

awesome, looks good!

Macro pooh_eating_honey:

after we merge celery and celery-extras yaml, we probably want to make it possible to configure different liveness probes per queue

This revision is now accepted and ready to land.Oct 16 2020, 8:06 PM