Page MenuHomePhabricator

[dagster-k8s] get_celery_engine_config
ClosedPublic

Authored by alangenfeld on Mar 26 2020, 5:21 PM.

Details

Summary
  • add a way to get a config fragment for celery engine in k8s cluster
  • fix up some graphql queries that were a bit busted
Test Plan

added test

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

alangenfeld created this revision.Mar 26 2020, 5:21 PM
alangenfeld edited the summary of this revision. (Show Details)Mar 27 2020, 4:44 PM
alangenfeld added reviewers: max, nate, sashank.
alangenfeld added inline comments.
python_modules/libraries/dagster-k8s/dagster_k8s/config.py
5–13

the thinking here was that you dont need the same config source options for the thing publishing to the queue as you do for whats initializing the workers

nate accepted this revision.Mar 27 2020, 5:49 PM

cool, this looks reasonable to me—are you thinking we'd also have some way to invoke get_celery_engine_config while loading a preset?

This revision is now accepted and ready to land.Mar 27 2020, 5:49 PM

ya normal PresetDefinition construction can take this in with the rest of a manually constructed dict, the from_files case is a little tougher and might justify a with so i can add that in a separate diff

This revision was automatically updated to reflect the committed changes.