Details

Reviewers

• nate
• catherinewu
alangenfeld

Commits

R1:fa9dbe5977a0: Update Scheduler helm schema to use enum types

Summary

Depends on D5748

As the title, enumify the helm scheduler values, rather
than relying on a boolean toggle.

Test Plan

bk/helm lint
integration

Diff Detail

Repository

R1 dagster

Branch

rl-json-schema-scheduler

Lint

Lint Passed

Unit

No Test Coverage

Event Timeline

rexledesma created this revision.Oct 27 2020, 5:52 PM

Harbormaster completed remote builds in B20237: Diff 24554.Oct 27 2020, 6:12 PM

rexledesma requested review of this revision.Oct 27 2020, 6:12 PM

use pydantic DSL

rexledesma retitled this revision from [3/n] Add schema for scheduler to [2/n] Add helm schema for Scheduler configuration.Oct 28 2020, 12:20 AM

rexledesma edited the summary of this revision. (Show Details)

rexledesma added a parent revision: D4895: [1/n] Add helm schema validation for Dagit configuration.

Harbormaster completed remote builds in B20287: Diff 24615.Oct 28 2020, 12:35 AM

rebase

Harbormaster completed remote builds in B20290: Diff 24619.Oct 28 2020, 1:05 AM

rebase

rexledesma retitled this revision from [2/n] Add helm schema for Scheduler configuration to [2/n] Add helm schema validation for Scheduler configuration.Oct 28 2020, 1:28 AM

Harbormaster completed remote builds in B20294: Diff 24623.Oct 28 2020, 1:38 AM

Harbormaster completed remote builds in B20296: Diff 24625.Oct 28 2020, 1:40 AM

Harbormaster completed remote builds in B20297: Diff 24626.Oct 28 2020, 1:43 AM

Harbormaster completed remote builds in B20295: Diff 24624.Oct 28 2020, 1:50 AM

rebase

rexledesma added a reviewer: Restricted Project.Oct 28 2020, 6:52 PM

rexledesma added inline comments.

python_modules/automation/automation/helm/schema/scheduler.py
39	support for conditional schemas isn't first class in pydantic so we'll have to override the schema here (https://github.com/samuelcolvin/pydantic/issues/529)

Harbormaster completed remote builds in B20343: Diff 24677.Oct 28 2020, 7:03 PM

alangenfeld added a subscriber: alangenfeld.Oct 28 2020, 11:06 PM

alangenfeld added inline comments.

helm/dagster/templates/configmap-instance.yaml
13–14	this is a breaking change right? do we want to do that ahead of `0.10.0`?
python_modules/automation/automation/helm/schema/scheduler.py
39	maybe leave this is a comment in the code

separate out breaking changes from things we can schema-tize without breaking

This revision now requires changes to proceed.Oct 29 2020, 6:29 PM

Will wait until cherry pick mode to land this - removing reviewers for now

This revision now requires review to proceed.Nov 2 2020, 7:35 PM

necro

resurrect from the dead

rexledesma edited the test plan for this revision. (Show Details)Dec 12 2020, 6:35 AM

rexledesma removed a subscriber: alangenfeld.

integration tests

Harbormaster completed remote builds in B22735: Diff 27653.Dec 12 2020, 6:52 AM

Harbormaster failed remote builds in B22737: Diff 27655!Dec 12 2020, 6:59 AM

make sure to document all the breaking changes in changes.md and migration.md

Harbormaster failed remote builds in B23695: Diff 28818!Jan 6 2021, 4:54 AM

rexledesma retitled this revision from [2/n] Add helm schema validation for Scheduler configuration to Update Scheduler helm schema to use enum types.Jan 6 2021, 5:01 AM

rexledesma edited the summary of this revision. (Show Details)

rebase

lint

Harbormaster failed remote builds in B23709: Diff 28832!Jan 6 2021, 9:40 AM

Harbormaster failed remote builds in B23710: Diff 28833!Jan 6 2021, 9:43 AM

Harbormaster failed remote builds in B23711: Diff 28834!Jan 6 2021, 9:53 AM

rebase

Harbormaster failed remote builds in B23712: Diff 28835!Jan 6 2021, 10:03 AM

Harbormaster completed remote builds in B23714: Diff 28837.Jan 6 2021, 10:11 AM

Harbormaster completed remote builds in B23715: Diff 28838.Jan 6 2021, 10:23 AM

rexledesma added reviewers: • nate, • catherinewu, alangenfeld.Jan 6 2021, 7:08 PM

rexledesma mentioned this in D5872: [1/2] Add helm configuration for dagster run launcher.Jan 8 2021, 1:11 AM

rexledesma added a child revision: Restricted Differential Revision.Jan 8 2021, 4:54 PM

rexledesma added a child revision: D5872: [1/2] Add helm configuration for dagster run launcher.Jan 8 2021, 6:08 PM

rexledesma edited the summary of this revision. (Show Details)

rexledesma added a parent revision: D5748: [2/2] Add helm schema for dagster compute log manager.

alangenfeld accepted this revision.Jan 8 2021, 7:31 PM

alangenfeld added inline comments.

helm/dagster/templates/configmap-instance.yaml
44–45	probably a custom option too just to be safe that takes module class and config?

This revision is now accepted and ready to land.Jan 8 2021, 7:31 PM

Closed by commit R1:fa9dbe5977a0: Update Scheduler helm schema to use enum types (authored by rexledesma). · Explain WhyJan 8 2021, 11:22 PM

This revision was automatically updated to reflect the committed changes.

rexledesma added a commit: R1:fa9dbe5977a0: Update Scheduler helm schema to use enum types.

rexledesma mentioned this in D5889: Expose custom scheduler option in helm values.Jan 9 2021, 12:06 AM

rexledesma mentioned this in R1:7a08806301b1: Expose custom scheduler option in helm values.Jan 12 2021, 7:07 PM

Update Scheduler helm schema to use enum typesClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 24623

.buildkite/defines.py

.buildkite/pipeline.py

docs/next/src/pages/deploying/k8s_part2.mdx

helm/dagster/templates/_helpers.tpl

helm/dagster/templates/configmap-instance.yaml

helm/dagster/values.schema.json

helm/dagster/values.yaml

integration_tests/python_modules/dagster-k8s-test-infra/dagster_k8s_test_infra/helm.py

js_modules/dagit/package.json

js_modules/dagit/src/PythonErrorInfo.tsx

js_modules/dagit/src/__tests__/App.test.tsx

js_modules/dagit/src/__tests__/__snapshots__/App.test.tsx.snap

js_modules/dagit/src/testing/ApolloTestProvider.test.tsx

js_modules/dagit/src/testing/ApolloTestProvider.tsx

js_modules/dagit/src/testing/defaultMocks.ts

js_modules/dagit/yarn.lock

python_modules/automation/automation/docker/images/buildkite-integration/snapshot-reqs-2.7.18.txt

python_modules/automation/automation/docker/images/buildkite-unit-snapshot-builder/Dockerfile

python_modules/automation/automation/docker/images/buildkite-unit/snapshot-reqs-2.7.18.txt

python_modules/automation/automation/docker/images/buildkite-unit/snapshot-reqs-3.6.11.txt

python_modules/automation/automation/docker/images/buildkite-unit/snapshot-reqs-3.7.8.txt

python_modules/automation/automation/docker/images/buildkite-unit/snapshot-reqs-3.8.3.txt

python_modules/automation/automation/docker/images/coverage-image/Dockerfile

python_modules/automation/automation/docker/images/coverage-image/last_updated.yaml

python_modules/automation/automation/helm/__init__.py

python_modules/automation/automation/helm/cli.py

python_modules/automation/automation/helm/schema/__init__.py

python_modules/automation/automation/helm/schema/dagit.py

python_modules/automation/automation/helm/schema/definitions.py

python_modules/automation/automation/helm/schema/kubernetes.py

python_modules/automation/automation/helm/schema/scheduler.py

python_modules/automation/automation/helm/schema/utils.py

python_modules/automation/automation/helm/schema/values.py

python_modules/automation/setup.py

python_modules/dagit/dagit_tests/test_debug_cli.py

python_modules/dagster-graphql/dagster_graphql/cli.py

python_modules/dagster-test/Dockerfile

python_modules/dagster/dagster/cli/api.py

python_modules/dagster/dagster/cli/workspace/cli_target.py

python_modules/dagster/dagster/core/code_pointer.py

python_modules/dagster/dagster/core/definitions/__init__.py

python_modules/dagster/dagster/core/definitions/composition.py

python_modules/dagster/dagster/core/definitions/config_mappable.py

python_modules/dagster/dagster/core/definitions/decorators/composite_solid.py

python_modules/dagster/dagster/core/definitions/decorators/pipeline.py

python_modules/dagster/dagster/core/definitions/dependency.py

python_modules/dagster/dagster/core/definitions/environment_configs.py

python_modules/dagster/dagster/core/definitions/graph.py

python_modules/dagster/dagster/core/definitions/i_solid_definition.py

python_modules/dagster/dagster/core/definitions/inference.py

python_modules/dagster/dagster/core/definitions/pipeline.py

python_modules/dagster/dagster/core/definitions/reconstructable.py

python_modules/dagster/dagster/core/definitions/solid.py

python_modules/dagster/dagster/core/definitions/solid_container.py

python_modules/dagster/dagster/core/execution/plan/plan.py

python_modules/dagster/dagster/core/execution/results.py

python_modules/dagster/dagster/core/snap/dep_snapshot.py

python_modules/dagster/dagster/core/storage/runs/sql_run_storage.py

python_modules/dagster/dagster/core/system_config/composite_descent.py

python_modules/dagster/dagster/serdes/ipc.py

python_modules/dagster/dagster/utils/test/__init__.py

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_cli_commands.py

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_execute_command.py

python_modules/dagster/dagster_tests/cli_tests/command_tests/test_launch_command.py

python_modules/dagster/dagster_tests/core_tests/config_types_tests/evaluator_tests/test_composite_descent.py

python_modules/dagster/dagster_tests/core_tests/definitions_tests/test_composition.py

python_modules/dagster/dagster_tests/core_tests/definitions_tests/test_definition_errors.py

python_modules/dagster/dagster_tests/core_tests/test_composites.py

python_modules/dagster/dagster_tests/core_tests/test_pipeline_execution.py

python_modules/dagster/dagster_tests/general_tests/compat_tests/test_back_compat.py

python_modules/dagster/dagster_tests/general_tests/compat_tests/test_change_snapshot_structure.py

python_modules/dagster/dagster_tests/general_tests/grpc_tests/test_persistent.py

python_modules/dagster/dev-requirements.txt

python_modules/libraries/dagster-dask/dagster_dask_tests/test_resources.py

Update Scheduler helm schema to use enum types
ClosedPublic
Actions

Revision Contents
Changeset List

js_modules/dagit/src/tests/App.test.tsx

js_modules/dagit/src/tests/snapshots/App.test.tsx.snap

python_modules/automation/automation/helm/init.py

python_modules/automation/automation/helm/schema/init.py

python_modules/dagster/dagster/core/definitions/init.py

python_modules/dagster/dagster/utils/test/init.py