Differential D4920 Diff 28833 MIGRATION.md

Changeset View

Standalone View

MIGRATION.md

Show First 20 Lines • Show All 117 Lines • ▼ Show 20 Lines

To migrate to 0.10.0, you'll need to update your `values.yaml` with the following renames:		To migrate to 0.10.0, you'll need to update your `values.yaml` with the following renames:

`pipeline_run` -> `pipelineRun`		`pipeline_run` -> `pipelineRun`
`dagster_home` -> `dagsterHome`		`dagster_home` -> `dagsterHome`
`env_secrets` -> `envSecrets`		`env_secrets` -> `envSecrets`
`env_config_maps` -> `envConfigMaps`		`env_config_maps` -> `envConfigMaps`

		## Restructured: scheduler in Helm values

		When specifying the Dagster instance scheduler, rather than using a boolean field to switch between the
		current options of `K8sScheduler` and `DagsterDaemonScheduler`, we now require the scheduler type to be
		explicitly defined under `scheduler.type`. If the user specified `scheduler.type` has required config,
		additional fields will need to be specified under `scheduler.config`.

		`scheduler.type` and corresponding `scheduler.config` values are enforced via [JSON Schema](https://helm.sh/docs/topics/charts/#schema-files).

# Migrating to 0.9.0		# Migrating to 0.9.0

## Removal: `config` argument		## Removal: `config` argument

We have removed the `config` argument to the `ConfigMapping`, `@composite_solid`, `@solid`,		We have removed the `config` argument to the `ConfigMapping`, `@composite_solid`, `@solid`,
`SolidDefinition`, `@executor`, `ExecutorDefinition`, `@logger`, `LoggerDefinition`, `@resource`,		`SolidDefinition`, `@executor`, `ExecutorDefinition`, `@logger`, `LoggerDefinition`, `@resource`,
and `ResourceDefinition` APIs, which we deprecated in 0.8.0, in favor of `config_schema`, as		and `ResourceDefinition` APIs, which we deprecated in 0.8.0, in favor of `config_schema`, as
described [here](#renaming-config).		described [here](#renaming-config).
Show All 18 Lines	- We have added an `intermediate_storage_defs` argument to `ModeDefinition`, which accepts a
storage.		storage.
- We have deprecated `system_storage_defs` argument to `ModeDefinition` in favor of		- We have deprecated `system_storage_defs` argument to `ModeDefinition` in favor of
`intermediate_storage_defs`. `system_storage_defs` will be removed in 0.10.0 at the earliest.		`intermediate_storage_defs`. `system_storage_defs` will be removed in 0.10.0 at the earliest.
- We have added an `@intermediate_storage` decorator, which makes it easy to define intermediate		- We have added an `@intermediate_storage` decorator, which makes it easy to define intermediate
storages.		storages.
- We have added `s3_file_manager` and `local_file_manager` resources to replace the file managers		- We have added `s3_file_manager` and `local_file_manager` resources to replace the file managers
that previously lived inside system storages. The airline demo has been updated to include		that previously lived inside system storages. The airline demo has been updated to include
an example of how to do this:		an example of how to do this:
https://github.com/dagster-io/dagster/blob/0.8.8/examples/airline_demo/airline_demo/solids.py#L171.		<https://github.com/dagster-io/dagster/blob/0.8.8/examples/airline_demo/airline_demo/solids.py#L171>.

For example, if your `ModeDefinition` looks like this:		For example, if your `ModeDefinition` looks like this:

```python		```python
from dagster_aws.s3 import s3_plus_default_storage_defs		from dagster_aws.s3 import s3_plus_default_storage_defs

ModeDefinition(system_storage_defs=s3_plus_default_storage_defs)		ModeDefinition(system_storage_defs=s3_plus_default_storage_defs)
```		```
▲ Show 20 Lines • Show All 592 Lines • Show Last 20 Lines