Awesome. Couple quqestions: Does this mean that whenever you, say, update code in a user deployment via helm, that it will run a migration check? I think generally 'dagster instance migrate' is idempotent, so that may be fine or even desirable, but just checking - one thing i'd potentially be worried in that case is other parts of the deployment interacting with the DB while its in the middle of migrating though.

If we did want to make it more of a manually-initiated thing (but still more streamlined than it is now in master), would there be a path for that?

Another option is to ask users to connect to their cluster and manually run kubectl apply -f dagster-instance-migrate.yaml to create the migration job so that users aren't caught off guard? for example, some users may want to be extra safe and backup their database before migration in case something unexpected happens

Also wanted to second daniel's concern about running the migration while dagit is in use.

In redash's upgrade process, they stop the service before migrating which we may need to do too

Does this mean that whenever you, say, update code in a user deployment via helm, that it will run a migration check?

Yes. However, once we split the helm chart to support isolated releases of user code and dagster infrastructure, this migration job will only be kicked off if the release for dagster infrastructure is updated.

If we did want to make it more of a manually-initiated thing (but still more streamlined than it is now in master), would there be a path for that?

We can put this automatic migration behind a flag dagit.migrationHook.enabled if that's of interest!

I'm not sure we should offer it as a flag if we're worried about the 'running other stuff during a migration' issue though - since if its an option in the helm chart we're implicitly endorsing it as safe.

@dgibson @catherinewu

I've made migration a more manual process. Rather than it being a hook, they'll have to manually kick off the job. I've also added steps for them to scale down their deployments so that no runs will be launched during the migration process.

this makes sense to me! @catherinewu or @johann is there anything else you all think should happen here? I put a couple of small comments about the docs inline.

FYI: we've landed D6835 later today - which affected the docs content structure. I'm happy to hop on a call to go over the new info hierarchy, especially the k8s guides!