if someone could commandeer this that would be great 😄

I think the last thing we need to figure out is how to toggle between the new set up and the old set up - assuming we don't want to do a hard cut-over immediately

can you split out the execute pipeline change to its own diff? would be good to land the launch pipeline fix ASAP

ya in my head dagster pipeline execute is for "just execute this pipeline right here in my terminal so i can debug it / see what happens", which might not be *right* but is what gives me pause here

curious if @schrockn has thoughts here

hmm dagster pipeline execute firing a pipeline execution in an un-managed grpc server is a bit worrisome. In our deployed cases the assumption is the grpc server acts as metadata-reads-only and all executions happen via run launcher. I'm not certain that dagster pipeline execute shouldn't boot up in a user process context directly and execute right there. I guess its a little weird if you are choosing out of a workspace that declares a different python env for that pipeline

error message not your fault but worth improving

flake failure

^ passed

bk failure was from flakey test

I wonder if there are any bugfixes in the more recent release of 3.8 - maybe worth updating the minor version of that we are pinned to

add a note to CHANGES.md breaking changes section

LocalRunLauncher is a good name

we can drop dagit_settings but can't break telemtry_settings

maybe hold off on landing this till after the weekly on Thursday. We can pile up all the diffs that move off the dagster-graphql stuff and land them together.

guess this can't hurt to land

Future: It’s pretty common to get unexpected values when working with data. We should probably handle unexpected values (like None) in EventMetadataEntry without crashing

The only place where repository_key and repository_name are ever different is the specific case where you specify an attribute in your load_from

as discussed in person - lets try a pipeline snapshot id resolution approach and see how that goes before mucking with storage schema

Maybe better for me to retire this change until we have some better entity to attach this metadata to?

up

I did some manual tests locally and if any of the files change the docker build context changes and the layer gets re-run so I think its legit - will have to keep an eye out

im guessing this will end up shifting around a bit with regards to recent conversation, but will still be needed for externally managed grpc server

generally worth going over K8sScheduler carefully and trimming it down to just what is actually needed in the k8s context

up

up

up

up

up

up

more tweaks

up

just wait to land til we know which week we are rolling 0.9.0