Preliminary to selecting engines through Dagit -- first make sure that we are using a single code path for all of our execution modalities.
can you update the summary with a little detail as to how these changes set us up for further improvement? I don't see any objectionable code changes here, but its also not immediately obvious where this is going.
is there a reason to do it this way vs importing the pipeline directly?
Yep, the second phase eliminates executor_config from RunConfig entirely, specifying it through the environment dict. It's key for this that all of our execution modalities use a single code path.
yes, only pipelines instantiated from handles can be executed across process boundaries
maybe check to ensure that config doesn't have multiple items since you're doing single_item here?
does this have test coverage? since you're promoting to utils might be nice to write a small test to demo what it does