up

Oh god I was doing prototyping in this area without bringing in the refactor

up

up

makes sense

My big question is why is this FromMultipleSources and not FromMultipleStepOutputs?

up

up

up

up

rebase

up

up

up

up

up

up

up

up

py2 failures expected. will land on fri

up

i can merge this with previous one if desired.

going to fail lint

up

for whatever reason getting this error when running locally

Wow this looks so much better

Interested to get @max 's take too and to get this going on a dbt installation to make sure it all makes sense

i think this looks great!

cool. plz consider final comments.

default_asset_store is a bit of a strange formulation in this context IMO. Verbally I would say in this diff that the default asset store is the in-memory asset store, but then that is assigned to the default_asset_store key? I think it kind of assumes that there will be multiple asset stores per-pipeline and I don't think that will be the common case. asset_store makes more sense IMO. variable name default_is_default exposes this awkardness

So I think we need another test case where the subset in question does *not* have a default input. That is config is required.

Well i want folks a chance to weigh in. I don't consider the decision made yet. Maybe tmrw?

I think one way to explore this is to ask how this API is going to expand in the future? Use cases including:

I'm really confused by JobConfig. From my perspective the current JobConfig in the diff represents the parameters to a run. It represents "I have determined that in this tick there should be a run launcher, and here are the parameters". JobConfig sounds like an argument that would be passed into the *job* definition.

worth an early push?

let's just change the warning to say it is being delete. i doubt it is very broadly used.

I do think this is an interesting case where overdoing it on "progressive disclosure of complexity" actually did active damage. The original idea was to be able to delay the introduction of the context object, at the expense of having an increase API surface. In this case it is pretty clear that that tradeoff was incorrect

I think we should delete it

def support landing this

got it. I reopen-ed https://github.com/dagster-io/dagster/issues/3169 for tracking if you want to put failures somewhere