xargs version

fixes

gogogo

test plan

delete dont comment

get the tests passing - I'm interested in seeing what the callsite changes look like

move again

lru_cache + functools32

ill let @prha take it from here

once you get ts to pass

queue management

rebase

now with working memoization

bad

fixes

fix lint

make graphql

black

parentHandleID

from in person discussion:

• move strict mode arg to mode (maybe rename it to describe the behavior of selective instantiation it turns on)
• remove required_resource_keys from composite solid and resolve that change up through the graphql layer
• explore cleaner options for providing the information currently handled by solid_def_names_in_execution

mmm low hanging fruit

gogogo

Don't need to block land on me - just throwing some ideas in the mix.

to @sashank's queue

Do we have an exact game plan for all this yet? I would push for deferring landing anything and trying to jump right to the end state unless there is a clear user need to satisfy.

should there be a test for multiprocess engine / execute plan APIs that ensures unused resources are not instantiated?

bringing these small utilities in libraries has a breaking-changeable surface area cost associated with it - so we should be thoughtful that the value provided surmounts that cost

if you do want to land any minor breaking changes - make sure to note them in changes.md so we dont lose track

unclear whats from mrjob and whats not - sending to your queue for another clean up pass. seems overall fine though

I have some hesitancy about this - I agree it is valuable but I feel like it makes things more confusing in particular where the function-based invocations meet the piped ones. I understand it since i know how it works - but I don't know of a good way to explain it to someone without just explaining how it works. To frame this feeling in another way - I think the combination of the function based capability and the pipe based capability is less than the sum of both since they combine in an unintuitive way.

to your queue

I feel like there has to be a way to get the nightly job to work like the main "Dagster" buildkite pipeline with that existing bot set up

can't we just hook up the nightly job to our buildkite slack channel bot?

gogogo

should probably grab dict/list also - file ticket as "good first issue" if you want to defer it