hmm always dubious of bool input args, wonder how else we could set this up

I think its worth considering framing this more abstractly. This same code could be used to prevent re-computation in cases that are not resume-retry. IE if i have some scheduled job and none of the inputs change for some subset i can copy the last results over instead of recomputing. It might be too difficult to abstract this now but its worth thinking through what this is doing from first principles and considering renaming/restructuring it along those lines instead of its current sole intended use case.

huh cute - i thought we would have to have dealt with success and fails in the previous run instead of just what intermediates are available. We should maybe test a solid that yields multiple outputs and then fails. Do we overwrite the copied intermediates when we rerun? Do we not copy those intermediates?

Good call, I'll add a test for this.

In general, there's a lot of subtlety into what the right behavior for retries is. You're right that it should not just be based on the successful generation of all the output (we should take into account the general step success/failure). Should it also take into account materializations? There were a couple test cases in our test suite that had solids performing undocumented side-effects (see test_basic_solid_with_config in dagster_tests/core_tests/test_solid_with_config.py) that made me think about where we might be challenging some assumptions.

I wonder if step_keys_to_execute / step_keys_to_skip could be properties of the execution plan? I could imagine us annotating the execution plan with various other metadata about how we want things to be executed down in the engines, seems like it'd be nice to encapsulate all of that in a single object and then hand it off to the engines to do with it as they will.

thoughts?

hmm, what happens here for a step key that's downstream of a failed task? I guess we're only skipping execution for steps that previously executed and therefore have their outputs covered?

maybe add check.inst_ calls here?

what is the exact relationship between stepKeys and forceReexecutionStepKeys? What would it look like to nuke reexecutionConfig and just express sufficient intent via executionParams?

this feels a bit off at a mental model level - maybe just the name generate_events on MemoizationStrategy. Is this just to generate the intermediate store operation events?

nice

might be better to set this up as a factory - especially if we derive the memoization subtype via the run_config

I was thinking that stepKeys and forceReexecutionStepKeys could be different, in that stepKeys is the set of keys we want to show in the run (but could be skipped due to memoization) and forceReexecutionStepKeys would be the set of keys that we wanted to compute regardless of the previous' runs success state.

In reality, these two are generally kept in lock-step for single-step reexecution.

For resume/retries, we could merge the two concepts, but that would mean that we'd show the step status as not having run at all instead of being skipped. Thoughts?

Yeah, in order to have retries of retries, we need to generate the intermediate store operation events so that the next retry can pull the results.

We need the event logs from the previous run, and it felt bad to fetch that directly in the execution api / engine.

ya just changing the names will make this feel better (to me):

for store_event in memoization_strategy.manage_intermediates(execution_plan):
  yield store_event

I like the goal of merging - its unclear that the subtle behavior achieved by having them separate is worth the confusion