- User Since
- Feb 28 2020, 11:37 PM (74 w, 5 d)
Mon, Aug 2
my hunch is later in the stack (after D9085), this shouldn't be too large of a change, e.g. could potentially toggle is on io manager config?
Fri, Jul 30
Which brings us back to Alex's initial q: what's the right way to toggle memoization on and off?
Thu, Jul 29
in terms of modeling similar "load-from-other-run-in-the-run-group look-ups" in the same way, i could see a path forward that we consolidate logics into something like [get_retry_steps_from_execution_plan](https://github.com/dagster-io/dagster/blob/bfb90e8b0b442f657e5256082d3116aefa8c330b/python_modules/dagster/dagster/core/execution/plan/resume_retry.py#L31) - at a high level, all the paths are basically fn(historical execution plann, logs) -> execution_plan.
Wed, Jul 28
Tue, Jul 27
loosen handle_run_event error
add error handling to in mem storage
yup pipeline failure sensor is already nested under the run status sensor and solid hooks are still linked - i didn't update the failure sensor section in this diff.
Mon, Jul 26
Fri, Jul 23
this is great!
Thu, Jul 22
PipelineSensorCursor -> RunStatusSensorCursor + backcompat fallback
Wed, Jul 21
one alternative would be to extend step_selection to also take an enum entry or marker type for describing a step selection to be calculated by dagster.
- extend step_selection to take StepSelection.FROM_FAILURE), step_selection=StepSelection.from_selection(["some_solid*"]), and ["some_solid*"] (backcompat)
- or a more generic arg reexecute_option and deprecate step_selection, it'd take ReexecuteOption.FROM_FAILURE, ReexecuteOption.from_selection(["some_solid*"]))
Tue, Jul 20
not directly related to this diff: how do people feel about having a separate new-project cli for the crag stuff?
PipelineSensor -> RunStatusSensor
Mon, Jul 19
the make build cmd won't hard fail so BK won't be able to capture it. instead, if anything errors in the middle, it'd print out the errors and results empty json output.
if api docs rendering is good, then it's good to go.
- added test to cover the run interleave case for both run-sharded storage and non-run-sharded
can we make sure the make build in docs still works - iirc we had some lib version related issue when building the api docs. i could be wrong but just wanted to make sure unpinning it wont break the docs build.
Fri, Jul 16
Thu, Jul 15
Wed, Jul 14
For started events, it's expected that I fire the sensor even though by the time I've fired it, the run might have already changed state?
yes bc it's looking at the events not the pipeline statuses, which i think is the desired behavior. alternatively, if we look at the run status, the evaluation may miss firing if the state changes faster than the tick interval.
Tue, Jul 13
Fri, Jul 9
Thu, Jul 8
Wed, Jul 7
Q management. lets do it after moving hacker news to public repo
no context either.. but this looks good to me
Tue, Jul 6
defer to other reviewers as I'm not qualified enough to review the content here
This approach makes sense to me. But let's make sure this content doesn't conflict with the "recommended pattern" below https://docs.dagster.io/concepts/types#associating-dagster-types-with-python-types
lg2m! defer to others for sign off :)