Page MenuHomePhabricator

RFC: rough sketch of compute function retries
Needs RevisionPublic

Authored by natekupp on Wed, Sep 11, 1:12 AM.

Details

Summary

Not for check-in - this is a rough sketch of how I'm thinking of implementing compute function retry logic.

I would need to plumb through the configuration (e.g. max retries but also time delay / exponential backoff approach) - wanted to get feedback on doing things this way before proceeding.

I think we probably have to implement the retry semantics at the bottom of the stack here vs. at the top level where we materialize events from the nested generators, but open to other ideas

Test Plan

n/a

Diff Detail

Repository
R1 dagster
Branch
mvp_retries
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

natekupp created this revision.Wed, Sep 11, 1:12 AM
natekupp retitled this revision from RFC: rough implementation of compute function retries to RFC: rough sketch of compute function retries.Wed, Sep 11, 1:21 AM
natekupp edited the summary of this revision. (Show Details)
natekupp added reviewers: alangenfeld, schrockn.

hm did you investigate making the change at the compilation step ie step.compute_fn ? I guess the question is whether or not the engines should be aware of the retries or not.

schrockn requested changes to this revision.Wed, Sep 11, 10:21 PM

going to do run-based retry first yes?

This revision now requires changes to proceed.Wed, Sep 11, 10:21 PM