Page MenuHomeElementl

Implemented request/retry/failure for input/output managers
ClosedPublic

Authored by cdecarolis on Dec 16 2020, 11:23 PM.

Details

Summary

This should enable Retrying and Failure from inside the body of input and output managers.

Test Plan

Unit tests

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Dec 16 2020, 11:37 PM
Harbormaster failed remote builds in B23061: Diff 28041!

I like it! The behavior of retrying /failing the entire step (not just the particular input or output) seems right to me. Interested to hear @alangenfeld 'd take.

python_modules/dagster/dagster/core/execution/plan/execute_step.py
530

We can use f-strings now that we're on python 3.

Definitely agree that these events (Failure, RetryRequested) should apply to the entire step

python_modules/dagster/dagster_tests/core_tests/storage_tests/test_input_manager.py
235

don't think the TODO is necessary. I actually think this is more clear for a test case

Change error messages to f strings

ya whole step retry for sure

sandyryza added inline comments.
python_modules/dagster/dagster/core/errors.py
243

Might be easier for users to connect with load_input if were called DagsterExecutionLoadInputError?

Not a strongly held opinion though.

This revision is now accepted and ready to land.Dec 21 2020, 7:41 PM