Page MenuHomeElementl
Differential D7406

[dagstermill] support Failure and RetryRequested
ClosedPublic
Actions

Authored by alangenfeld on Apr 13 2021, 4:28 PM.

Details

Reviewers
max
owen
• sashank
Commits
R1:d6502d922d66: [dagstermill] support Failure and RetryRequested
Summary

Papermill doesn't send over the exception object on raise so we have to use yield_event

resolves https://github.com/dagster-io/dagster/issues/4030

Test Plan

added tests

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

alangenfeld created this revision.Apr 13 2021, 4:28 PM
alangenfeld edited the summary of this revision. (Show Details)Apr 13 2021, 4:28 PM
alangenfeld updated this revision to Diff 35308.Apr 13 2021, 4:30 PM

self review

Harbormaster failed remote builds in B28762: Diff 35306!Apr 13 2021, 4:49 PM
Harbormaster completed remote builds in B28764: Diff 35308.Apr 13 2021, 4:53 PM
alangenfeld requested review of this revision.Apr 13 2021, 4:53 PM
schrockn added a subscriber: schrockn.Apr 13 2021, 5:12 PM

To be clear in vanilla solids you still have to raise these right?

alangenfeld added a comment.Apr 13 2021, 7:05 PM

To be clear in vanilla solids you still have to raise these right?

Yep - I tried to match that API here too - but PapermillExecutionException only captures the string name of the raised exception (in addition to context of the cell that raised it) so we are stuck using yield_event

owen accepted this revision.Apr 13 2021, 11:00 PM
This revision is now accepted and ready to land.Apr 13 2021, 11:00 PM
Closed by commit R1:d6502d922d66: [dagstermill] support Failure and RetryRequested (authored by alangenfeld). · Explain WhyApr 14 2021, 4:08 PM
This revision was automatically updated to reflect the committed changes.
alangenfeld added a commit: R1:d6502d922d66: [dagstermill] support Failure and RetryRequested.

Revision Contents
Changeset List

PathSize
python_modules/
dagster/
dagster/
core/
execution/
9 lines
libraries/
dagstermill/
dagstermill/
examples/
notebooks/
21 lines
2 lines
raise_failure.ipynb
hello_world.ipynb
18 lines
raise_retry.ipynb
hello_world.ipynb
6 lines
yield_failure.ipynb
hello_world.ipynb
7 lines
yield_retry.ipynb
hello_world.ipynb
7 lines
14 lines
18 lines
20 lines
dagstermill_tests/
40 lines

Diff 35370

View Options

python_modules/dagster/dagster/core/execution/results.py

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/notebooks/hello_world.ipynb

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/notebooks/hello_world_explicit_yield.ipynb

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/notebooks/raise_failure.ipynb

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/notebooks/raise_retry.ipynb

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/notebooks/yield_failure.ipynb

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/notebooks/yield_retry.ipynb

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/examples/repository.py

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/manager.py

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill/solids.py

Loading...
View Options

python_modules/libraries/dagstermill/dagstermill_tests/test_solids.py

Loading...