Page MenuHomeElementl

mappable impl
Needs ReviewPublic

Authored by catherinewu on Oct 28 2020, 3:51 PM.
This revision needs review, but there are no reviewers specified.

Details

Reviewers
None
Test Plan

bk

Diff Detail

Repository
R1 dagster
Branch
mappable_pipeline
Lint
Lint Warnings
SeverityLocationCodeMessage
Warningpython_modules/dagster/dagster/core/execution/plan/compute.py:111W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/execute_step.py:58W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/execute_step.py:69W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/objects.py:12W0611Unused Import
Warningpython_modules/dagster/dagster/core/execution/plan/objects.py:358W0222Signature Differs
Warningpython_modules/dagster/dagster/core/execution/plan/objects.py:407W0612Unused Variable
Warningpython_modules/dagster/dagster/core/execution/plan/plan.py:95W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/plan.py:177W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/plan.py:238W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/plan.py:242W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/plan.py:384W0002Print Call
Warningpython_modules/dagster/dagster/core/execution/plan/plan.py:385W0002Print Call
Warningpython_modules/dagster/dagster/core/executor/multiprocess.py:217W0002Print Call
Warningpython_modules/dagster/dagster/core/storage/intermediate_storage.py:155W0002Print Call
Warningpython_modules/dagster/dagster_tests/core_tests/execution_tests/test_mappable.py:1W0611Unused Import
Warningpython_modules/dagster/dagster_tests/core_tests/execution_tests/test_mappable.py:1W0611Unused Import
Warningpython_modules/dagster/dagster_tests/core_tests/execution_tests/test_mappable.py:1W0611Unused Import
Warningpython_modules/dagster/dagster_tests/core_tests/execution_tests/test_mappable.py:1W0611Unused Import
Unit
No Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 28 2020, 4:18 PM
Harbormaster failed remote builds in B20308: Diff 24639!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 28 2020, 6:11 PM
Harbormaster failed remote builds in B20330: Diff 24663!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 28 2020, 6:37 PM
Harbormaster failed remote builds in B20336: Diff 24670!
Harbormaster returned this revision to the author for changes because remote builds failed.Oct 28 2020, 7:05 PM
Harbormaster failed remote builds in B20342: Diff 24676!

support dagit/graphql "retry from failure" use case

prevent graphql error that happens during re-exec

no show stoppers i can see - figuring out a clean way to do what is currently handled by deconstruct_step_key will be an interesting challenge

python_modules/dagster/dagster/core/definitions/output.py
99–103

nit: is_mappable not a constructor arg but just result of overridden method

python_modules/dagster/dagster/core/events/__init__.py
203–204 ↗(On Diff #25124)

would expect this to go be in event_specific_data

python_modules/dagster/dagster/core/execution/memoization.py
63–67 ↗(On Diff #25124)

one value of having separate types is failing check.inst calls when mappable things show up where they are not expected

python_modules/dagster/dagster/core/execution/plan/objects.py
164–234

think there is some value in modeling the unresolved input more clearly as a separate thing - this class is already a bit confusing and having the mappable / resolve like this makes it even more confusing

357

definitely weary of using inheritance here

python_modules/dagster/dagster/core/execution/plan/plan.py
311–330

will have to come up with a clean way to model this all and update callsites accordingly