Page MenuHomePhabricator

Fix dagstermill reexecution
ClosedPublic

Authored by max on Jul 29 2019, 9:08 PM.

Details

Reviewers
schrockn
alangenfeld
Group Reviewers
Restricted Project
Commits
R1:0f9733049ba9: Fix dagstermill reexecution
Summary
Test Plan

Unit

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

max created this revision.Jul 29 2019, 9:08 PM
max updated this revision to Diff 3299.Jul 29 2019, 9:48 PM

Actually cache on sub-pipeline creation

max added reviewers: schrockn, Restricted Project.Jul 29 2019, 9:52 PM
max updated this revision to Diff 3300.Jul 29 2019, 9:55 PM

Fix return value

alangenfeld requested changes to this revision.Jul 29 2019, 9:57 PM
alangenfeld added a subscriber: alangenfeld.

looks like arc diff took several of your commits on this one, arc diff again with just the top?

This revision now requires changes to proceed.Jul 29 2019, 9:57 PM
max updated this revision to Diff 3301.Jul 29 2019, 10:01 PM

Rebase

schrockn requested changes to this revision.Jul 29 2019, 10:13 PM

also are all these lint fixes supposed to be in this?

python_modules/dagster/dagster/core/definitions/handle.py
167

I'd prefer a named tuple to an anonymous tuple so we can document what is going on here

This revision now requires changes to proceed.Jul 29 2019, 10:13 PM
max updated this revision to Diff 3304.Jul 29 2019, 10:14 PM

Rebase

schrockn requested changes to this revision.Jul 29 2019, 10:15 PM

can we do namedtuple so we have names?

python_modules/dagster/dagster/core/definitions/handle.py
188

can we do a namedtuple here?

This revision now requires changes to proceed.Jul 29 2019, 10:15 PM
alangenfeld added inline comments.Jul 29 2019, 10:16 PM
python_modules/dagster/dagster/core/definitions/handle.py
300

you could do ** here also if you switch to namedtuple

max added inline comments.Jul 29 2019, 10:17 PM
python_modules/dagster/dagster/core/definitions/handle.py
167

yep

max updated this revision to Diff 3306.Jul 29 2019, 10:23 PM

Use a namedtuple subclass

schrockn accepted this revision.Jul 29 2019, 10:25 PM

lgtm. please heed final comment

python_modules/dagster/dagster/core/definitions/handle.py
94

if you aren't going to do type checking might as well do:

ExecutionTargetHandleCacheEntry = namedtuple(...)

max added inline comments.Jul 29 2019, 10:26 PM
python_modules/dagster/dagster/core/definitions/handle.py
94

hmm, we have this same pattern elsewhere

schrockn added inline comments.Jul 29 2019, 10:27 PM
python_modules/dagster/dagster/core/definitions/handle.py
94

then do a typecheck in the new

max updated this revision to Diff 3309.Jul 29 2019, 10:29 PM

Typecheck

max updated this revision to Diff 3310.Jul 29 2019, 10:30 PM

Fix splat

max updated this revision to Diff 3313.Jul 29 2019, 10:35 PM

Retvalue

This revision is now accepted and ready to land.Jul 29 2019, 10:44 PM
This revision was automatically updated to reflect the committed changes.
Harbormaster failed remote builds in B2645: Diff 3310!