- User Since
- Mar 20 2019, 8:25 PM (117 w, 6 d)
Mon, Jun 21
Fri, Jun 18
unclear if inheritance is the way to go with In and Out once there is this behavior difference. Might be best to have a simple value object and interpret it in to the underlying def, maybe with a scheme like 
check inlines before landing - i think its worth cleaning up
i thiiink this is fine but will swap out for folks who have done practitioner-ing to weigh in
lets defer dict style support to a follow up
Accept a dict of Ins instead of a List. I went in thinking I would do this, but came out not thinking it was particularly better. The number of characters a user needs to type is identical. It doesn't work for Outs, because, if we want the tuple-stuff mentioned above to work, the Outs need to be ordered, and supplying an OrderedDict is a pain.
the Outs need to be ordered, and supplying an OrderedDict is a pain
Tue, Jun 15
Mon, Jun 14
should add a test for lazy job construction with partition too
works for me
Fri, Jun 11
It seems odd to provide this and not just a name param
test, lint, mypy
Add some other reviewers so that hopefully this is the last rename diff
maybe mention "crag" in the repository location that loads these repos if you add it to the root workspace (Which i think makes sense to do)
Thu, Jun 10
would like @prha or someone who has spent more time thinking about partitions to weigh in on if this is the right API for the future
where do we do partition set look ups? Is there a way to store a different type of targeting on the object that refers to it, like JobScoped instead of RepoScoped so you go get the Job and fetch it of there instead of getting it out of the repo?
Will this config be used when invoking the job from a schedule or sensor? I had been imagining that it would.
would be nice to have a mapping alembic version to dagster version
well we should at least test for sqlite and PG reseting various snapshots back to base and then doing an instance migrate.
is this safe? are all of our migrations idempotent? I faintly remember doing this manually and was getting errors on various migrations so I had to find the actual version right version to start from
Wed, Jun 9
flexible typing got us
some fixes for dagit
alright should be good to go
I think yes, we are still validating inputs right?
I think this is a bit cleaner, thanks for the quick turn around
Tue, Jun 8
we could obfuscate or add a layer of indirection if we wanted
I think the main thing worth discussing here is that this shifts ModeDefinitions from objects that can live inside multiple pipelines to objects that are specific to a single pipeline
In legacy non-crag world, do we think its legitimate for a user to use the config_mapping arg on ModeDefinition, or is this just a hack to get crag working that doesn't require adding JobDefinition, etc.?
request review if you disagree with proposed alternative approach
if we did this we would want it behind an opt-in gating of some kind right?
rebase and take a pass on initial feedback?
I guess if the decorated fxn isn't a generator we can validate, and then if it is a generator, we wrap it in another generator that performs the validation?
add a test case for the latest tuple update
follow up diff tho
we would ban them on composite solids for consistency
nit: mypy the new functions
its a bit complex but we *could* do output validation if we wanted
drop the attempt
Mon, Jun 7
I understand this is a bit of a spicy change, but I thought I would at least put it out there.
see how retry_attempt_number feels