- User Since
- Mar 20 2019, 8:23 PM (101 w, 5 d)
Fri, Feb 19
screenshot looks good. some of those events are deprecated and we should consider not rendering them
Thu, Feb 18
"My interpretation of the the feedback we got was that "data flow between reusable, logical components" is too abstract to convey useful information. I also think we should downplay reusability in our core messaging - in most cases, writing reusable solids is not the right pattern."
Mon, Feb 15
Tue, Feb 9
mixing await and yield is a bit odd. is that normative in python?
Tue, Feb 2
did you consider process-level isolation?
Mon, Feb 1
Did we ever consider breaking tags into its own column?
Jan 29 2021
Sent via Superhuman ( https://firstname.lastname@example.org )
definitely think we should keep prefix like @dgibson. grepability is good
Very much support getting rid of the registry. Thanks for doing this! Let's just try to mitigate risk as much as possible with a diff this large
Jan 27 2021
I meant code artifact, like "solid", "pipeline", "resource" etc
Agree with nearly all of sandy's and rex's feedback.
Jan 24 2021
Very excited to see this land. Will defer to dish on the deets of the js.
Jan 23 2021
adding @sandyryza here. I think this makes a lot of sense and I don't think having the events be left-to-right and the graphs right-to-left is a big problem but open to other's perspective
"I'm pretty resistant to directions that involve using type annotations for anything other than annotating the actual Python type that's expected somewhere. It's fairly abusive of a Python language feature, and it causes non-trivial trouble for anyone who's trying to use that language feature in the correct way."
We could also get clever and try to reuse input_defs and output_defs supporting a few different variants:
I'm concerned about too many breaking changes and too much thrash. But I think that the critical thing here would be coming up with a more condense syntax for inputs and outputs. Right now if we forced users to InputDefinition and OuptutDefinition for the dagster-type-only case the code explosion would be brutal.
Jan 22 2021
Yeah my first instinct is stress and terror given our past forays into magically commingling the python and dagster type system.
Jan 19 2021
Here's a thought:
Jan 14 2021
Jan 13 2021
can you display pipeline name in addition to schedule name?
excited to see this land. I will let @prha look at the lint ish as I have little context on that
could you throw up a screenshot?
Jan 12 2021
"hardcoded_resource_instances" is a pretty rough name since they aren't really hardcoded. they are not fixed in code. maybe "resource_instances_to_override" or something?
Seems good. Will let others do a detailed review!
Jan 8 2021
just doing q mgmt. any other alternative names?
is this still a relevant diff? req'ing changes for q mgmt
So just brainstorming here a bit. Another approach might be special-casing at a different spot.
How fixed are we on the term "root"? I'm trying to recall other words that we have used or would use to describe inputs that have no upstream. I just want to make sure this is the best we have come up with.
Jan 7 2021
lgtm. just want to know about the linting issue
let's kill forEach
Very happy with the tracking changes
Excellent. Really exciting move forward. I had some minor comments around places where you can be a little more defensive, but I think this ended up at a nice place
this is great. very clearly written
will let you guys handle this
Jan 6 2021
oof ok I didn't think about that deeply enough. that definitely needs to work
looking good! will let someone else do final accept
This really is lovely. Excited to see it land.
Jan 5 2021
cool stuff. looking forward to this landing!
Jan 4 2021
to be clear i love the new syntax.
I don't think we should eliminate the check calls until we turn on mypy typing
seems like we could make a UI affordance the use case where you turn on the schedule without a daemon process running
Please see comment on https://dagster.phacility.com/D5768 I would like to have a plan and messaging in place for the team before we merge this.
So before you and @max start to do this I would like to see 1) a plan for adopting this and goals around it and 2) language that we will announce to the team around expectations here. E.g. is all new code typed? who is responsible for migrating? what is the overall plan?
Dec 28 2020
Dec 23 2020
Never assume anything is happening locally if it doesn’t happen in the ci/cd pipeline :-)
Dec 22 2020
Yeah would like to have a clear plan for actually turning on mypy. Adding typehints without automated checks is worse than having no typehints imo
I can either revert it or we can add the backwards compat variant. Your call!
Dec 21 2020
Since you already have the profiling set up, do you have a notion what the hot paths are for the remaining 9 seconds?
I would rather eliminate it as I anticipate that we will add resources other artifacts like sensors to enable local dev and testing. This change makes this abstraction applicable across other axes than just graphs
I've been thinking about this a bunch and name issue continues to gnaw at me. configured, while elegant, expose this problem but it is this diff that in particularly makes it more acute.
Very concerned about using the step key DSL in _update_tracking_dict and friends. I think should be persisting the structured step key information DagsterEvent to reduce the surface area of our reliance on the DSL
Thanks for your patience in terms of me getting to this:
I have fairly grave concerns about this. I think we open ourselves to a lot of chicanery.
Definitely agree that these events (Failure, RetryRequested) should apply to the entire step
Dec 19 2020
Dec 15 2020
cool. plz consider final comments
Yeah I was thinking like a week of historical data or something.
Dec 14 2020
I think this is excellent. I have some fairly minor comment which I'd love to see considered. Please check them out prior to landing.
arc patch works great thanks!