send this up to github so we get the preview

up

Let's add a test for this?

I think this feature might be a good one to put in front of dish next week. While this is incrementally good, I bet we can do a deeper rethink to implement this more elegantly.

up

This is cool

up

up

up

up

up

up

seems quite reasonable to add. will let @alangenfeld handle his objections :-)

clearing my queue. readd if you start pushing on this again

yup. should we do a warning for a bit?

I would also strongly support renaming DagsterInstance.get to deter its usage.

yeah agree that testing for every experiment seems like a bit much, although we could write a test helper that would make the trivial

So one thing here. 1) I don't think we should include this in the top-level includes and 2) I think we should mark it as experimenetal

cool. please look at final comments!

Ha no problem. I'm the worst when it comes to typos.

I would copy and paste into google docs or something for a spell check

much more clear thank

What about language like "we will introduce deprecations every four releases (or so), and there will be a minimum of four releases between deprecation and removal" or something to that effect?

Big point of discussion is are we going to unconditionally up the minor version every 4 weeks. Or should we just mark/determine that approximately every 4 dot releases we can introduce deprecations?

time to bikeshed!

makes sense. test?

A few pieces of feedback/questions:

I can't wait for the project that deems itself a platform for domain-specific package managers.

How does Spark dynamically register new things? E.g. if you "install" delta lake does it dynamically register a new format?

This is a case where I think our "list" approach fails us and we may want to consider a different type of hook and processes one event at a time. For hooks of this nature I'm concerned about long-running computations where getting the events only at the end will feel like a bug. We've spent a bunch of effort making the whole core streaming so want to keep that going.

yeah you're right. i vote for switching. i like autoformatting so much that i sometimes purposefully screw it up so it autofixes and I get the emotional satisfication

I might be wrong here since the hardcoded slack token won't work anyways hmmm

Sent via Superhuman ( https://sprh.mn/?vip=schrockn@elementl.com )

Mock is new and hardcoded is not. However I do think we should consider porting to resource for improved ergonomics, although it is more non-standard and documentable so there are tradeoffs

@yuhan feel free to commandeer if that is helpful

🙏🏻

Cool. I think it's a toss up so let's go with that one and see what user feedback is

makes sense to me

The following problem made me consider the defaults when I was prepping a new workspace.yaml to load in several examples at the same time.

@alangenfeld any real risk with this one? if not can @prha, @dgibson or @max accept? thanks

Great stuff. Please consider final comments as I think it makes code clearer. But your call!

cool.

This looks very good overall. I just wanted to make sure you saw the feedback around getting rid of DagsterInstance.get() calls. I could be missing something, but seems possible to eliminate them here

Also for python_file targets, what do you think about defaulting to the directory where that python files lives?

Can you give a little more context on what the user-behavior facing ends up being in the diff?

❤️

Great stuff. Just a couple minor comments but excited to land this

Per in-person convo:

Third option seems fine. "Users" is just going to be the team and core contributors, so it seems pretty straightfoward to manage and to easily change. Back to your queue

Code is great. Just question on naming

Super clean API. This turned out very nicely.

please heed final comments

ok almost there. most critically we definitely don't want to be mutating the context.

I was also thinking that we would have some config or some way to disable the experimentation warnings for those want to opt into this stuff

__init__ seems fine

for classes would be annotate the __init__ method?