Page MenuHomeElementl

Create interface for determining python type of all DagsterTypes
ClosedPublic

Authored by owen on Jun 5 2021, 12:06 AM.

Details

Summary

OUTDATED SUMMARY BELOW:

This is the initial state, which I don't really like. It seems like what we really want to expose is something like a typing_type for each of these. This initially came up in the context of me wanting to write an IOManager that could alter its loading behavior based on what type it will be loaded as downstream. This is trivial if you're using DagsterTypes directly, but if you're relying on the TypeHintInferredDagsterType stuff, you really want to rely on the behavior that if you annotate a solid:

@solid
def my_solid(in1: List[Optional[int]]):
...

Then context.dagster_type.python_type will be equivalent to List[Optional[int]]. However, List and Optional are not valid types, so with this diff it'll actually just give you list. Putting this diff out just as a point of reference, but I'll probably try to go down the typing_type route.

Test Plan

bk

Diff Detail

Repository
R1 dagster
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 5 2021, 12:29 AM
Harbormaster failed remote builds in B31744: Diff 39092!
owen requested review of this revision.Jun 7 2021, 6:59 PM
owen added a reviewer: sandyryza.
python_modules/dagster/dagster/core/types/dagster_type.py
99

Can we add an entry to the doc above?

550

If the provided python_type is a tuple of types, this won't be a valid typing type - is that a problem? I don't know if there's an easy way to construct a Union type at runtime?

python_modules/dagster/dagster/core/types/dagster_type.py
550

Good catch! I don't think this is actually a huge problem, but it is worth fixing (and is possible to do so, so why not)

add a test case for the latest tuple update

This revision is now accepted and ready to land.Jun 8 2021, 7:25 PM
owen marked 2 inline comments as done.

up

This revision was landed with ongoing or failed builds.Jun 8 2021, 9:29 PM
This revision was automatically updated to reflect the committed changes.