Page MenuHomePhabricator

sandyryza (Sandy Ryza)
User

Projects

User does not belong to any projects.

User Details

User Since
Apr 3 2020, 4:04 PM (42 w, 2 d)

Recent Activity

Sat, Jan 23

sandyryza added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

I like the variants you listed that involve overloading the input_defs arg on solid.  I think that kind of overloading is pretty Pythonic / expected. When reading that code, it's very difficult to misinterpret what's going on.

Sat, Jan 23, 7:07 PM
sandyryza added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

@schrockn those concerns make a lot of sense.

Sat, Jan 23, 1:23 AM

Fri, Jan 22

sandyryza added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

@schrockn do you have thoughts here? I think I would be OK with any of the options raised by @alangenfeld. I vaguely recall that you had some concerns about getting fancy with auto-creation/parameterization of types.

Fri, Jan 22, 10:26 PM
sandyryza planned changes to D5934: file IO manager.
Fri, Jan 22, 10:23 PM
sandyryza added inline comments to D6076: [mypy][core] execution/plan.
Fri, Jan 22, 5:20 PM
sandyryza accepted D6076: [mypy][core] execution/plan.

It would be nice to have a better story for these named tuples. Unfortunately, it might require mucking around with metaclasses to support overriding init if we want to do runtime type checking.

Fri, Jan 22, 5:19 PM
sandyryza accepted D6106: pin mypy.
Fri, Jan 22, 4:16 PM
sandyryza accepted D6099: Rename tests.
Fri, Jan 22, 1:54 AM

Thu, Jan 21

sandyryza accepted D6067: [mypy][core] mypy typing for configurable and config definitions.
Thu, Jan 21, 6:51 PM
sandyryza added inline comments to D6082: [mypy][core] mypy typing for definitions/dependency.py.
Thu, Jan 21, 6:51 PM
sandyryza accepted D6085: get_output_manager -> get_io_manager.
Thu, Jan 21, 6:31 PM
sandyryza added inline comments to D6067: [mypy][core] mypy typing for configurable and config definitions.
Thu, Jan 21, 1:03 AM

Wed, Jan 20

sandyryza requested review of D5934: file IO manager.
Wed, Jan 20, 11:24 PM
sandyryza added inline comments to D6058: [mypy][core] definitions/decorators/composite_solid.py mypy.
Wed, Jan 20, 8:00 PM
sandyryza accepted D6064: Document the file managers.
Wed, Jan 20, 7:50 PM
sandyryza added inline comments to D6058: [mypy][core] definitions/decorators/composite_solid.py mypy.
Wed, Jan 20, 5:11 PM
sandyryza accepted D6046: [Memoized Dev Example 1/2]Add Memoizable IO Manager to API Docs, fix typo in Memoizable IO Manager documentation..
Wed, Jan 20, 1:32 AM
sandyryza updated the diff for D6033: include system framing message in user errors.

up

Wed, Jan 20, 1:32 AM
sandyryza added a comment to D6033: include system framing message in user errors.

what led away from the approach in https://dagster.phacility.com/D5784 ? Just updating test call sites?
I think just raising the outer error and improving our display / printing of it makes the most sense to me.

Wed, Jan 20, 1:28 AM

Tue, Jan 19

sandyryza closed D5978: remove define_solid_config_cls, which is only used by tests.
Tue, Jan 19, 7:05 PM
sandyryza committed R1:bd5b0e9ea88c: remove define_solid_config_cls, which is only used by tests (authored by sandyryza).
remove define_solid_config_cls, which is only used by tests
Tue, Jan 19, 7:05 PM
sandyryza requested review of D5978: remove define_solid_config_cls, which is only used by tests.
Tue, Jan 19, 5:28 PM
sandyryza resigned from D5321: Different scenarios to consider when doing input/output management.
Tue, Jan 19, 4:38 PM
sandyryza requested changes to D5988: [NOT release-blocking] RFC convert OBJECT_STORE_OPERATION to LOADED_INPUT/HANDLED_OUTPUT.

I agree that, given that converting from ObjectStoreOperation to HandledOutput / LoadedInput would be lossy, probably best to punt.

Tue, Jan 19, 4:37 PM
sandyryza added reviewers for D6036: Make type_check_fn optional on DagsterType: schrockn, alangenfeld, max.
Tue, Jan 19, 4:35 PM

Sat, Jan 16

sandyryza requested review of D5115: don't error on solid type annotations that don't resolve to dagster types.
Sat, Jan 16, 5:16 PM
sandyryza requested review of D6036: Make type_check_fn optional on DagsterType.
Sat, Jan 16, 5:14 PM
sandyryza updated the diff for D5794: pyspark smoke test example.

up

Sat, Jan 16, 5:01 PM

Fri, Jan 15

sandyryza added reviewers for D6033: include system framing message in user errors: alangenfeld, nate, yuhan.
Fri, Jan 15, 9:59 PM
sandyryza updated the diff for D6033: include system framing message in user errors.

up

Fri, Jan 15, 9:35 PM
sandyryza requested review of D6033: include system framing message in user errors.
Fri, Jan 15, 9:32 PM
sandyryza accepted D6032: [mypy][core] execution/context.
Fri, Jan 15, 8:44 PM
sandyryza added inline comments to D6032: [mypy][core] execution/context.
Fri, Jan 15, 6:50 PM
sandyryza added a comment to D5977: update nothing docs now that we have IO managers.

shouldn't this be modeled w/ IOManagers instead of Nothings?

Fri, Jan 15, 4:36 PM
sandyryza accepted D6020: [mypy][core] execution api.py and active.py.
Fri, Jan 15, 4:34 PM
sandyryza added a comment to D6020: [mypy][core] execution api.py and active.py.

Hell yeah!

Fri, Jan 15, 4:34 PM

Thu, Jan 14

sandyryza closed D6018: update partitions and backfills doc with screenshots of new UI.
Thu, Jan 14, 11:23 PM
sandyryza committed R1:d5f4588d49b7: update partitions and backfills doc with screenshots of new UI (authored by sandyryza).
update partitions and backfills doc with screenshots of new UI
Thu, Jan 14, 11:23 PM
sandyryza updated the diff for D6018: update partitions and backfills doc with screenshots of new UI.

up

Thu, Jan 14, 11:05 PM
sandyryza accepted D6011: Remove dangling references to SystemStorageDefinition.
Thu, Jan 14, 9:25 PM
sandyryza closed D6002: mark input and output metadata experimental.
Thu, Jan 14, 7:57 PM
sandyryza committed R1:8a7a39d42b42: mark input and output metadata experimental (authored by sandyryza).
mark input and output metadata experimental
Thu, Jan 14, 7:57 PM
sandyryza updated the diff for D6002: mark input and output metadata experimental.

up

Thu, Jan 14, 7:26 PM
sandyryza updated the diff for D6002: mark input and output metadata experimental.

up

Thu, Jan 14, 7:05 PM
sandyryza accepted D5981: Deprecate FileManager and FileHandle #3535.

Looks good!

Thu, Jan 14, 4:27 PM
sandyryza accepted D5984: context.log paths in default IO manager implementations #3531.

looks good!

Thu, Jan 14, 1:56 AM
sandyryza accepted D5990: keep AssetStoreOperationType in GRAVEYARD.
Thu, Jan 14, 1:26 AM

Wed, Jan 13

sandyryza closed D5980: mark root input managers as experimental.
Wed, Jan 13, 11:57 PM
sandyryza committed R1:35638fcedeb4: mark root input managers as experimental (authored by sandyryza).
mark root input managers as experimental
Wed, Jan 13, 11:57 PM
sandyryza closed D5967: improve IO manager docstrings.
Wed, Jan 13, 11:56 PM
sandyryza committed R1:b51b3ae5d3ae: improve IO manager docstrings (authored by sandyryza).
improve IO manager docstrings
Wed, Jan 13, 11:56 PM
sandyryza closed D5942: deprecate has_configurable_inputs, has_configurable_outputs, and has_config_entry.
Wed, Jan 13, 11:47 PM
sandyryza committed R1:075ad535ecc3: deprecate has_configurable_inputs, has_configurable_outputs, and… (authored by sandyryza).
deprecate has_configurable_inputs, has_configurable_outputs, and…
Wed, Jan 13, 11:47 PM
sandyryza requested review of D5980: mark root input managers as experimental.
Wed, Jan 13, 10:43 PM
sandyryza requested review of D5977: update nothing docs now that we have IO managers.
Wed, Jan 13, 10:22 PM
sandyryza updated the diff for D5942: deprecate has_configurable_inputs, has_configurable_outputs, and has_config_entry.

up

Wed, Jan 13, 10:22 PM
sandyryza updated the diff for D5967: improve IO manager docstrings.

up

Wed, Jan 13, 10:15 PM
sandyryza updated the diff for D5967: improve IO manager docstrings.

up

Wed, Jan 13, 10:14 PM
sandyryza accepted D5970: 4/ Clarify return types and interfaces.

I love it.

Wed, Jan 13, 9:11 PM
sandyryza accepted D5971: 5/ Upgrade moto.
Wed, Jan 13, 9:07 PM
sandyryza requested review of D5967: improve IO manager docstrings.
Wed, Jan 13, 7:58 PM
sandyryza accepted D5955: 0.10.0 API removals.

noice

Wed, Jan 13, 7:42 PM
sandyryza accepted D5963: 1/ Modernize airline_demo setup.

Looks good. Left a comment.

Wed, Jan 13, 7:41 PM
sandyryza closed D5918: manager_key -> io_manager_key.
Wed, Jan 13, 6:30 PM
sandyryza committed R1:936cd12bc414: manager_key -> io_manager_key (authored by sandyryza).
manager_key -> io_manager_key
Wed, Jan 13, 6:30 PM
sandyryza updated the diff for D5918: manager_key -> io_manager_key.

up

Wed, Jan 13, 6:29 PM
sandyryza requested review of D5942: deprecate has_configurable_inputs, has_configurable_outputs, and has_config_entry.
Wed, Jan 13, 5:27 PM
sandyryza updated the diff for D5918: manager_key -> io_manager_key.

up

Wed, Jan 13, 5:10 PM
sandyryza accepted D5948: [docs] mention IO manager docs in the migration guide #3455.
Wed, Jan 13, 5:05 PM
sandyryza accepted D5947: cleanup memoization copy_required_intermediates_for_execution.

That was quick! Love the big red portion.

Wed, Jan 13, 4:56 PM
sandyryza updated the diff for D5918: manager_key -> io_manager_key.

up

Wed, Jan 13, 4:49 PM
sandyryza closed D5874: test object manager composites behavior.
Wed, Jan 13, 1:51 AM
sandyryza committed R1:a630449903cb: test object manager composites behavior (authored by sandyryza).
test object manager composites behavior
Wed, Jan 13, 1:51 AM
sandyryza closed D5925: InputManager -> RootInputManager.
Wed, Jan 13, 1:50 AM
sandyryza committed R1:6e5c16256411: InputManager -> RootInputManager (authored by sandyryza).
InputManager -> RootInputManager
Wed, Jan 13, 1:50 AM
sandyryza accepted D5937: rename s3_io_manager to s3_pickled_io_manager, adls2_io_manager to adls2_pickled_io_manager, and gcs_io_manager to gcs_pickled_io_manager..

oops meant to hit accept

Wed, Jan 13, 1:49 AM
sandyryza updated the diff for D5874: test object manager composites behavior.

up

Wed, Jan 13, 1:16 AM
sandyryza updated the diff for D5925: InputManager -> RootInputManager.

up

Wed, Jan 13, 1:10 AM
sandyryza updated the summary of D5918: manager_key -> io_manager_key.
Wed, Jan 13, 1:05 AM
sandyryza updated the diff for D5925: InputManager -> RootInputManager.

up

Wed, Jan 13, 1:01 AM
sandyryza added inline comments to D5925: InputManager -> RootInputManager.
Wed, Jan 13, 1:01 AM

Tue, Jan 12

sandyryza updated subscribers of D5937: rename s3_io_manager to s3_pickled_io_manager, adls2_io_manager to adls2_pickled_io_manager, and gcs_io_manager to gcs_pickled_io_manager..

CC @schrockn in case he wants to weigh in.

Tue, Jan 12, 10:42 PM
sandyryza accepted D5474: ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput.
Tue, Jan 12, 10:28 PM
sandyryza accepted D5922: [docs] Eliminate 'intermediates' in docs #3516.

A couple small comments - otherwise looks great!

Tue, Jan 12, 10:17 PM
sandyryza requested review of D5918: manager_key -> io_manager_key.
Tue, Jan 12, 9:41 PM
sandyryza accepted D5474: ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput.

lgtm!

Tue, Jan 12, 9:39 PM
sandyryza closed D5924: unexpose output managers.
Tue, Jan 12, 9:37 PM
sandyryza committed R1:817b31a29425: unexpose output managers (authored by sandyryza).
unexpose output managers
Tue, Jan 12, 9:37 PM
sandyryza abandoned D3588: (value-resource-3) Add value_resource.
Tue, Jan 12, 9:35 PM
sandyryza closed D5852: mypy everything (almost).
Tue, Jan 12, 9:17 PM
sandyryza committed R1:4c721ea7b5c4: mypy everything (almost) (authored by sandyryza).
mypy everything (almost)
Tue, Jan 12, 9:17 PM
sandyryza requested review of D5924: unexpose output managers.
Tue, Jan 12, 7:48 PM
sandyryza requested review of D5925: InputManager -> RootInputManager.
Tue, Jan 12, 7:47 PM
sandyryza updated the diff for D5852: mypy everything (almost).

up

Tue, Jan 12, 7:27 PM
sandyryza closed D5843: root input manager.
Tue, Jan 12, 6:03 PM
sandyryza committed R1:ae09fcdb1a03: root input manager (authored by sandyryza).
root input manager
Tue, Jan 12, 6:03 PM
sandyryza updated the diff for D5843: root input manager.

up

Tue, Jan 12, 5:18 PM
sandyryza added a reviewer for D5780: eliminate Persistable: max.
Tue, Jan 12, 4:51 PM
sandyryza updated the summary of D5780: eliminate Persistable.
Tue, Jan 12, 4:51 PM
sandyryza requested changes to D5474: ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput.
Tue, Jan 12, 4:48 PM