Page MenuHomeElementl

sandyryza (Sandy Ryza)
User

Projects

User does not belong to any projects.

User Details

User Since
Apr 3 2020, 4:04 PM (63 w, 4 d)

Recent Activity

Yesterday

sandyryza added a comment to D8468: feat: add support for dict of ins and outs.

Do Ins/Outs need to have names if we're going with the dict approach?

Tue, Jun 22, 3:51 AM

Mon, Jun 21

sandyryza added a comment to D8387: RFC: single asset sensor definition.

Overall I am on board with this. It still needs docs etc. and I left a comment on one of the details of the API.

Mon, Jun 21, 9:52 PM
sandyryza accepted D8484: Allow classes to be decorated with @experimental.

nice! I was hoping someone would add this

Mon, Jun 21, 6:21 PM
sandyryza added a comment to D8484: Allow classes to be decorated with @experimental.

Mind adding a test for a class that defines an init fn?

Mon, Jun 21, 5:30 PM
sandyryza added a comment to D8435: In and Out.

@schrockn I’m very on board with the dict format now that I know that 3.6+ dicts are ordered.

Mon, Jun 21, 5:34 AM
sandyryza requested review of D8465: [RFC] daemon logger.
Mon, Jun 21, 5:25 AM

Thu, Jun 17

sandyryza closed D8445: in hardcoded_resource doc, don't imply its only useful in tests.
Thu, Jun 17, 10:02 PM
sandyryza committed R1:a365f58d1dde: in hardcoded_resource doc, don't imply its only useful in tests (authored by sandyryza).
in hardcoded_resource doc, don't imply its only useful in tests
Thu, Jun 17, 10:02 PM
sandyryza requested review of D8445: in hardcoded_resource doc, don't imply its only useful in tests.
Thu, Jun 17, 9:54 PM
sandyryza updated the diff for D8426: op.

up

Thu, Jun 17, 9:29 PM
sandyryza updated the diff for D8426: op.

up

Thu, Jun 17, 9:06 PM
sandyryza requested review of D8435: In and Out.
Thu, Jun 17, 9:04 PM
sandyryza accepted D8431: [docs] add testing section for hooks.
Thu, Jun 17, 5:34 PM
sandyryza accepted D8432: audit documentation for context builder apis.
Thu, Jun 17, 4:25 PM
sandyryza updated the diff for D8425: add graph to top-level imports.

up

Thu, Jun 17, 3:48 PM
sandyryza requested review of D8426: op.
Thu, Jun 17, 2:15 AM
sandyryza requested review of D8425: add graph to top-level imports.
Thu, Jun 17, 1:20 AM

Wed, Jun 16

sandyryza accepted D8416: feat: add default val for step_key and name on build_output_context.
Wed, Jun 16, 8:29 PM
sandyryza accepted D8383: ensure legacy `Materialization` event has a default tags value.
Wed, Jun 16, 6:34 PM
sandyryza closed D8378: add a job with a sensor to the crag toys and add toys to top-level workspace.yaml.
Wed, Jun 16, 4:55 PM
sandyryza committed R1:123ce9d4eaa6: add a job with a sensor to the crag toys and add toys to top-level workspace. (authored by sandyryza).
add a job with a sensor to the crag toys and add toys to top-level workspace.
Wed, Jun 16, 4:55 PM
sandyryza added a comment to D8385: RFC: add instance log manager interface.

Does this change imply that daemon output streams should get handled by the same InstanceLogManager as the step process output streams? Or will they be separately configurable?

Wed, Jun 16, 12:32 AM

Tue, Jun 15

sandyryza closed D8283: [crag] partitions on modes.
Tue, Jun 15, 9:46 PM
sandyryza committed R1:cc5e781dcbe5: [crag] partitions on modes (authored by sandyryza).
[crag] partitions on modes
Tue, Jun 15, 9:46 PM
sandyryza updated the diff for D8283: [crag] partitions on modes.

up

Tue, Jun 15, 8:59 PM
sandyryza accepted D8361: make step_key and name optional on output context and build_output_context.
Tue, Jun 15, 8:29 PM
sandyryza requested review of D8378: add a job with a sensor to the crag toys and add toys to top-level workspace.yaml.
Tue, Jun 15, 4:56 PM
sandyryza updated the diff for D8283: [crag] partitions on modes.

up

Tue, Jun 15, 4:53 PM
sandyryza closed D8356: don't require context arg when solid has required resources.
Tue, Jun 15, 4:29 PM
sandyryza committed R1:4a0806894e7e: don't require context arg when solid has required resources (authored by sandyryza).
don't require context arg when solid has required resources
Tue, Jun 15, 4:29 PM
sandyryza added inline comments to D8356: don't require context arg when solid has required resources.
Tue, Jun 15, 4:18 PM
sandyryza requested changes to D8362: rename SensorExecutionContext to SensorEvaluationContext.

requesting changes for q management

Tue, Jun 15, 4:11 PM
sandyryza added inline comments to D8283: [crag] partitions on modes.
Tue, Jun 15, 4:01 PM
sandyryza accepted D8372: expose PipelineRunStatus as a top-level API.

bulbasaur

Tue, Jun 15, 3:32 PM
sandyryza added a comment to D8362: rename SensorExecutionContext to SensorEvaluationContext.

I like SensorEvaluationContext and ScheduleEvaluationContext too.

Tue, Jun 15, 5:46 AM
sandyryza closed D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.
Tue, Jun 15, 1:24 AM
sandyryza committed R1:6f0520630762: [crag] a set of pipelines and schedules to prototype cragit UI on top of (authored by sandyryza).
[crag] a set of pipelines and schedules to prototype cragit UI on top of
Tue, Jun 15, 1:24 AM
sandyryza added inline comments to D8356: don't require context arg when solid has required resources.
Tue, Jun 15, 1:23 AM
sandyryza closed D8344: [crag] job name.
Tue, Jun 15, 1:22 AM
sandyryza committed R1:e09712d42073: [crag] job name (authored by sandyryza).
[crag] job name
Tue, Jun 15, 1:22 AM

Mon, Jun 14

sandyryza requested review of D8356: don't require context arg when solid has required resources.
Mon, Jun 14, 8:51 PM
sandyryza accepted D8331: [docs] add resource testing section.
Mon, Jun 14, 8:40 PM
sandyryza closed D8355: mark to_job as experimental.
Mon, Jun 14, 8:34 PM
sandyryza committed R1:5469ba2fee4f: mark to_job as experimental (authored by sandyryza).
mark to_job as experimental
Mon, Jun 14, 8:33 PM

Fri, Jun 11

sandyryza requested review of D8355: mark to_job as experimental.
Fri, Jun 11, 11:52 PM
sandyryza updated the diff for D8283: [crag] partitions on modes.

up

Fri, Jun 11, 11:34 PM
sandyryza closed D8191: make input and output definition metadata unexperimental.
Fri, Jun 11, 11:19 PM
sandyryza committed R1:632895150a62: make input and output definition metadata unexperimental (authored by sandyryza).
make input and output definition metadata unexperimental
Fri, Jun 11, 11:19 PM
sandyryza updated the diff for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.

up

Fri, Jun 11, 11:08 PM
sandyryza updated the diff for D8283: [crag] partitions on modes.

up

Fri, Jun 11, 11:04 PM
sandyryza planned changes to D8283: [crag] partitions on modes.
Fri, Jun 11, 10:21 PM
sandyryza requested review of D8353: Dockerfile for hacker news example.
Fri, Jun 11, 10:08 PM
sandyryza retitled D8283: [crag] partitions on modes from [RFC] [crag] partitions on modes to [crag] partitions on modes.
Fri, Jun 11, 9:56 PM
sandyryza updated the diff for D8283: [crag] partitions on modes.

up

Fri, Jun 11, 9:56 PM
sandyryza updated the diff for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.

up

Fri, Jun 11, 9:00 PM
sandyryza added a comment to D8344: [crag] job name.

I had a conversation with @schrockn and I am receptive to his argument that it's more elegant for jobs to have names than suffixes. Here's a re-spin that does that.

Fri, Jun 11, 8:36 PM
sandyryza retitled D8344: [crag] job name from [crag] job suffix to [crag] job name.
Fri, Jun 11, 8:35 PM
sandyryza updated the diff for D8344: [crag] job name.

up

Fri, Jun 11, 8:35 PM
sandyryza accepted D8349: feat: add flag to disable all warnings when running dagit.

lgtm!

Fri, Jun 11, 8:33 PM
sandyryza requested changes to D8349: feat: add flag to disable all warnings when running dagit.
Fri, Jun 11, 6:45 PM
sandyryza requested review of D8335: mlflow CI.
Fri, Jun 11, 6:15 PM
sandyryza updated the diff for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.

up

Fri, Jun 11, 5:30 PM

Thu, Jun 10

sandyryza added a comment to D8283: [crag] partitions on modes.

where do we do partition set look ups? Is there a way to store a different type of targeting on the object that refers to it, like JobScoped instead of RepoScoped so you go get the Job and fetch it of there instead of getting it out of the repo?

Thu, Jun 10, 11:20 PM
sandyryza accepted D8318: [crag] default_config.

awesome

Thu, Jun 10, 6:48 PM
sandyryza added a comment to D8318: [crag] default_config.

Will this config be used when invoking the job from a schedule or sensor? I had been imagining that it would.

Thu, Jun 10, 5:32 PM
sandyryza updated the summary of D8283: [crag] partitions on modes.
Thu, Jun 10, 3:52 PM

Wed, Jun 9

sandyryza added a reviewer for D8283: [crag] partitions on modes: rexledesma.
Wed, Jun 9, 10:49 PM
sandyryza accepted D8307: [dagit] Crag: Add mode to search.

We are not currently omitting it in the header of the Pipeline (soon Job?) page, nor in the URL. But I can omit it here if desired.

Wed, Jun 9, 8:09 PM
sandyryza accepted D8278: Hackernews demo in public repo.

After this goes in, what will the procedure be for making changes to the pipelines? Will there be some intermediate period during which we need to replicate changes across both the public repo and the demo repo? Not the end of the world, but ultimately will be pretty painful. Do we have a plan for making the demo depend on the public repo version of the pipeline?

Wed, Jun 9, 5:31 PM
sandyryza accepted D8291: Hook Invocation.
Wed, Jun 9, 4:48 PM
sandyryza added a comment to D8307: [dagit] Crag: Add mode to search.

Not blocking for this diff, but we've discussed omitting the mode name when the mode is "default", so I think we'd want to do that here as well if we did that elsewhere.

Wed, Jun 9, 4:46 PM

Tue, Jun 8

sandyryza added a reviewer for D8283: [crag] partitions on modes: schrockn.
Tue, Jun 8, 10:16 PM
sandyryza accepted D8261: [crag] config mapping via mode.

Yeah my partitions diff definitely has the same issue.

Tue, Jun 8, 10:09 PM
sandyryza closed D8276: type annotations for _CacheingDefinitionIndex.
Tue, Jun 8, 9:49 PM
sandyryza committed R1:0c3e0776c3a4: type annotations for _CacheingDefinitionIndex (authored by sandyryza).
type annotations for _CacheingDefinitionIndex
Tue, Jun 8, 9:49 PM
sandyryza added inline comments to D8259: [guide] file processing.
Tue, Jun 8, 9:11 PM
sandyryza added a comment to D8209: [RFC] send python logging messages to the DagsterLogManager.

if we did this we would want it behind an opt-in gating of some kind right?

Tue, Jun 8, 9:05 PM
sandyryza requested review of D8283: [crag] partitions on modes.
Tue, Jun 8, 4:57 PM
sandyryza added a comment to D8261: [crag] config mapping via mode.

I think the main thing worth discussing here is that this shifts ModeDefinitions from objects that can live inside multiple pipelines to objects that are specific to a single pipeline. In legacy non-crag world, do we think its legitimate for a user to use the config_mapping arg on ModeDefinition, or is this just a hack to get crag working that doesn't require adding JobDefinition, etc.? I'd be more comfortable with the latter than the former.

Tue, Jun 8, 4:47 PM
sandyryza added a comment to D8272: ban io_manager_key and root_manager_key on composite solids.

looks like we don't hard error on io_manager_key being set on composite, should we?

Tue, Jun 8, 4:36 PM
sandyryza added inline comments to D8276: type annotations for _CacheingDefinitionIndex.
Tue, Jun 8, 4:35 PM
sandyryza updated the diff for D8276: type annotations for _CacheingDefinitionIndex.

up

Tue, Jun 8, 4:35 PM
sandyryza closed D8280: move last_empty_partition to test file.
Tue, Jun 8, 4:13 PM
sandyryza committed R1:d0a8956fd448: move last_empty_partition to test file (authored by sandyryza).
move last_empty_partition to test file
Tue, Jun 8, 4:13 PM
sandyryza added a comment to D8239: Sensor invocation, remove evaluate_tick from API docs, change testing story.

Do we want to make the context arg itself optional?

Tue, Jun 8, 4:12 PM
sandyryza resigned from D7895: monitor sensor 4/[dagit] show origin runs on the sensor's page.

I'm going to defer to @prha

Tue, Jun 8, 4:10 PM
sandyryza added inline comments to D8262: Create interface for determining python type of all DagsterTypes.
Tue, Jun 8, 4:09 PM
sandyryza added a comment to D8272: ban io_manager_key and root_manager_key on composite solids.

If you look in test_io_manager_composites.py, only the IOManagers on the leaf solids take effect. So my expectation was that for root input managers, we would ban them on composite solids for consistency. Are there reasons that doesn't make sense?

Tue, Jun 8, 4:05 PM
sandyryza accepted D8277: Small bugfix for logger invocation, and add testing section to logger docs.
Tue, Jun 8, 3:58 PM
sandyryza accepted D8213: pipeline sensor - fetch events without run id + sqlite trick.

LGTM!

Tue, Jun 8, 3:58 PM
sandyryza updated the diff for D8276: type annotations for _CacheingDefinitionIndex.

up

Tue, Jun 8, 3:47 PM
sandyryza updated the diff for D8276: type annotations for _CacheingDefinitionIndex.

up

Tue, Jun 8, 3:43 PM
sandyryza updated the diff for D8276: type annotations for _CacheingDefinitionIndex.

up

Tue, Jun 8, 3:38 PM
sandyryza updated the diff for D8276: type annotations for _CacheingDefinitionIndex.

up

Tue, Jun 8, 1:43 AM

Mon, Jun 7

sandyryza requested review of D8280: move last_empty_partition to test file.
Mon, Jun 7, 11:43 PM
sandyryza requested review of D8276: type annotations for _CacheingDefinitionIndex.
Mon, Jun 7, 8:57 PM

Fri, Jun 4

sandyryza requested review of D8180: [RFC] funchouse.
Fri, Jun 4, 10:50 PM
sandyryza requested review of D8259: [guide] file processing.
Fri, Jun 4, 10:37 PM
sandyryza accepted D8257: EnvironmentConfig -> ResolvedRunConfig.

Amazing

Fri, Jun 4, 9:46 PM
sandyryza accepted D7999: build_init_resource_context.
Fri, Jun 4, 7:02 PM