Page MenuHomeElementl
Feed Advanced Search

Jul 22 2021

sandyryza closed D9021: changes for 0.12.3.
Jul 22 2021, 7:55 PM
sandyryza committed R1:af93a816e25a: changes for 0.12.3 (authored by sandyryza).
changes for 0.12.3
Jul 22 2021, 7:55 PM
sandyryza updated the diff for D9021: changes for 0.12.3.

up

Jul 22 2021, 7:22 PM
sandyryza added a comment to D9021: changes for 0.12.3.

@jordansanders how strongly do you feel about stars vs. dashes? I don't mind changing this time, but fairly annoying if that needs to be another step in the release guide. (the stars are generated by the Quip markdown export).

Jul 22 2021, 7:18 PM
sandyryza published D9021: changes for 0.12.3 for review.
Jul 22 2021, 7:05 PM
sandyryza accepted D9015: run_status_sensor api doc.
Jul 22 2021, 6:04 PM
sandyryza published D9015: run_status_sensor api doc for review.
Jul 22 2021, 6:04 PM
sandyryza accepted D8942: [crag] Job description.
Jul 22 2021, 3:47 PM
sandyryza accepted D8943: [crag] fix snapshots handling composed @graph.
Jul 22 2021, 3:33 PM
sandyryza accepted D8900: RunStatusSensorDefinition and @run_status_sensor.

This looks great

Jul 22 2021, 12:41 AM
sandyryza added inline comments to D8900: RunStatusSensorDefinition and @run_status_sensor.
Jul 22 2021, 12:14 AM

Jul 21 2021

sandyryza accepted D8992: [dagit] Fix link to partition from schedule page.
Jul 21 2021, 11:32 PM
sandyryza updated the summary of D8969: [caprisun] @asset and build_assets_job.
Jul 21 2021, 11:32 PM
sandyryza updated the summary of D8969: [caprisun] @asset and build_assets_job.
Jul 21 2021, 11:30 PM
sandyryza retitled D8969: [caprisun] @asset and build_assets_job from assets to [RFC] @asset and build_assets_job.
Jul 21 2021, 11:29 PM
sandyryza added inline comments to D8900: RunStatusSensorDefinition and @run_status_sensor.
Jul 21 2021, 11:28 PM
sandyryza requested review of D8969: [caprisun] @asset and build_assets_job.
Jul 21 2021, 11:01 PM
sandyryza requested review of D9000: add title and description to graph migration page.
Jul 21 2021, 10:50 PM
sandyryza requested review of D8998: check.is_dict.
Jul 21 2021, 10:29 PM
sandyryza closed D8976: [new-project] sensor targets non-existent mode.
Jul 21 2021, 10:27 PM
sandyryza committed R1:d774b3ff1ca0: [new-project] sensor targets non-existent mode (authored by sandyryza).
[new-project] sensor targets non-existent mode
Jul 21 2021, 10:27 PM
sandyryza accepted D8979: [crag] partition schedules migration guide.

This looks great

Jul 21 2021, 9:48 PM
sandyryza added a comment to D8963: [memoization 3/n] move core memoization logic to live on the execution plan.

I agree with Alex regarding the opt-in question

Jul 21 2021, 4:34 PM
sandyryza added inline comments to D8979: [crag] partition schedules migration guide.
Jul 21 2021, 12:45 AM

Jul 20 2021

sandyryza requested review of D8976: [new-project] sensor targets non-existent mode.
Jul 20 2021, 11:21 PM
sandyryza added inline comments to D8900: RunStatusSensorDefinition and @run_status_sensor.
Jul 20 2021, 11:02 PM
sandyryza abandoned D8180: [RFC] funchouse.
Jul 20 2021, 5:24 PM
sandyryza added a comment to D8180: [RFC] funchouse.

abandoning this in favor of a version that will live inside dagster core

Jul 20 2021, 5:24 PM
sandyryza added a comment to D8942: [crag] Job description.

Do you have thoughts on what the dagit experience should be like? I.e. where does graph description vs. job description show up?

Jul 20 2021, 3:32 PM
sandyryza accepted D8954: [dagit] Crag: solid -> op.
Jul 20 2021, 3:27 PM
sandyryza resigned from D8501: refactor: convert toys to graph + job + op(eration)s.
Jul 20 2021, 3:24 PM
sandyryza closed D8918: improvements to graph/job/op migration guide.
Jul 20 2021, 3:17 PM
sandyryza committed R1:b372aa199679: improvements to graph/job/op migration guide (authored by sandyryza).
improvements to graph/job/op migration guide
Jul 20 2021, 3:17 PM

Jul 19 2021

sandyryza added a comment to D8956: [dagit] Show start/end information on latest run in Job details.

Thoughts on replacing the duration with the most recent start time in the header? I think the most recent start time is a more useful piece of information than the duration.

Jul 19 2021, 9:34 PM
sandyryza accepted D8949: [dagit] Fix non-Crag schedule/sensor job metadata.
Jul 19 2021, 8:52 PM
sandyryza requested review of D8946: nothing example misuses type annotations.
Jul 19 2021, 5:58 PM
sandyryza accepted D8945: [crag] jobs entry in @repository dict form.
Jul 19 2021, 5:47 PM
sandyryza retitled D8918: improvements to graph/job/op migration guide from resource instance in graph/job/op migration guide to improvements to graph/job/op migration guide.
Jul 19 2021, 4:50 PM
sandyryza requested review of D8918: improvements to graph/job/op migration guide.
Jul 19 2021, 4:47 PM

Jul 16 2021

sandyryza accepted D8903: Add solid as parameter to build_hook_context.
Jul 16 2021, 12:48 AM
sandyryza accepted D8916: put under test that jobs with partitioned config can be successfully backfilled..

great that this already works

Jul 16 2021, 12:46 AM

Jul 15 2021

sandyryza closed D8907: nav entry for the graph/job/op migration guide.
Jul 15 2021, 7:51 PM
sandyryza committed R1:dd6d4ceb2240: nav entry for the graph/job/op migration guide (authored by sandyryza).
nav entry for the graph/job/op migration guide
Jul 15 2021, 7:51 PM
sandyryza updated the diff for D8907: nav entry for the graph/job/op migration guide.

up

Jul 15 2021, 5:35 PM
sandyryza updated the diff for D8907: nav entry for the graph/job/op migration guide.

up

Jul 15 2021, 5:07 PM

Jul 14 2021

sandyryza accepted D8897: [dagit] Move job metadata into header.

Nice

Jul 14 2021, 11:36 PM
sandyryza added a comment to D8900: RunStatusSensorDefinition and @run_status_sensor.

Thoughts on calling this pipeline_event_sensor? pipeline_sensor could be interpreted as a sensor that kicks off a pipeline.

Jul 14 2021, 11:04 PM
sandyryza accepted D8678: [RFC] time-based partitioned config and schedule from partitioned config.

I left a final comment that should be addressed before merging. Otherwise, looks great.

Jul 14 2021, 2:30 AM

Jul 13 2021

sandyryza requested changes to D8678: [RFC] time-based partitioned config and schedule from partitioned config.
Jul 13 2021, 8:47 PM
sandyryza closed D8821: fix error message when solid is missing context.
Jul 13 2021, 5:24 PM
sandyryza committed R1:d206f5e08beb: fix error message when solid is missing context (authored by sandyryza).
fix error message when solid is missing context
Jul 13 2021, 5:24 PM
sandyryza requested changes to D8565: [crag] permit resource instances on to_job.

Oops accepted by accident

Jul 13 2021, 5:24 PM
sandyryza accepted D8565: [crag] permit resource instances on to_job.

I had originally been a strong proponent of this, but, having reviewed the resources in our dogfood pipelines, I'm less sure. It's only really the IOManagers that would benefit from this. I wonder if it would make more sense to just make it a little less verbose to make a hardcoded resource. Right now it requires ResourceDefinition.hardcoded_resource. We could make hardcoded_resource a top-level API?

Jul 13 2021, 5:23 PM
sandyryza added a comment to D8678: [RFC] time-based partitioned config and schedule from partitioned config.

Thanks a ton for picking this up. To make it accessible to users, it needs a few more things:

  • Docstrings for the _partitioned_config functions.
  • Experimental warnings.
  • Inclusion in the experimental API docs page.
  • Inclusion in the top-level public API.
Jul 13 2021, 5:14 PM

Jul 12 2021

sandyryza closed D8824: crag migration guide.
Jul 12 2021, 11:27 PM
sandyryza committed R1:11a266443fdf: crag migration guide (authored by sandyryza).
crag migration guide
Jul 12 2021, 11:27 PM
sandyryza accepted D8836: Permit resource instance on execute_in_process in addition to resource definitions.

Nice

Jul 12 2021, 10:19 PM
sandyryza updated the diff for D8824: crag migration guide.

up

Jul 12 2021, 10:15 PM
sandyryza added inline comments to D8824: crag migration guide.
Jul 12 2021, 10:15 PM
sandyryza closed D8823: default schedule name when none provided.
Jul 12 2021, 10:00 PM
sandyryza committed R1:55cc0b990214: default schedule name when none provided (authored by sandyryza).
default schedule name when none provided
Jul 12 2021, 10:00 PM
sandyryza updated the diff for D8823: default schedule name when none provided.

up

Jul 12 2021, 9:06 PM
sandyryza requested review of D8824: crag migration guide.

test failure is unrelated

Jul 12 2021, 7:53 PM
sandyryza requested review of D8823: default schedule name when none provided.

Test failure is unrelated

Jul 12 2021, 7:01 PM
sandyryza requested review of D8821: fix error message when solid is missing context.
Jul 12 2021, 5:24 PM

Jul 8 2021

sandyryza resigned from D8732: Create migration to extract asset_keys columns for asset catalog perf.
Jul 8 2021, 5:42 PM
sandyryza abandoned D7762: [crag] unbound sensors.
Jul 8 2021, 5:14 PM
sandyryza accepted D8749: [crag] api docs section.

I think this looks great

Jul 8 2021, 1:40 AM

Jul 7 2021

sandyryza accepted D8746: [dagit] Fix: Choose partition set for session tab.
Jul 7 2021, 9:12 PM
sandyryza resigned from D6810: [core][dagster-mysql] Fixing @experimental tag to work for classes.
Jul 7 2021, 8:07 PM
sandyryza added a comment to D8604: [job] add executor_def.

I chatted with Alex offline and he convinced me on the approach here. I'll leave it to @cdecarolis for final review, but consider my concerns addressed.

Jul 7 2021, 3:53 PM
sandyryza resigned from D8629: [crag] Add mode to the pipeline runs filter.
Jul 7 2021, 3:27 PM
sandyryza accepted D8392: Refactor dagster-dbt integration / add dbt cli resource.
Jul 7 2021, 3:25 PM
sandyryza added a comment to D8392: Refactor dagster-dbt integration / add dbt cli resource.

This looks great!

Jul 7 2021, 3:25 PM
sandyryza resigned from D8708: docs: remove references to Gitpod.

Resigning because I don't have a ton of context.

Jul 7 2021, 3:12 PM
sandyryza added a reviewer for D8708: docs: remove references to Gitpod: yuhan.
Jul 7 2021, 3:12 PM
sandyryza closed D8618: [crag] single config parameter to to_job.
Jul 7 2021, 5:38 AM
sandyryza committed R1:da3ee30cfb50: [crag] single config parameter to to_job (authored by sandyryza).
[crag] single config parameter to to_job
Jul 7 2021, 5:38 AM
sandyryza requested changes to D8685: Make context argument optional on resource definition.
Jul 7 2021, 5:37 AM
sandyryza accepted D8687: [crag] Pipeline.graph_name.

I might not be the best person to review this, but looks reasonable to me.

Jul 7 2021, 5:17 AM
sandyryza updated the diff for D8618: [crag] single config parameter to to_job.

up

Jul 7 2021, 5:02 AM
sandyryza added inline comments to D8618: [crag] single config parameter to to_job.
Jul 7 2021, 5:01 AM

Jul 6 2021

sandyryza added inline comments to D8685: Make context argument optional on resource definition.
Jul 6 2021, 8:54 PM
sandyryza accepted D8623: [op] Get rid of MultiOut, get rid of name on Out, change to accepting bare dict, clean up tuple typing implementation..

charmander

Jul 6 2021, 8:26 PM
sandyryza accepted D8706: de-experimental dynamic mapping & collect.

Yeehaw!

Jul 6 2021, 8:25 PM
sandyryza added inline comments to D8392: Refactor dagster-dbt integration / add dbt cli resource.
Jul 6 2021, 5:25 PM
sandyryza added a comment to D8604: [job] add executor_def.

To clarify, you mean just fs_io_manager and not also the defaulting of multiprocess execution?

Jul 6 2021, 4:37 PM
sandyryza added inline comments to D8695: executor-based IO-management.
Jul 6 2021, 4:12 PM
sandyryza closed D8525: jobs can have loggers.
Jul 6 2021, 3:37 PM
sandyryza committed R1:b8118ff24498: jobs can have loggers (authored by sandyryza).
jobs can have loggers
Jul 6 2021, 3:37 PM
sandyryza added inline comments to D8695: executor-based IO-management.
Jul 6 2021, 3:33 PM

Jul 2 2021

sandyryza accepted D8605: [docs] use pipeline_failure_sensor in failure sensor example.

This looks great.

Jul 2 2021, 11:46 PM
sandyryza resigned from D8550: [op] permit op config entry.

Going to defer to Alex on this one

Jul 2 2021, 11:43 PM
sandyryza resigned from D8643: [dagit] Crag: Populate empty config if only one preset.

+1 for doing this. Resigning because I'm not qualified to comment on the implementation.

Jul 2 2021, 11:28 PM
sandyryza updated the diff for D8525: jobs can have loggers.

up

Jul 2 2021, 11:27 PM
sandyryza added a comment to D8604: [job] add executor_def.

+1 to adding executor_def on jobs. The only part of this change that makes me nervous is switching the default to fs_io_manager. execute_in_process(**job.unpack_for_test()) feels gross to me. This could potentially give us the best of both worlds: https://dagster.phacility.com/D8695?

Jul 2 2021, 11:25 PM
sandyryza requested review of D8695: executor-based IO-management.
Jul 2 2021, 11:18 PM
sandyryza added inline comments to D8618: [crag] single config parameter to to_job.
Jul 2 2021, 11:17 PM
sandyryza updated the diff for D8618: [crag] single config parameter to to_job.

up

Jul 2 2021, 11:12 PM
sandyryza updated the diff for D8618: [crag] single config parameter to to_job.

up

Jul 2 2021, 9:58 PM