Page MenuHomePhabricator
Feed All Stories

Today

schrockn requested review of D5317: Add ScopedResourceSet to thread down through the config parsing code paths.
Sat, Nov 28, 9:30 PM
schrockn requested review of D5316: (refactor-input-source-to-thunk-4) Add checks to catch errors when new source types are added.
Sat, Nov 28, 7:27 PM
schrockn requested review of D5315: (refactor-input-source-to-thunk-3) Add compute_version to StepInputSource hierarchy.
Sat, Nov 28, 7:15 PM
schrockn requested review of D5314: (refactor-input-source-to-thunk-2) Move required_resource_keys to be polymorphic on StepInputSource.
Sat, Nov 28, 7:10 PM
schrockn added inline comments to D5312: (refactor-input-source-to-thunk-1) Push loading behavior to StepInputSource and make it a thunk.
Sat, Nov 28, 6:42 PM
schrockn updated the diff for D5312: (refactor-input-source-to-thunk-1) Push loading behavior to StepInputSource and make it a thunk.

up

Sat, Nov 28, 6:41 PM
schrockn requested review of D5313: Push loading behavior to StepInputSource and make it a thunk and add versioning and required resource keys to source.
Sat, Nov 28, 6:41 PM
schrockn requested review of D5312: (refactor-input-source-to-thunk-1) Push loading behavior to StepInputSource and make it a thunk.
Sat, Nov 28, 5:44 PM
schrockn requested review of D5311: Push loading behavior to StepInputSource.
Sat, Nov 28, 5:10 PM

Yesterday

schrockn added inline comments to D5309: fix fan-in with asset stores.
Fri, Nov 27, 8:44 PM
schrockn accepted D5309: fix fan-in with asset stores.
Fri, Nov 27, 8:41 PM
schrockn added a comment to D5309: fix fan-in with asset stores.

Oh god I was doing prototyping in this area without bringing in the refactor

Fri, Nov 27, 8:41 PM
schrockn requested review of D5310: strict things.
Fri, Nov 27, 8:31 PM
sandyryza requested review of D5309: fix fan-in with asset stores.
Fri, Nov 27, 8:28 PM
sandyryza requested review of D5307: RFC: loader_key.
Fri, Nov 27, 7:41 PM

Thu, Nov 26

sandyryza added a comment to D5268: fix fan-in input mappings in composites.

When you say "fixing things", is there a github issue or something that describes what needs to be fixed?

Thu, Nov 26, 4:43 PM
yichendai updated the diff for D5225: Fix schreenshots test: Part 1.

Update: Allow a hover error to be seen and fix all the new URLs

Thu, Nov 26, 1:18 AM
sandyryza accepted D5257: enable reexecution when all required asset stores are non mem.

Looks good! It feels like there are some tricky edge cases here, e.g. if only some inputs of a solid come from a prior run, so maybe worth thinking through if there are tests to write that would give us confidence on them.

Thu, Nov 26, 12:25 AM

Wed, Nov 25

sashank requested review of D5306: [dagit] Remove refetch from WorkspaceContext.
Wed, Nov 25, 11:18 PM
sashank closed D5299: Fix apollo cache reset race condition.
Wed, Nov 25, 10:59 PM
sashank committed R1:1d91ee8bccdb: Fix apollo cache reset race condition (authored by sashank).
Fix apollo cache reset race condition
Wed, Nov 25, 10:59 PM
sashank updated the diff for D5299: Fix apollo cache reset race condition.

Add comment

Wed, Nov 25, 10:59 PM
GitHub <noreply@github.com> committed R1:e70dfb6fa235: Update README.md (authored by Yichen <59324923+nancydyc@users.noreply.github.com>).
Update README.md
Wed, Nov 25, 10:58 PM
sashank added inline comments to D5299: Fix apollo cache reset race condition.
Wed, Nov 25, 10:57 PM
alangenfeld added a comment to D5291: [RFC] Cancel queued runs.

just throwing some ideas around -

  • we add column to run storage, user_action, user_override or something like that
  • add a new event type, not engine event, that corresponds
  • run storage response to this new event does a smart column update so that it is only written from "origin host process" ie dagit and shouldn't conflict with other writes
  • add a new API (which we could use for other reasons) of instance.get_run_row_by_id to allow fetching the full run row including this column
  • use this for cancel / terminate from the dequeuer / run worker
Wed, Nov 25, 10:46 PM
dgibson added a comment to D5291: [RFC] Cancel queued runs.

If we do that it should cover terminated too IMO

Wed, Nov 25, 10:31 PM
alangenfeld added a comment to D5291: [RFC] Cancel queued runs.

nit: reported something as "failed" that never started feels weird to me - i think an explicit canceled state may be worth it

Wed, Nov 25, 10:28 PM
alangenfeld added a comment to D5291: [RFC] Cancel queued runs.

oof ya the race conditions here are really gnarly. Probably worth drawing up the full state machine diagram. I feel like we need to have a safer overall scheme like @dgibson is talking about. I think delete and mark-as-failed are emergency escape hatches that may put the system in to a bad state if the user does them at the wrong time. In all other cases there should be a clear "owner" for the run and only that "owner" should manage transitions.

Wed, Nov 25, 10:22 PM
yuhan retitled D5257: enable reexecution when all required asset stores are non mem from enable reexecution when 'asset_store' is set to enable reexecution when all required asset stores are non mem.
Wed, Nov 25, 10:14 PM
bengotow requested review of D5292: Fix "View Asset Dashboard" link using href={} for a local app route.
Wed, Nov 25, 10:04 PM
johann updated the diff for D5120: Daemon heartbeats backend.

up

Wed, Nov 25, 10:03 PM
sashank added a comment to D5304: Fix undo functionatlity in Dagit playground.

Double check that switching tabs in the editor doesn't create a continuous undo history

Wed, Nov 25, 10:00 PM
schrockn updated the diff for D5271: Refactor configured infrastructure to eliminate "underscore" params on public definitions.

up

Wed, Nov 25, 10:00 PM
schrockn updated the diff for D5271: Refactor configured infrastructure to eliminate "underscore" params on public definitions.

up

Wed, Nov 25, 9:51 PM
Harbormaster failed remote builds in B21798: Diff 26478 for D5120: Daemon heartbeats backend!
Wed, Nov 25, 9:47 PM
sashank updated the diff for D5304: Fix undo functionatlity in Dagit playground.

up

Wed, Nov 25, 9:46 PM
bengotow added a comment to D5304: Fix undo functionatlity in Dagit playground.

This patch package is pretty intriguing. I'm OK with doing this in this isolated instance because we're likely to abandon codemirror at some point in the future (for Monaco) and forking / fixing the library to make this change seems like overkill. I'm curious to get @dish 's thoughts + approval on this though because he investigated that change.

Wed, Nov 25, 9:45 PM
schrockn requested review of D5271: Refactor configured infrastructure to eliminate "underscore" params on public definitions.
Wed, Nov 25, 9:38 PM
sashank published D5304: Fix undo functionatlity in Dagit playground for review.
Wed, Nov 25, 9:35 PM
alangenfeld closed D5302: del stale todo.
Wed, Nov 25, 9:32 PM
alangenfeld committed R1:e6845016df37: del stale todo (authored by alangenfeld).
del stale todo
Wed, Nov 25, 9:32 PM
sashank accepted D5302: del stale todo.
Wed, Nov 25, 9:31 PM
johann updated the diff for D5120: Daemon heartbeats backend.

up

Wed, Nov 25, 9:30 PM
alangenfeld requested review of D5302: del stale todo.
Wed, Nov 25, 9:12 PM
sashank closed D4808: (config-scaffold 2/n) Add scaffold config button to Dagit playground.
Wed, Nov 25, 9:09 PM
sashank committed R1:1d3359632e5f: (config-scaffold 2/n) Add scaffold config button to Dagit playground (authored by sashank).
(config-scaffold 2/n) Add scaffold config button to Dagit playground
Wed, Nov 25, 9:09 PM
sashank updated the diff for D4808: (config-scaffold 2/n) Add scaffold config button to Dagit playground.

up

Wed, Nov 25, 9:01 PM
sashank updated the diff for D4808: (config-scaffold 2/n) Add scaffold config button to Dagit playground.

up

Wed, Nov 25, 8:46 PM
prha committed R1:7f347aecd782: fix toy sensor directory name (authored by prha).
fix toy sensor directory name
Wed, Nov 25, 8:45 PM
prha closed D5301: fix bug in toy sensor.
Wed, Nov 25, 8:38 PM
prha committed R1:dee511780178: fix bug in toy sensor (authored by prha).
fix bug in toy sensor
Wed, Nov 25, 8:38 PM
dish accepted D5301: fix bug in toy sensor.
Wed, Nov 25, 8:37 PM
prha published D5301: fix bug in toy sensor for review.
Wed, Nov 25, 8:37 PM
sashank updated the diff for D4808: (config-scaffold 2/n) Add scaffold config button to Dagit playground.

up

Wed, Nov 25, 8:34 PM
GitHub <noreply@github.com> committed R1:c13015272dc2: Update bug_report.md (authored by Yichen <59324923+nancydyc@users.noreply.github.com>).
Update bug_report.md
Wed, Nov 25, 8:31 PM
GitHub <noreply@github.com> committed R1:f2361a79db4c: Update request_a_feature.md (authored by Yichen <59324923+nancydyc@users.noreply.github.com>).
Update request_a_feature.md
Wed, Nov 25, 8:30 PM
dish closed D5300: [dagit] Add sensors pages.
Wed, Nov 25, 8:28 PM
dish committed R1:c2f7a7631720: [dagit] Add sensors pages (authored by dish).
[dagit] Add sensors pages
Wed, Nov 25, 8:28 PM
GitHub <noreply@github.com> committed R1:7506d04d4022: Update bug_report.md (authored by Yichen <59324923+nancydyc@users.noreply.github.com>).
Update bug_report.md
Wed, Nov 25, 8:24 PM
dgibson accepted D5300: [dagit] Add sensors pages.

this all seems quite straightforward, we can sort out whether we need an 'Unloadable Sensors' section separately

Wed, Nov 25, 8:22 PM
alangenfeld added a reviewer for D5268: fix fan-in input mappings in composites: sandyryza.
Wed, Nov 25, 8:21 PM
Harbormaster failed remote builds in B21788: Diff 26462 for D5217: StepInput refactor!
Wed, Nov 25, 8:20 PM
alangenfeld closed D5217: StepInput refactor.
Wed, Nov 25, 8:20 PM
alangenfeld committed R1:905fd981bcf1: StepInput refactor (authored by alangenfeld).
StepInput refactor
Wed, Nov 25, 8:20 PM
schrockn accepted D5217: StepInput refactor.

makes sense

Wed, Nov 25, 8:14 PM
alangenfeld added a comment to D5217: StepInput refactor.

@schrockn im going to land this to allow @sandyryza to build on top of it. I'm pretty sure my answer to your question should clear, if you have any other feedback i can incorporate it in to D5268

Wed, Nov 25, 8:08 PM
dish requested review of D5300: [dagit] Add sensors pages.
Wed, Nov 25, 8:06 PM
alangenfeld closed D5269: delete redundant code path.
Wed, Nov 25, 8:02 PM
alangenfeld committed R1:928530b7d028: delete redundant code path (authored by alangenfeld).
delete redundant code path
Wed, Nov 25, 8:02 PM
sandyryza accepted D5217: StepInput refactor.
Wed, Nov 25, 8:02 PM
alangenfeld updated the diff for D5217: StepInput refactor.

sentinel-ify

Wed, Nov 25, 7:58 PM
cdecarolis published D5275: adls2 asset store for review.

build completed successfully, not sure why the hook didn't publish.

Wed, Nov 25, 7:50 PM
yuhan updated the diff for D5257: enable reexecution when all required asset stores are non mem.

gql error msg

Wed, Nov 25, 7:26 PM
alangenfeld added a comment to D5269: delete redundant code path.

updated summary

Wed, Nov 25, 7:25 PM
alangenfeld updated the summary of D5269: delete redundant code path.
Wed, Nov 25, 7:24 PM
sashank added inline comments to D5299: Fix apollo cache reset race condition.
Wed, Nov 25, 7:21 PM
dgibson added inline comments to D5291: [RFC] Cancel queued runs.
Wed, Nov 25, 7:19 PM
dgibson added a comment to D5291: [RFC] Cancel queued runs.

This seems reasonable to me - curious what @alangenfeld thinks and how this might interact with https://github.com/dagster-io/dagster/issues/3239 and/or https://github.com/dagster-io/dagster/issues/3236 ? Left some thoughts on the race condition inline.

Wed, Nov 25, 7:16 PM
max abandoned D5068: Test integration trigger.
Wed, Nov 25, 7:10 PM
max added a comment to D4937: up pickle protocol.

Ok, since we've dropped 2.7, we can go to pickle protocol 4, but not to HIGHEST_PROTOCOL

Wed, Nov 25, 7:10 PM
max added a comment to D5126: RFC use custom yaml loader.

What's the error message, or does parsing just proceed without the datetime being parsed? YAML is complex enough that I can imagine someone tearing their hair out about this behavior with no clue how to proceed..

Wed, Nov 25, 7:08 PM
johann added inline comments to D5291: [RFC] Cancel queued runs.
Wed, Nov 25, 7:01 PM
dish accepted D5299: Fix apollo cache reset race condition.
Wed, Nov 25, 6:52 PM
johann updated the diff for D5291: [RFC] Cancel queued runs.

up

Wed, Nov 25, 6:45 PM
johann added inline comments to D5291: [RFC] Cancel queued runs.
Wed, Nov 25, 6:44 PM
Harbormaster failed remote builds in B21780: Diff 26453 for D4808: (config-scaffold 2/n) Add scaffold config button to Dagit playground!
Wed, Nov 25, 6:42 PM
Harbormaster failed remote builds in B21777: Diff 26450 for D5291: [RFC] Cancel queued runs!
Wed, Nov 25, 6:36 PM
alangenfeld added a comment to D5217: StepInput refactor.

My big question is why is this FromMultipleSources and not FromMultipleStepOutputs?

Wed, Nov 25, 6:32 PM
prha closed D5296: add sensor mutation, fix sensorOrError.
Wed, Nov 25, 6:32 PM
prha committed R1:d255c089b844: add sensor mutation, fix sensorOrError (authored by prha).
add sensor mutation, fix sensorOrError
Wed, Nov 25, 6:32 PM
sashank requested review of D5299: Fix apollo cache reset race condition.
Wed, Nov 25, 6:28 PM
sashank updated the diff for D4808: (config-scaffold 2/n) Add scaffold config button to Dagit playground.

up

Wed, Nov 25, 6:25 PM
dgibson accepted D5296: add sensor mutation, fix sensorOrError.
Wed, Nov 25, 6:25 PM
dgibson added a comment to D5269: delete redundant code path.

(but also maybe update the summary a bit more?)

Wed, Nov 25, 6:22 PM
dgibson accepted D5269: delete redundant code path.

I'm not super familiar with this path and it doesn't seem immediately obvious that it is in fact redundant, but it seems plausible and I trust you? Is it because dependencies_dict includes transitive dependencies already? re-request for review if you need more detailed scrutiny

Wed, Nov 25, 6:21 PM
johann updated the diff for D5291: [RFC] Cancel queued runs.

up

Wed, Nov 25, 6:20 PM
yuhan updated the diff for D5257: enable reexecution when all required asset stores are non mem.

up

Wed, Nov 25, 6:14 PM
prha requested review of D5296: add sensor mutation, fix sensorOrError.
Wed, Nov 25, 6:08 PM
Harbormaster failed remote builds in B21769: Diff 26442 for D5217: StepInput refactor!
Wed, Nov 25, 6:08 PM
schrockn requested changes to D5217: StepInput refactor.

My big question is why is this FromMultipleSources and not FromMultipleStepOutputs?

Wed, Nov 25, 6:00 PM