Page MenuHomePhabricator
Feed All Stories

Yesterday

dgibson abandoned D6119: [12/n] Reconstruct ExecutionPlan from the ExecutionPlanSnapshot when it is available.
Sun, Jan 24, 11:29 PM
dgibson abandoned D6095: [10/n]Make StepOutput user-code-free.

squashed into https://dagster.phacility.com/D6133

Sun, Jan 24, 11:28 PM
dgibson abandoned D6094: [9/n] Make StepInputSource use snapshots.

squashed into https://dagster.phacility.com/D6133

Sun, Jan 24, 11:28 PM
dgibson abandoned D6093: [8/n] Make StepInput.dagster_type a snapshot instead (no user code).

squashed into https://dagster.phacility.com/D6133

Sun, Jan 24, 11:28 PM
dgibson abandoned D6086: [5/n] Wait to determine step compute_fn until you're actually in step execution.

squashed into https://dagster.phacility.com/D6133

Sun, Jan 24, 11:28 PM
dgibson abandoned D6083: [3/n] Include pipeline snapshot and execution plan snapshot on system execution context.
Sun, Jan 24, 11:27 PM
dgibson abandoned D6088: [6and7/n] remove solid from ExecutionStep.

squashed into https://dagster.phacility.com/D6133

Sun, Jan 24, 11:27 PM
dgibson retitled D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists from (Squashed) Reconstruct ExecutionPlan from the ExecutionPlanSnapshot to Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.
Sun, Jan 24, 11:26 PM
dgibson updated the diff for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.

up

Sun, Jan 24, 11:21 PM
dgibson updated the diff for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.

up

Sun, Jan 24, 11:07 PM
Harbormaster failed remote builds in B24793: Diff 30192 for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists!
Sun, Jan 24, 9:45 PM
dgibson updated the diff for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.

up

Sun, Jan 24, 9:27 PM
Harbormaster failed remote builds in B24792: Diff 30191 for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists!
Sun, Jan 24, 8:30 PM
Harbormaster failed remote builds in B24791: Diff 30190 for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists!
Sun, Jan 24, 8:23 PM
dgibson updated the diff for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.

up

Sun, Jan 24, 8:13 PM
dgibson updated the diff for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.

up

Sun, Jan 24, 8:04 PM
Harbormaster failed remote builds in B24790: Diff 30189 for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists!
Sun, Jan 24, 7:22 PM
dgibson updated the diff for D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.

up

Sun, Jan 24, 7:03 PM
dgibson requested review of D6133: Reconstruct ExecutionPlan from the ExecutionPlanSnapshot in the run worker if one exists.
Sun, Jan 24, 4:56 AM
catherinewu closed D6134: Increase UI sensor polling rate.
Sun, Jan 24, 1:02 AM
catherinewu committed R1:f2bd19e1b3d4: Increase UI sensor polling rate (authored by catherinewu).
Increase UI sensor polling rate
Sun, Jan 24, 1:02 AM
prha accepted D6134: Increase UI sensor polling rate.

squirtle

Sun, Jan 24, 12:46 AM
catherinewu published D6134: Increase UI sensor polling rate for review.
Sun, Jan 24, 12:45 AM
schrockn resigned from D6132: UI support for dynamic outputs, Run view state cleanup.

Very excited to see this land. Will defer to dish on the deets of the js.

Sun, Jan 24, 12:12 AM

Sat, Jan 23

schrockn added a comment to D6029: Flip asset materialization UI so newest time/partition is on the left.

adding @sandyryza here. I think this makes a lot of sense and I don't think having the events be left-to-right and the graphs right-to-left is a big problem but open to other's perspective

Sat, Jan 23, 11:58 PM
schrockn added a reviewer for D6029: Flip asset materialization UI so newest time/partition is on the left: sandyryza.
Sat, Jan 23, 11:57 PM
bengotow requested review of D6132: UI support for dynamic outputs, Run view state cleanup.
Sat, Jan 23, 11:50 PM
bengotow updated the diff for D6029: Flip asset materialization UI so newest time/partition is on the left.
  • Instead of flipping, scroll the asset materialization grid to the right by default
  • Fix React warnings in the useViewport component, some useCallback's missing
Sat, Jan 23, 10:34 PM
dgibson added a comment to D6131: Replace dagster api grpc with 'python -m dagster.grpc' in docs?.

"As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first."

Sat, Jan 23, 7:22 PM
schrockn added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

"I'm pretty resistant to directions that involve using type annotations for anything other than annotating the actual Python type that's expected somewhere. It's fairly abusive of a Python language feature, and it causes non-trivial trouble for anyone who's trying to use that language feature in the correct way."

Sat, Jan 23, 7:22 PM
dgibson added a comment to D6131: Replace dagster api grpc with 'python -m dagster.grpc' in docs?.

possibly relevant:

Sat, Jan 23, 7:21 PM
dgibson closed D6081: Remove pipeline_def from top-level SystemExecutionContext, only use in step execution contexts.
Sat, Jan 23, 7:20 PM
dgibson committed R1:ac7dec89ecca: Remove pipeline_def from top-level SystemExecutionContext, only use in step… (authored by dgibson).
Remove pipeline_def from top-level SystemExecutionContext, only use in step…
Sat, Jan 23, 7:20 PM
dgibson published D6131: Replace dagster api grpc with 'python -m dagster.grpc' in docs? for review.
Sat, Jan 23, 7:17 PM
sandyryza added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

I like the variants you listed that involve overloading the input_defs arg on solid.  I think that kind of overloading is pretty Pythonic / expected. When reading that code, it's very difficult to misinterpret what's going on.

Sat, Jan 23, 7:07 PM
Harbormaster failed remote builds in B24770: Diff 30165 for D6119: [12/n] Reconstruct ExecutionPlan from the ExecutionPlanSnapshot when it is available!
Sat, Jan 23, 6:38 PM
dgibson updated the diff for D6119: [12/n] Reconstruct ExecutionPlan from the ExecutionPlanSnapshot when it is available.

up

Sat, Jan 23, 6:21 PM
dgibson added inline comments to D6081: Remove pipeline_def from top-level SystemExecutionContext, only use in step execution contexts.
Sat, Jan 23, 6:19 PM
dgibson updated the diff for D6081: Remove pipeline_def from top-level SystemExecutionContext, only use in step execution contexts.

up

Sat, Jan 23, 6:17 PM
dgibson requested review of D6119: [12/n] Reconstruct ExecutionPlan from the ExecutionPlanSnapshot when it is available.
Sat, Jan 23, 4:42 PM
dgibson requested review of D6129: Skip event log watch tests on local mac runs.
Sat, Jan 23, 4:11 PM
schrockn added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

We could also get clever and try to reuse input_defs and output_defs supporting a few different variants:

Sat, Jan 23, 1:30 PM
schrockn added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

I'm concerned about too many breaking changes and too much thrash. But I think that the critical thing here would be coming up with a more condense syntax for inputs and outputs. Right now if we forced users to InputDefinition and OuptutDefinition for the dagster-type-only case the code explosion would be brutal.

Sat, Jan 23, 1:27 PM
rexledesma closed D6121: Remove Kubernetes part 2.
Sat, Jan 23, 7:32 AM
rexledesma committed R1:f259b1469301: Remove Kubernetes part 2 (authored by rexledesma).
Remove Kubernetes part 2
Sat, Jan 23, 7:32 AM
rexledesma updated the diff for D6121: Remove Kubernetes part 2.

rebase

Sat, Jan 23, 7:31 AM
rexledesma closed D6122: Update url path deploying/k8s -> deploying/kubernetes.
Sat, Jan 23, 7:30 AM
rexledesma committed R1:a4e2c52cf9c8: Update url path deploying/k8s -> deploying/kubernetes (authored by rexledesma).
Update url path deploying/k8s -> deploying/kubernetes
Sat, Jan 23, 7:30 AM
rexledesma updated the diff for D6122: Update url path deploying/k8s -> deploying/kubernetes.

rebase

Sat, Jan 23, 5:27 AM
rexledesma updated the diff for D6122: Update url path deploying/k8s -> deploying/kubernetes.

rebase

Sat, Jan 23, 5:23 AM
rexledesma closed D6120: Update kubernetes deploy guide.
Sat, Jan 23, 5:19 AM
rexledesma committed R1:6f65dcac5755: Update kubernetes deploy guide (authored by rexledesma).
Update kubernetes deploy guide
Sat, Jan 23, 5:19 AM
yichendai added inline comments to D5651: [Github Community] Add a PR template.
Sat, Jan 23, 5:15 AM
yichendai updated the diff for D5651: [Github Community] Add a PR template.

Update

Sat, Jan 23, 5:13 AM
yichendai updated the diff for D5651: [Github Community] Add a PR template.

Add contributors channel

Sat, Jan 23, 5:08 AM
yichendai updated the diff for D5949: Add an example of how to trigger a pipeline by graphQL.

Improve test coverage line by line

Sat, Jan 23, 4:50 AM
yichendai planned changes to D5949: Add an example of how to trigger a pipeline by graphQL.
Sat, Jan 23, 4:30 AM
prha requested review of D6124: add future tick, run requests to schema.
Sat, Jan 23, 3:22 AM
prha requested review of D6123: pass optional execution time to partition function.
Sat, Jan 23, 3:21 AM
catherinewu requested changes to D5651: [Github Community] Add a PR template.

requesting a few changes. next time, might be better to do this in a quip doc for faster iteration?

Sat, Jan 23, 3:01 AM
catherinewu accepted D6025: [Documentation] Refresh the Contributing guides.
Sat, Jan 23, 2:57 AM
catherinewu accepted D6122: Update url path deploying/k8s -> deploying/kubernetes.
Sat, Jan 23, 2:55 AM
catherinewu accepted D6120: Update kubernetes deploy guide.

Macro pooh_eating_honey:

Sat, Jan 23, 2:53 AM
rexledesma retitled D6122: Update url path deploying/k8s -> deploying/kubernetes from Update deploying/k8s -> deploying/kubernetes to Update url path deploying/k8s -> deploying/kubernetes.
Sat, Jan 23, 1:43 AM
prha updated the diff for D6125: adds future schedule tick preview options.

rebase

Sat, Jan 23, 1:35 AM
prha requested review of D6125: adds future schedule tick preview options.
Sat, Jan 23, 1:32 AM
sandyryza added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

@schrockn those concerns make a lot of sense.

Sat, Jan 23, 1:23 AM
rexledesma updated the diff for D6121: Remove Kubernetes part 2.

remove unused images

Sat, Jan 23, 1:17 AM
rexledesma requested review of D6122: Update url path deploying/k8s -> deploying/kubernetes.
Sat, Jan 23, 1:17 AM
rexledesma updated the diff for D6120: Update kubernetes deploy guide.

update last references of k8s -> kubernetes

Sat, Jan 23, 1:03 AM
yuhan accepted D6121: Remove Kubernetes part 2.
Sat, Jan 23, 12:55 AM
rexledesma requested review of D6121: Remove Kubernetes part 2.
Sat, Jan 23, 12:51 AM
rexledesma requested review of D6120: Update kubernetes deploy guide.
Sat, Jan 23, 12:46 AM

Fri, Jan 22

schrockn added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

Yeah my first instinct is stress and terror given our past forays into magically commingling the python and dagster type system.

Fri, Jan 22, 11:36 PM
alangenfeld closed D6076: [mypy][core] execution/plan.
Fri, Jan 22, 10:35 PM
alangenfeld committed R1:6b5f4e846913: [mypy][core] execution/plan (authored by alangenfeld).
[mypy][core] execution/plan
Fri, Jan 22, 10:35 PM
alangenfeld added a comment to D6076: [mypy][core] execution/plan.

It would be nice to have a better story for these named tuples. Unfortunately, it might require mucking around with metaclasses to support overriding init if we want to do runtime type checking.

Fri, Jan 22, 10:35 PM
alangenfeld resigned from D6048: [Memoized Dev Example 2/2] Memoized development example.

others have more context on this than I

Fri, Jan 22, 10:34 PM
alangenfeld requested changes to D6095: [10/n]Make StepOutput user-code-free.
Fri, Jan 22, 10:33 PM
GitHub <noreply@github.com> committed R1:10e62cef7ce8: Fix for the example (access to ADL client) (#3575) (authored by ericct <eric.cheminot@gmail.com>).
Fix for the example (access to ADL client) (#3575)
Fri, Jan 22, 10:29 PM
alangenfeld requested changes to D6094: [9/n] Make StepInputSource use snapshots.
Fri, Jan 22, 10:27 PM
sandyryza added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

@schrockn do you have thoughts here? I think I would be OK with any of the options raised by @alangenfeld. I vaguely recall that you had some concerns about getting fancy with auto-creation/parameterization of types.

Fri, Jan 22, 10:26 PM
sandyryza planned changes to D5934: file IO manager.
Fri, Jan 22, 10:23 PM
alangenfeld requested changes to D6093: [8/n] Make StepInput.dagster_type a snapshot instead (no user code).
Fri, Jan 22, 10:15 PM
dish requested review of D6117: [dagit] Simplify Table styles.
Fri, Jan 22, 8:30 PM
max closed D6114: Dagstermill CLI housekeeping.
Fri, Jan 22, 7:43 PM
max committed R1:9bc5d4b985da: Dagstermill CLI housekeeping (authored by max).
Dagstermill CLI housekeeping
Fri, Jan 22, 7:43 PM
yuhan accepted D6114: Dagstermill CLI housekeeping.
Fri, Jan 22, 7:41 PM
max closed D6113: Repin ipykernel.
Fri, Jan 22, 7:39 PM
max committed R1:3e75d33befd6: Repin ipykernel (authored by max).
Repin ipykernel
Fri, Jan 22, 7:39 PM
dgibson accepted D6113: Repin ipykernel.
Fri, Jan 22, 7:37 PM
max requested review of D6114: Dagstermill CLI housekeeping.
Fri, Jan 22, 7:29 PM
max requested review of D6113: Repin ipykernel.
Fri, Jan 22, 7:28 PM
cdecarolis requested review of D6048: [Memoized Dev Example 2/2] Memoized development example.
Fri, Jan 22, 7:09 PM
max closed D6099: Rename tests.
Fri, Jan 22, 5:28 PM
max committed R1:dd2c9f05751e: Rename tests (authored by max).
Rename tests
Fri, Jan 22, 5:28 PM
sandyryza added inline comments to D6076: [mypy][core] execution/plan.
Fri, Jan 22, 5:20 PM
sandyryza accepted D6076: [mypy][core] execution/plan.

It would be nice to have a better story for these named tuples. Unfortunately, it might require mucking around with metaclasses to support overriding init if we want to do runtime type checking.

Fri, Jan 22, 5:19 PM
johann updated the diff for D6010: Run queuing enabled endpoint.

rebase

Fri, Jan 22, 5:13 PM
prha accepted D6102: yield a SkipReason when a partitioned schedule can't find its partition.
Fri, Jan 22, 5:02 PM