Page MenuHomePhabricator
Feed All Stories

Today

prha committed R1:feb2c0a63d43: Re-enable test_reconstructable.py tests (authored by prha).
Re-enable test_reconstructable.py tests
Tue, Aug 11, 12:30 AM
prha closed D4143: Re-enable test_reconstructable.py tests.
Tue, Aug 11, 12:30 AM
aj.nadel requested review of D4154: WIP implement composite solids.
Tue, Aug 11, 12:20 AM

Yesterday

nate accepted D4155: kind cluster info dump in buildkite.

see one comment, otherwise LGTM! thanks for setting this up

Mon, Aug 10, 11:16 PM
Harbormaster failed remote builds in B16747: Diff 20422 for D4129: implement configured intermediate_storage, system_storage!
Mon, Aug 10, 11:16 PM
johann requested review of D4155: kind cluster info dump in buildkite.
Mon, Aug 10, 11:12 PM
aj.nadel updated the diff for D4129: implement configured intermediate_storage, system_storage.

black

Mon, Aug 10, 11:02 PM
Harbormaster failed remote builds in B16745: Diff 20419 for D4129: implement configured intermediate_storage, system_storage!
Mon, Aug 10, 10:58 PM
aj.nadel updated the diff for D4129: implement configured intermediate_storage, system_storage.

up

Mon, Aug 10, 10:33 PM
yuhan planned changes to D4107: step-selection-1 parse step selection.
Mon, Aug 10, 9:26 PM
dgibson abandoned D4078: Move execute_pipeline to not run any user code.

Moved this diff to do the exact opposite, in https://dagster.phacility.com/D4156

Mon, Aug 10, 8:53 PM
aj.nadel requested review of D4129: implement configured intermediate_storage, system_storage.
Mon, Aug 10, 6:23 PM
sandyryza added inline comments to D4087: nicer error on failure to load repository in external process.
Mon, Aug 10, 5:25 PM
sandyryza accepted D4098: (Depends on D4070) Incorporated IntermediateStore features into IntermediateStorage, fixed tests and subclasses accordingly..

This looks great! My one lingering concern is about whether we need to maintain compatibility with the TypeStoragePlugin interface for now.

Mon, Aug 10, 5:09 PM
dgibson updated the diff for D4145: Refactor CLI tests to pass in contexts for managed and unmanaged GRPC servers.

mock DagsterInstance.get() when calling CliRunner functions, so that run_launcher.join() will work

Mon, Aug 10, 4:30 PM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

slightly grosser fix to deal with managed grpc servers in a similar way - need to find any handles that the grpc run launcher launched, shut down their server processes, and wait for them to finish (we don't create them in a test context so we can't clean them up there)

Mon, Aug 10, 3:48 PM
johann requested review of D4152: Pass tags to PipelineSubsetDef.
Mon, Aug 10, 3:39 PM
dgibson requested review of D4151: Throw from sync_get_external_execution_plan rather than relying on callsites to check error data.
Mon, Aug 10, 3:14 PM
dgibson requested changes to D4122: Even softer termination scheme.

yeah I think we concluded at the end of last week that the existing behavior is actually giving us what we want?

Mon, Aug 10, 3:13 PM
dgibson added inline comments to D4087: nicer error on failure to load repository in external process.
Mon, Aug 10, 2:14 PM
Harbormaster failed remote builds in B16730: Diff 20403 for D4145: Refactor CLI tests to pass in contexts for managed and unmanaged GRPC servers!
Mon, Aug 10, 1:51 PM
dgibson updated the diff for D4145: Refactor CLI tests to pass in contexts for managed and unmanaged GRPC servers.

up

Mon, Aug 10, 1:29 PM
dgibson accepted D4143: Re-enable test_reconstructable.py tests.
Mon, Aug 10, 1:10 PM
dgibson abandoned D4111: Make CLI tests that execute over pipelines include managed and unmanaged GRPC pipelines.
Mon, Aug 10, 1:09 PM
Harbormaster failed remote builds in B16729: Diff 20402 for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention)!
Mon, Aug 10, 2:04 AM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

up

Mon, Aug 10, 1:41 AM

Sun, Aug 9

dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

up

Sun, Aug 9, 1:37 AM
Harbormaster failed remote builds in B16726: Diff 20399 for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention)!
Sun, Aug 9, 1:17 AM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

py2

Sun, Aug 9, 1:00 AM

Sat, Aug 8

Harbormaster failed remote builds in B16725: Diff 20398 for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention)!
Sat, Aug 8, 7:03 PM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

Also wait for any grpc server processes to make sure they're all cleaned up before the instance tries to clean up.

Sat, Aug 8, 6:49 PM
dgibson planned changes to D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).
Sat, Aug 8, 6:10 PM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

test fixes

Sat, Aug 8, 5:20 PM
Harbormaster failed remote builds in B16721: Diff 20394 for D4140: test subset execution in celery-k8s!
Sat, Aug 8, 12:05 PM
johann updated the diff for D4140: test subset execution in celery-k8s.

rebase

Sat, Aug 8, 11:48 AM
dgibson planned changes to D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).
Sat, Aug 8, 2:28 AM
yuhan added a comment to D4119: [docs] examples/hooks.

I've followed this example flow locally and replaced the token string in the yaml with my own one. It feels pretty lightweight and easy to follow.

Sat, Aug 8, 12:22 AM
Harbormaster failed remote builds in B16717: Diff 20390 for D4140: test subset execution in celery-k8s!
Sat, Aug 8, 12:19 AM

Fri, Aug 7

dgibson updated the summary of D4145: Refactor CLI tests to pass in contexts for managed and unmanaged GRPC servers.
Fri, Aug 7, 11:58 PM
Harbormaster failed remote builds in B16715: Diff 20388 for D4140: test subset execution in celery-k8s!
Fri, Aug 7, 11:56 PM
dgibson requested review of D4145: Refactor CLI tests to pass in contexts for managed and unmanaged GRPC servers.
Fri, Aug 7, 11:42 PM
yuhan requested review of D4147: ResourceDefinition.mock_resource.
Fri, Aug 7, 11:41 PM
johann updated the summary of D4140: test subset execution in celery-k8s.
Fri, Aug 7, 11:39 PM
johann updated the diff for D4140: test subset execution in celery-k8s.

reset change

Fri, Aug 7, 11:34 PM
johann committed R1:7ad30528a4a9: 0.9.1 (authored by johann).
0.9.1
Fri, Aug 7, 11:26 PM
johann updated the diff for D4140: test subset execution in celery-k8s.

test that it fails without fixing diff

Fri, Aug 7, 11:21 PM
johann committed R1:81c7a007d30c: 0.9.1.pre1 (authored by johann).
0.9.1.pre1
Fri, Aug 7, 11:11 PM
johann committed R1:fafd0515a840: changelog (authored by johann).
changelog
Fri, Aug 7, 11:09 PM
johann closed D4146: changelog.
Fri, Aug 7, 11:09 PM
johann requested review of D4146: changelog.
Fri, Aug 7, 11:04 PM
johann committed R1:02f5802cb031: 0.9.1.pre0 (authored by johann).
0.9.1.pre0
Fri, Aug 7, 10:47 PM
johann updated the diff for D4140: test subset execution in celery-k8s.

modify config

Fri, Aug 7, 10:19 PM
Harbormaster failed remote builds in B16709: Diff 20381 for D4140: test subset execution in celery-k8s!
Fri, Aug 7, 9:40 PM
johann added reviewers for D4140: test subset execution in celery-k8s: nate, prha, max.
Fri, Aug 7, 9:10 PM
johann updated the summary of D4140: test subset execution in celery-k8s.
Fri, Aug 7, 9:09 PM
johann updated the diff for D4140: test subset execution in celery-k8s.

rebase

Fri, Aug 7, 9:08 PM
dgibson committed R1:c7396610b677: One more flaky test failure on windows (authored by dgibson).
One more flaky test failure on windows
Fri, Aug 7, 8:55 PM
dgibson closed D4144: One more flaky test failure on windows.
Fri, Aug 7, 8:55 PM
prha accepted D4144: One more flaky test failure on windows.
Fri, Aug 7, 8:41 PM
dgibson requested review of D4144: One more flaky test failure on windows.
Fri, Aug 7, 8:40 PM
Harbormaster failed remote builds in B16705: Diff 20374 for D4140: test subset execution in celery-k8s!
Fri, Aug 7, 8:17 PM
prha requested review of D4143: Re-enable test_reconstructable.py tests.
Fri, Aug 7, 8:13 PM
johann updated the diff for D4140: test subset execution in celery-k8s.

up

Fri, Aug 7, 7:43 PM
johann committed R1:e5a1e449ac5f: support subsets in execute_step_with_structured_logs (authored by johann).
support subsets in execute_step_with_structured_logs
Fri, Aug 7, 7:32 PM
johann closed D4139: support subsets in execute_step_with_structured_logs.
Fri, Aug 7, 7:32 PM
prha accepted D4139: support subsets in execute_step_with_structured_logs.

betty

Fri, Aug 7, 7:29 PM
johann added reviewers for D4139: support subsets in execute_step_with_structured_logs: prha, max, nate.
Fri, Aug 7, 7:27 PM
johann updated the test plan for D4139: support subsets in execute_step_with_structured_logs.
Fri, Aug 7, 7:27 PM
prha committed R1:172a9b0ca1ff: make ipc errors easier to read (authored by prha).
make ipc errors easier to read
Fri, Aug 7, 6:50 PM
prha closed D4133: make ipc errors easier to read.
Fri, Aug 7, 6:50 PM
schrockn accepted D4133: make ipc errors easier to read.
Fri, Aug 7, 6:25 PM
prha added a comment to D4133: make ipc errors easier to read.

spurious sqlite issue ^^ rerunning-turns-green

Fri, Aug 7, 6:24 PM
prha requested review of D4133: make ipc errors easier to read.
Fri, Aug 7, 6:23 PM
dgibson requested review of D4138: Replace --host CLI param with --grpc_host to avoid conflicting with dagit --host param.
Fri, Aug 7, 6:14 PM
johann requested review of D4139: support subsets in execute_step_with_structured_logs.
Fri, Aug 7, 5:57 PM
cdecarolis requested review of D4021: lakehouse example.
Fri, Aug 7, 5:49 PM
johann requested review of D4140: test subset execution in celery-k8s.
Fri, Aug 7, 5:48 PM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

disable telemetry by default in CLI tests, use a helper method for mocked instances that will persist if you call them in subprocesses

Fri, Aug 7, 5:48 PM
dgibson committed R1:c7e0ed5f5b2a: Disable tests currently breaking Windows (authored by dgibson).
Disable tests currently breaking Windows
Fri, Aug 7, 5:48 PM
dgibson closed D4135: Disable tests currently breaking Windows.
Fri, Aug 7, 5:48 PM
dgibson planned changes to D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

tests are still failing on windows due to the telemetry log

Fri, Aug 7, 5:38 PM
Harbormaster failed remote builds in B16691: Diff 20355 for D4137: Add cleanup thread to grpc server!
Fri, Aug 7, 4:44 PM
dgibson requested review of D4137: Add cleanup thread to grpc server.
Fri, Aug 7, 4:39 PM
Harbormaster failed remote builds in B16693: Diff 20357 for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention)!
Fri, Aug 7, 4:33 PM
Harbormaster failed remote builds in B16692: Diff 20356 for D4135: Disable tests currently breaking Windows!
Fri, Aug 7, 4:30 PM
schrockn accepted D4135: Disable tests currently breaking Windows.
Fri, Aug 7, 4:25 PM
nate committed R1:24a576bd26fe: Rename runmaster to run_coordinator (authored by nate).
Rename runmaster to run_coordinator
Fri, Aug 7, 4:18 PM
nate closed D4127: Rename runmaster to run_coordinator.
Fri, Aug 7, 4:18 PM
dgibson updated the diff for D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).

up

Fri, Aug 7, 4:18 PM
dgibson updated the diff for D4135: Disable tests currently breaking Windows.

one more

Fri, Aug 7, 4:17 PM
cdecarolis requested review of D4098: (Depends on D4070) Incorporated IntermediateStore features into IntermediateStorage, fixed tests and subclasses accordingly..
Fri, Aug 7, 3:43 PM
johann updated the diff for D4134: add editable image to dagster-image.

update k8s-example image

Fri, Aug 7, 3:37 PM
johann accepted D4127: Rename runmaster to run_coordinator.
Fri, Aug 7, 3:32 PM
nate accepted D4134: add editable image to dagster-image.
Fri, Aug 7, 3:25 PM
cdecarolis requested review of D4070: renamed InMemoryIntermediatesManager and IntermediateStoreIntermediatesManager.
Fri, Aug 7, 3:24 PM
nate added reviewers for D4127: Rename runmaster to run_coordinator: catherinewu, johann.
Fri, Aug 7, 3:11 PM
dgibson requested review of D4136: Re-enable CLI execute/launch/backfill commands on windows (by eliminating sources of tempdir contention).
Fri, Aug 7, 2:54 PM
dgibson updated the diff for D4135: Disable tests currently breaking Windows.

more cli tests

Fri, Aug 7, 2:25 PM
dgibson updated the diff for D4135: Disable tests currently breaking Windows.

add issue #

Fri, Aug 7, 2:19 PM
dgibson requested review of D4135: Disable tests currently breaking Windows.
Fri, Aug 7, 2:17 PM