Page MenuHomePhabricator
Feed Advanced Search

Today

schrockn requested changes to D2364: Use our tokenized text field for log filtering so type: and step: features are easy to find.

Yes it seems to. Req'ing changes so we can chat before landing

Wed, Apr 8, 12:59 AM
schrockn added a comment to D2364: Use our tokenized text field for log filtering so type: and step: features are easy to find.

One last question on this one: does it totally eliminate free text search?

Wed, Apr 8, 12:57 AM

Yesterday

schrockn requested changes to D2467: (container-supported-dagit-4)-refactor-fetch-type-resolvers.
Tue, Apr 7, 11:19 PM
schrockn requested changes to D2466: (container-supported-dagit-3)-hook-container-context-into-cli.
Tue, Apr 7, 11:10 PM
schrockn added inline comments to D2466: (container-supported-dagit-3)-hook-container-context-into-cli.
Tue, Apr 7, 11:07 PM
schrockn requested changes to D2464: (container-supported-dagit-2) - Integrate DagsterContainerGraphQLContext into pipeline_or_X resolvers and add preset error handling..
Tue, Apr 7, 11:00 PM
schrockn requested changes to D2461: (container-supported-dagit-1) - Implement DagsterContainerGraphQLContext and hook into pipelines_or_error.

cool. some minor feedback

Tue, Apr 7, 10:54 PM
schrockn requested changes to D2405: #2346-1 [RFC] multi-step (arbitrary subset) re-execution through dagit Run.

We need to consider what this means from a product perspective. This diff leaves the world with overlapping notions of step subset selection that leaves the world in a more confused state.

Tue, Apr 7, 6:31 PM

Mon, Apr 6

schrockn accepted D2458: Consolidate implementations of dictionary merge #2350.

got it! that all makes sense

Mon, Apr 6, 11:30 PM
schrockn updated the diff for D2438: add IRunLauncher and RunLauncherNext All multiprocess run launcher.

up

Mon, Apr 6, 11:22 PM
schrockn requested changes to D2458: Consolidate implementations of dictionary merge #2350.

bit concerned about bundling up behavior change here

Mon, Apr 6, 11:11 PM
schrockn updated the diff for D2438: add IRunLauncher and RunLauncherNext All multiprocess run launcher.

up

Mon, Apr 6, 11:09 PM
schrockn updated the diff for D2438: add IRunLauncher and RunLauncherNext All multiprocess run launcher.

upmessage

Mon, Apr 6, 10:44 PM
schrockn accepted D2451: Rough error when you pass an uninvoked solid during composition #2308.

thumbsup

Mon, Apr 6, 7:05 PM
schrockn accepted D2455: Fix isort indent to 4 spaces.
Mon, Apr 6, 7:04 PM
schrockn requested changes to D2451: Rough error when you pass an uninvoked solid during composition #2308.

just bouncing to you for queue management while you investigate the weird isort issue per convo in slack

Mon, Apr 6, 6:32 PM
schrockn requested changes to D2451: Rough error when you pass an uninvoked solid during composition #2308.

Cool. Can you include a test case that demonstrates when this is hit and then assert against the error message?

Mon, Apr 6, 4:18 PM

Fri, Apr 3

schrockn updated the diff for D2437: add execution target handle to run launch.

up

Fri, Apr 3, 10:57 PM
schrockn updated the diff for D2437: add execution target handle to run launch.

try optional

Fri, Apr 3, 10:46 PM
schrockn created D2438: add IRunLauncher and RunLauncherNext All multiprocess run launcher.
Fri, Apr 3, 10:33 PM
schrockn created D2437: add execution target handle to run launch.
Fri, Apr 3, 10:32 PM
schrockn created D2434: Remove RemoteDagitRunLauncher.
Fri, Apr 3, 7:42 PM
schrockn abandoned D2407: Changing PipelineRun ctor.
Fri, Apr 3, 7:29 PM
schrockn accepted D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.
Fri, Apr 3, 7:29 PM
schrockn abandoned D2417: Move run creation logic from launcher into instance.

Abandoning this for now to let to run state machine play out

Fri, Apr 3, 7:19 PM
schrockn added inline comments to D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.
Fri, Apr 3, 7:13 PM
schrockn added a comment to D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

Ok this looks good then. Do we have a test to ensure that this works? e.g. that the notebook is actually somewhere where a downstream solid can access it? Also the materialization should still exist even if the output isn't specifed.

Fri, Apr 3, 7:12 PM
schrockn requested changes to D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

This is a definite improvement. The one thing we need to think about here is the the FileManager stuff as current written is meant for ephemeral files, meaning that the files will be deleted after the pipeline is done running. That is probably not what we want here (and we should test for that). (see local_file_manager)

Fri, Apr 3, 6:37 PM
schrockn abandoned D2317: (snap-backed-graphql-11) Content-addressable pipeline.
Fri, Apr 3, 5:41 PM
schrockn accepted D2426: Changelog for 0.7.6.

lgtm

Fri, Apr 3, 12:22 AM

Thu, Apr 2

schrockn added a comment to D2399: pull out step_key into top-level event_log schema.

Definitely think we need to be flexible on data migrations. We can't *require* upgrading all rows for all cases.

Thu, Apr 2, 10:58 PM
schrockn added a comment to D2368: Replace the playground's execution plan preview with a more useful config map.

Also we definitely need to do a pass on improving error messages in general

Thu, Apr 2, 10:55 PM
schrockn accepted D2368: Replace the playground's execution plan preview with a more useful config map.

most excellent

Thu, Apr 2, 10:54 PM
schrockn created Image Macro "yesbaby".
Thu, Apr 2, 10:54 PM
schrockn updated the diff for D2417: Move run creation logic from launcher into instance.

upmessage

Thu, Apr 2, 5:56 PM
schrockn committed R1:fdc8e728f977: (refactor-validation-context-2) Change from post_process bool to enum (authored by schrockn).
(refactor-validation-context-2) Change from post_process bool to enum
Thu, Apr 2, 5:14 PM
schrockn closed D2411: (refactor-validation-context-2) Change from post_process bool to enum.
Thu, Apr 2, 5:14 PM
schrockn accepted D2415: add Nothing input to dbt solid.

we should also talk to dwall about his dbt integration at some point to see if we can upstream

Thu, Apr 2, 5:13 PM
schrockn updated the diff for D2417: Move run creation logic from launcher into instance.

up

Thu, Apr 2, 5:11 PM
schrockn updated the diff for D2417: Move run creation logic from launcher into instance.

up

Thu, Apr 2, 5:07 PM
schrockn updated the diff for D2417: Move run creation logic from launcher into instance.

up

Thu, Apr 2, 5:06 PM
schrockn created D2417: Move run creation logic from launcher into instance.
Thu, Apr 2, 5:01 PM
schrockn updated the diff for D2407: Changing PipelineRun ctor.

up

Thu, Apr 2, 3:30 PM
schrockn updated the diff for D2407: Changing PipelineRun ctor.

up

Thu, Apr 2, 3:24 PM
schrockn updated the diff for D2155: Warm the BK cluster.

up

Thu, Apr 2, 3:12 PM
schrockn accepted D2412: bugfix - use os "agnostic" tmp directory.
Thu, Apr 2, 1:04 AM
schrockn committed R1:ad8d2d116614: (refactor-validation-context-1) Rename ValidationContext to TraversalContext (authored by schrockn).
(refactor-validation-context-1) Rename ValidationContext to TraversalContext
Thu, Apr 2, 12:13 AM
schrockn closed D2410: (refactor-validation-context-1) Rename ValidationContext to TraversalContext.
Thu, Apr 2, 12:13 AM
schrockn added a comment to D2410: (refactor-validation-context-1) Rename ValidationContext to TraversalContext.

haha

Thu, Apr 2, 12:03 AM
schrockn created D2411: (refactor-validation-context-2) Change from post_process bool to enum.
Thu, Apr 2, 12:02 AM

Wed, Apr 1

schrockn created D2410: (refactor-validation-context-1) Rename ValidationContext to TraversalContext.
Wed, Apr 1, 11:57 PM
schrockn committed R1:7da7ac38ff10: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and… (authored by schrockn).
(stable-pipeline-snapshot-2) Add snapshot snapshot tests and…
Wed, Apr 1, 11:55 PM
schrockn closed D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.
Wed, Apr 1, 11:55 PM
schrockn committed R1:df553f5d0106: (stable-pipeline-snapshot-1) Use json rep default value in config type key (authored by schrockn).
(stable-pipeline-snapshot-1) Use json rep default value in config type key
Wed, Apr 1, 11:54 PM
schrockn closed D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.
Wed, Apr 1, 11:53 PM
schrockn added a comment to D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

will do refactor in followup

Wed, Apr 1, 11:53 PM
schrockn updated the diff for D2407: Changing PipelineRun ctor.

up

Wed, Apr 1, 11:52 PM
schrockn updated the diff for D2407: Changing PipelineRun ctor.

up

Wed, Apr 1, 11:48 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

upmessage

Wed, Apr 1, 11:43 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

rebase

Wed, Apr 1, 11:40 PM
schrockn planned changes to D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.
Wed, Apr 1, 11:37 PM
schrockn created D2407: Changing PipelineRun ctor.
Wed, Apr 1, 11:25 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

rebase

Wed, Apr 1, 11:15 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

up

Wed, Apr 1, 11:11 PM
schrockn accepted D2392: (2-read-only-containers)-link-to-pipelin-cli-print-command.

πŸ‘πŸ»

Wed, Apr 1, 11:09 PM
schrockn added inline comments to D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.
Wed, Apr 1, 10:59 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

upmessage

Wed, Apr 1, 10:58 PM
schrockn abandoned D2362: Incorporate default value in type key as serialized json.

I am going to combine this with https://dagster.phacility.com/D2393 because I think it is actually easier to review together

Wed, Apr 1, 10:53 PM
schrockn planned changes to D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.
Wed, Apr 1, 10:51 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

up

Wed, Apr 1, 10:49 PM
schrockn planned changes to D2362: Incorporate default value in type key as serialized json.
Wed, Apr 1, 10:44 PM
schrockn updated the diff for D2362: Incorporate default value in type key as serialized json.

up

Wed, Apr 1, 10:39 PM
schrockn added a comment to D2362: Incorporate default value in type key as serialized json.

So here is the version that just uses the value as the hash. The thing I liked about the previous scheme is that I could cache the hash value and it doesn't consume that much memory. But I don't feel too strongly

Wed, Apr 1, 9:03 PM
schrockn updated the diff for D2362: Incorporate default value in type key as serialized json.

upmessage

Wed, Apr 1, 9:02 PM
schrockn committed R1:408a4e49355e: Only allow raw config values as default_value (authored by schrockn).
Only allow raw config values as default_value
Wed, Apr 1, 8:56 PM
schrockn closed D2388: Only allow raw config values as default_value.
Wed, Apr 1, 8:56 PM
schrockn updated the diff for D2388: Only allow raw config values as default_value.

up

Wed, Apr 1, 8:36 PM
schrockn accepted D2339: InputDefinition.default_value.

redfordnod

Wed, Apr 1, 12:02 PM

Tue, Mar 31

schrockn added a reviewer for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key: themissinghlink.
Tue, Mar 31, 11:39 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

upmessage

Tue, Mar 31, 11:39 PM
schrockn accepted D2324: Add hello/email banner, add telemetry banner, add whitelisted function list to code + docs, make telemetry opt-out.
Tue, Mar 31, 11:25 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

up

Tue, Mar 31, 11:25 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

up

Tue, Mar 31, 11:17 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

up

Tue, Mar 31, 11:06 PM
schrockn accepted D2394: Move InvocationArgs to top level DagsterOperatorInvocationArgs.

πŸ‘πŸ»

Tue, Mar 31, 10:59 PM
schrockn updated the diff for D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

up

Tue, Mar 31, 10:58 PM
schrockn accepted D2395: rename definition/container.py to definition/solid_container.py.

πŸ‘πŸ»

Tue, Mar 31, 10:53 PM
schrockn created D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.
Tue, Mar 31, 10:52 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

up

Tue, Mar 31, 10:42 PM
schrockn added inline comments to D2392: (2-read-only-containers)-link-to-pipelin-cli-print-command.
Tue, Mar 31, 10:39 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

upmessage

Tue, Mar 31, 10:35 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

up

Tue, Mar 31, 10:29 PM
schrockn updated the diff for D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

up

Tue, Mar 31, 10:21 PM
schrockn added inline comments to D2268: Make _make_airflow_dag a public api.
Tue, Mar 31, 10:16 PM
schrockn created D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.
Tue, Mar 31, 10:15 PM
schrockn added a comment to D2388: Only allow raw config values as default_value.

Current state of things the error in the list case is a bit rough but I don't really think we should fix it. We may want to add a specialized error message for this case in the config validation code path (which would apply to non-default-value cases too).

Tue, Mar 31, 8:59 PM
schrockn updated the diff for D2388: Only allow raw config values as default_value.

tests

Tue, Mar 31, 8:55 PM
schrockn accepted D2389: Allow integer types for float config fields.

ya go for it

Tue, Mar 31, 8:29 PM
schrockn committed R1:8680e6f979e1: Add utility is_enum_value (authored by schrockn).
Add utility is_enum_value
Tue, Mar 31, 8:07 PM
schrockn closed D2387: Add utility is_enum_value.
Tue, Mar 31, 8:07 PM