Page MenuHomePhabricator
Feed All Stories

Today

nate requested review of D3144: Move remaining tests to new buildkite ModuleBuildSpec.
Thu, May 28, 5:51 AM
prha committed R1:2e6c85f6e6f9: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName… (authored by prha).
[graphql-environment-8] convert PipelineSelector to use repositoryLocationName…
Thu, May 28, 5:17 AM
prha closed D3138: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName instead of environmentName.
Thu, May 28, 5:17 AM
schrockn requested review of D3141: (hijack-cli-api-all-things-5) Move test_execute_schedule.py to test infra.
Thu, May 28, 5:06 AM
prha updated the diff for D3138: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName instead of environmentName.

rebase

Thu, May 28, 4:47 AM
prha committed R1:2624e8479c87: [graphql-environment-7] convert misc pipeline selector queries off of legacy… (authored by prha).
[graphql-environment-7] convert misc pipeline selector queries off of legacy…
Thu, May 28, 4:34 AM
prha closed D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.
Thu, May 28, 4:34 AM
prha committed R1:cc67d004d7cc: [graphql-environment-6] Rename EnvironmentSchema, environmentConfig =>… (authored by prha).
[graphql-environment-6] Rename EnvironmentSchema, environmentConfig =>…
Thu, May 28, 4:27 AM
prha closed D3120: [graphql-environment-6] Rename EnvironmentSchema, environmentConfig => RunConfigSchema, runConfig (graphql/js).
Thu, May 28, 4:27 AM
catherinewu requested review of D3145: Fan In Pipeline doc example.
Thu, May 28, 4:20 AM
prha committed R1:1dfff874cc1a: [graphql-environment-5] Rename EnvironmentSchema => RunConfigSchema (dagster… (authored by prha).
[graphql-environment-5] Rename EnvironmentSchema => RunConfigSchema (dagster…
Thu, May 28, 4:19 AM
prha closed D3119: [graphql-environment-5] Rename EnvironmentSchema => RunConfigSchema (dagster core only).
Thu, May 28, 4:19 AM
Harbormaster failed remote builds in B12191: Diff 14975 for D3138: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName instead of environmentName!
Thu, May 28, 4:12 AM
prha committed R1:13597313733e: [graphql-environment-4] use pipeline selector for pipelineOrError (authored by prha).
[graphql-environment-4] use pipeline selector for pipelineOrError
Thu, May 28, 4:08 AM
prha closed D3118: [graphql-environment-4] use pipeline selector for pipelineOrError.
Thu, May 28, 4:07 AM
prha committed R1:0394d9114710: [graphql-environment-3] replace pipeline name context with environment context… (authored by prha).
[graphql-environment-3] replace pipeline name context with environment context…
Thu, May 28, 4:02 AM
prha closed D3117: [graphql-environment-3] replace pipeline name context with environment context provider.
Thu, May 28, 4:02 AM
prha committed R1:67b7db1bed94: [graphql-environment-2] add graphql endpoints for Environment (authored by prha).
[graphql-environment-2] add graphql endpoints for Environment
Thu, May 28, 4:01 AM
prha closed D3116: [graphql-environment-2] add graphql endpoints for Environment.
Thu, May 28, 4:01 AM
catherinewu updated the diff for D3142: Docs example for pipeline tags.

up

Thu, May 28, 3:59 AM
prha committed R1:12f65342cdfe: [graphql-environment-1] rename PipelineSelector to PipelineExplorerPath (authored by prha).
[graphql-environment-1] rename PipelineSelector to PipelineExplorerPath
Thu, May 28, 3:57 AM
prha closed D3104: [graphql-environment-1] rename PipelineSelector to PipelineExplorerPath.
Thu, May 28, 3:56 AM
catherinewu requested review of D3142: Docs example for pipeline tags.
Thu, May 28, 3:55 AM
prha updated the diff for D3138: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName instead of environmentName.

rebase

Thu, May 28, 3:51 AM
schrockn added a comment to D3132: [graphql] return HistoricalPipeline from PipelineRun.

*long-term

Thu, May 28, 2:41 AM
schrockn requested changes to D3132: [graphql] return HistoricalPipeline from PipelineRun.

My req changes is solely because I'm curious about what your vision for the future of PipelineReference is? Does it exist in the long-temr/

Thu, May 28, 2:41 AM
catherinewu requested review of D3140: Airflow ingest doc example.
Thu, May 28, 2:37 AM
schrockn accepted D3139: rm dead backcompat code.

Macro burn-it-down:

Thu, May 28, 2:36 AM
alangenfeld requested review of D3132: [graphql] return HistoricalPipeline from PipelineRun.
Thu, May 28, 2:25 AM
schrockn updated the diff for D3136: (hijack-cli-api-all-things-4) Move compute logs to test suite.

up

Thu, May 28, 2:22 AM
alangenfeld requested review of D3139: rm dead backcompat code.
Thu, May 28, 2:22 AM
schrockn updated the diff for D3135: (hijack-cli-api-all-things-3) Move the RetryExecution test suite to use a test matrix.

rebase

Thu, May 28, 2:21 AM
nate added inline comments to D3106: PySpark EMR example.
Thu, May 28, 2:18 AM
nate accepted D3134: Reenable test.

any theories on what changed?

Thu, May 28, 2:17 AM
max committed R1:e5c809203208: Reorganize decorators (authored by max).
Reorganize decorators
Thu, May 28, 2:16 AM
max closed D3137: Reorganize decorators.
Thu, May 28, 2:16 AM
schrockn accepted D3138: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName instead of environmentName.

👍🏻

Thu, May 28, 1:28 AM
schrockn accepted D3137: Reorganize decorators.

so many decorations!

Thu, May 28, 1:27 AM
schrockn accepted D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

cool

Thu, May 28, 1:26 AM
prha requested review of D3138: [graphql-environment-8] convert PipelineSelector to use repositoryLocationName instead of environmentName.
Thu, May 28, 12:25 AM
max requested review of D3137: Reorganize decorators.
Thu, May 28, 12:24 AM
prha added a comment to D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

No, but I was getting kind of difficult merges, so it actually got rebased into https://dagster.phacility.com/D3118

Thu, May 28, 12:18 AM
schrockn requested changes to D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

did you miss a commit in your rebase? i don't see get_legacy_pipeline_selector added in utils.py (there are no changes to utils)

Thu, May 28, 12:16 AM
alangenfeld committed R1:35d8c39d4297: DagsterEnvironment -> RepositoryLocation (authored by alangenfeld).
DagsterEnvironment -> RepositoryLocation
Thu, May 28, 12:08 AM
alangenfeld closed D3129: DagsterEnvironment -> RepositoryLocation.
Thu, May 28, 12:08 AM
yuhan committed R1:72095f5b882f: selector 2/ allow solid_subset to take DSL syntax in python api… (authored by yuhan).
selector 2/ allow solid_subset to take DSL syntax in python api…
Thu, May 28, 12:06 AM
yuhan closed D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
Thu, May 28, 12:06 AM
yuhan retitled D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator from selector 2/ add solid_subset_query to python api execute_pipeline and execute_pipeline_iterator to selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
Thu, May 28, 12:05 AM
yuhan committed R1:08e1ef534471: selector 1/ subselection query to solid_subset (authored by yuhan).
selector 1/ subselection query to solid_subset
Thu, May 28, 12:03 AM
yuhan closed D3112: selector 1/ subselection query to solid_subset.
Thu, May 28, 12:03 AM
max accepted D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
Thu, May 28, 12:00 AM

Yesterday

max accepted D3112: selector 1/ subselection query to solid_subset.
Wed, May 27, 11:59 PM
prha updated the diff for D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

fix tests, update, rebase

Wed, May 27, 11:53 PM
Harbormaster failed remote builds in B12178: Diff 14958 for D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector!
Wed, May 27, 11:52 PM
schrockn requested review of D3136: (hijack-cli-api-all-things-4) Move compute logs to test suite.
Wed, May 27, 11:45 PM
Harbormaster failed remote builds in B12177: Diff 14957 for D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector!
Wed, May 27, 11:43 PM
prha updated the diff for D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

fix rebase

Wed, May 27, 11:41 PM
prha updated the diff for D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

rebase

Wed, May 27, 11:31 PM
prha updated the diff for D3120: [graphql-environment-6] Rename EnvironmentSchema, environmentConfig => RunConfigSchema, runConfig (graphql/js).

rebase

Wed, May 27, 11:30 PM
prha updated the diff for D3119: [graphql-environment-5] Rename EnvironmentSchema => RunConfigSchema (dagster core only).

rebase

Wed, May 27, 11:29 PM
prha updated the diff for D3118: [graphql-environment-4] use pipeline selector for pipelineOrError.

rebase

Wed, May 27, 11:29 PM
prha updated the diff for D3117: [graphql-environment-3] replace pipeline name context with environment context provider.

rebase

Wed, May 27, 11:28 PM
max requested review of D3134: Reenable test.
Wed, May 27, 11:19 PM
sandyryza committed R1:e676762f1a03: Lakehouse renovation (authored by sandyryza).
Lakehouse renovation
Wed, May 27, 11:16 PM
sandyryza closed D2925: Lakehouse renovation.
Wed, May 27, 11:16 PM
sandyryza added inline comments to D2925: Lakehouse renovation.
Wed, May 27, 11:16 PM
alangenfeld updated the diff for D3129: DagsterEnvironment -> RepositoryLocation.

up

Wed, May 27, 11:14 PM
sashank planned changes to D3122: (dagit-out-of-process-2) Add method to get external pipeline w/ API.
Wed, May 27, 11:11 PM
schrockn committed R1:b3700d09ce52: (hijack-cli-api-all-things-2) Rename sync_get_all_logs_for_run to… (authored by schrockn).
(hijack-cli-api-all-things-2) Rename sync_get_all_logs_for_run to…
Wed, May 27, 11:10 PM
schrockn closed D3133: (hijack-cli-api-all-things-2) Rename sync_get_all_logs_for_run to get_all_logs_finished_for_run_via_subscription.
Wed, May 27, 11:10 PM
schrockn committed R1:c716b1dd1076: Move test_execute_pipeline to use entire test executing test matrix (authored by schrockn).
Move test_execute_pipeline to use entire test executing test matrix
Wed, May 27, 11:10 PM
schrockn closed D3131: Move test_execute_pipeline to use entire test executing test matrix.
Wed, May 27, 11:09 PM
schrockn requested review of D3135: (hijack-cli-api-all-things-3) Move the RetryExecution test suite to use a test matrix.
Wed, May 27, 11:09 PM
alangenfeld accepted D3131: Move test_execute_pipeline to use entire test executing test matrix.
Wed, May 27, 11:09 PM
alangenfeld accepted D3133: (hijack-cli-api-all-things-2) Rename sync_get_all_logs_for_run to get_all_logs_finished_for_run_via_subscription.

names

Wed, May 27, 11:07 PM
alangenfeld added inline comments to D3122: (dagit-out-of-process-2) Add method to get external pipeline w/ API.
Wed, May 27, 11:06 PM
schrockn accepted D3129: DagsterEnvironment -> RepositoryLocation.
Wed, May 27, 11:02 PM
alangenfeld added a comment to D3064: Allow PresetDefinitions to wrap ModeDefinitions.

hmm definitely not crazy - will need to think on this one a bit

Wed, May 27, 11:02 PM
yuhan updated the summary of D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
Wed, May 27, 10:52 PM
yuhan updated the diff for D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
  • instead of making it an extra arg, we use the existing solid_subset: execute_pipeline(pipeline, solid_subset=['*some_solid', 'other_solid_a', 'other_solid_b+'])
  • more tests to cover multiple clauses (disjoint, overlapping) and invalid queries
Wed, May 27, 10:50 PM
schrockn requested review of D3133: (hijack-cli-api-all-things-2) Rename sync_get_all_logs_for_run to get_all_logs_finished_for_run_via_subscription.
Wed, May 27, 10:19 PM
alangenfeld closed D3125: pipeline snapshot lineage.
Wed, May 27, 10:08 PM
alangenfeld committed R1:a77e9e2cc703: pipeline snapshot lineage (authored by alangenfeld).
pipeline snapshot lineage
Wed, May 27, 10:08 PM
yuhan updated the summary of D3112: selector 1/ subselection query to solid_subset.
Wed, May 27, 10:05 PM
alangenfeld updated the diff for D3129: DagsterEnvironment -> RepositoryLocation.

up

Wed, May 27, 10:05 PM
yuhan added inline comments to D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
Wed, May 27, 9:58 PM
yuhan updated the diff for D3112: selector 1/ subselection query to solid_subset.
  • get_solid_subset_from_query(pipeline_def, query) where query is str => parse_solid_subset(pipeline_def, solid_subset) where solid_subset is List[str]
  • dedup
  • more tests to cover multiple clauses, invalid syntax, and duplication
Wed, May 27, 9:58 PM
alangenfeld added inline comments to D3129: DagsterEnvironment -> RepositoryLocation.
Wed, May 27, 9:49 PM
schrockn accepted D3125: pipeline snapshot lineage.

makeitso

Wed, May 27, 9:47 PM
alangenfeld accepted D2925: Lakehouse renovation.

fine place to start - some edges to polish as we use it a bit and understand the ergonomics better

Wed, May 27, 9:44 PM
schrockn updated the diff for D3131: Move test_execute_pipeline to use entire test executing test matrix.

up

Wed, May 27, 9:38 PM
alangenfeld updated the diff for D3125: pipeline snapshot lineage.

how bout just PipelineSnapshotLineage?

Wed, May 27, 9:35 PM
schrockn updated the diff for D3131: Move test_execute_pipeline to use entire test executing test matrix.

up

Wed, May 27, 9:32 PM
alangenfeld added inline comments to D3113: selector 2/ allow solid_subset to take DSL syntax in python api execute_pipeline and execute_pipeline_iterator.
Wed, May 27, 9:30 PM
schrockn requested changes to D3127: [graphql-environment-7] convert misc pipeline selector queries off of legacy selector.

Seems like this name should indicate that it is a legacy/single-env-single-repo codepath

Wed, May 27, 9:28 PM
schrockn requested changes to D3125: pipeline snapshot lineage.

mostly for name

Wed, May 27, 9:25 PM
schrockn requested changes to D3129: DagsterEnvironment -> RepositoryLocation.

Just req'ing changes so we can get the docs in there that @prha mentioned because I think that's a good call.

Wed, May 27, 9:17 PM
nate committed R1:f4d54382ec87: Move pylint to package tox files (authored by nate).
Move pylint to package tox files
Wed, May 27, 9:14 PM
nate closed D3109: Move pylint to package tox files.
Wed, May 27, 9:14 PM
schrockn added a comment to D3129: DagsterEnvironment -> RepositoryLocation.

Broadly speaking i think the code i more clear if we say "repository_location" rather than "location" in most places

Wed, May 27, 9:13 PM