Page MenuHomePhabricator
Feed Advanced Search

Today

nate committed R1:abfa399da70a: Move dagster-airflow construct_variables to dagster-graphql (authored by nate).
Move dagster-airflow construct_variables to dagster-graphql
Thu, Apr 2, 7:59 PM
nate closed D2418: Move dagster-airflow construct_variables to dagster-graphql.
Thu, Apr 2, 7:59 PM
nate updated the diff for D2418: Move dagster-airflow construct_variables to dagster-graphql.

up

Thu, Apr 2, 7:31 PM
nate planned changes to D2409: RFC: DinD + Celery.
Thu, Apr 2, 6:14 PM
nate planned changes to D2334: [dagster-aws] EMR pyspark deploy modes.
Thu, Apr 2, 6:07 PM
nate updated the diff for D2334: [dagster-aws] EMR pyspark deploy modes.

up

Thu, Apr 2, 6:07 PM
nate updated the summary of D2418: Move dagster-airflow construct_variables to dagster-graphql.
Thu, Apr 2, 5:40 PM
nate updated the diff for D2418: Move dagster-airflow construct_variables to dagster-graphql.

up

Thu, Apr 2, 5:37 PM
nate created D2418: Move dagster-airflow construct_variables to dagster-graphql.
Thu, Apr 2, 5:35 PM
nate updated the diff for D2409: RFC: DinD + Celery.

up

Thu, Apr 2, 5:23 PM
nate accepted D2414: unpin grpcio in examples..
Thu, Apr 2, 4:08 PM
nate updated the diff for D2409: RFC: DinD + Celery.

up

Thu, Apr 2, 12:47 AM
nate added a comment to D2410: (refactor-validation-context-1) Rename ValidationContext to TraversalContext.

The Wheel of Time turns, and Ages come and pass, leaving memories that become legend. Legend fades to myth, and even myth is long forgotten when the Age that gave it birth comes again

Thu, Apr 2, 12:15 AM

Yesterday

nate created D2409: RFC: DinD + Celery.
Wed, Apr 1, 11:57 PM
nate accepted D2381: (1-read-only-containers)-container-snapshot-provider-scaffolding.
Wed, Apr 1, 8:51 PM
nate committed R1:bd8c904bb212: Update default celery queue in k8s (authored by nate).
Update default celery queue in k8s
Wed, Apr 1, 8:45 PM
nate closed D2401: Update default celery queue in k8s.
Wed, Apr 1, 8:45 PM
nate added a comment to D2401: Update default celery queue in k8s.

This wasn't working before, because we were servicing the "default" queue by default; but dagster-celery uses "dagster" by default https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-celery/dagster_celery/defaults.py#L13

Wed, Apr 1, 7:03 PM
nate added a comment to D2381: (1-read-only-containers)-container-snapshot-provider-scaffolding.

I think the repository.yaml constraint is reasonable, but maybe we can relax the requirement for the entrypoint being set? I think in our other images we haven't set this, that way the end user can choose to run dagster, dagster-graphql, dagit, etc. on the same image depending on use case

Wed, Apr 1, 4:49 PM
nate accepted D2402: Ignore unmatched partitions in schedule view.
Wed, Apr 1, 2:53 PM
nate created D2401: Update default celery queue in k8s.
Wed, Apr 1, 4:57 AM
nate added inline comments to D2381: (1-read-only-containers)-container-snapshot-provider-scaffolding.
Wed, Apr 1, 3:36 AM
nate added a comment to D2339: InputDefinition.default_value.

this is super nice and clean

Wed, Apr 1, 3:28 AM
nate committed R1:3c6ca147c835: Fix docs issue (authored by nate).
Fix docs issue
Wed, Apr 1, 1:30 AM
nate closed D2398: Fix docs issue.
Wed, Apr 1, 1:30 AM

Tue, Mar 31

nate created D2398: Fix docs issue.
Tue, Mar 31, 11:14 PM
nate planned changes to D2334: [dagster-aws] EMR pyspark deploy modes.

marking planned changes again—going to continue chipping away at this, but wanted to get WIP off my laptop

Tue, Mar 31, 6:59 PM
nate updated the diff for D2334: [dagster-aws] EMR pyspark deploy modes.

up

Tue, Mar 31, 6:57 PM
nate added a comment to D2268: Make _make_airflow_dag a public api.

nice work!

Tue, Mar 31, 3:13 PM

Mon, Mar 30

nate closed D2363: Unset default Dagster image/tag for Helm chart.
Mon, Mar 30, 9:00 PM
nate committed R1:2f2b92187224: Unset default Dagster image/tag for Helm chart (authored by nate).
Unset default Dagster image/tag for Helm chart
Mon, Mar 30, 9:00 PM
nate updated the diff for D2363: Unset default Dagster image/tag for Helm chart.

up

Mon, Mar 30, 8:45 PM
nate abandoned D2133: Move automation to bin.

cool, I'll abandon this diff for now then and we can revisit

Mon, Mar 30, 7:10 PM
nate closed D2366: Helm fix for renamed PG secret.
Mon, Mar 30, 4:10 PM
nate committed R1:6cb744638de2: Helm fix for renamed PG secret (authored by nate).
Helm fix for renamed PG secret
Mon, Mar 30, 4:10 PM
nate updated the diff for D2366: Helm fix for renamed PG secret.

!@$)*&@$)&!% Helm

Mon, Mar 30, 3:53 PM
nate added a comment to D2133: Move automation to bin.

@schrockn thoughts on this? this isn't really a module, so I'm comfortable stashing these in bin/ - just doesn't feel right IMO to have this as a top-level thing in python_modules/

Mon, Mar 30, 3:39 PM
nate retitled D2366: Helm fix for renamed PG secret from Helm fix to Helm fix for renamed PG secret.
Mon, Mar 30, 3:18 PM
nate created D2366: Helm fix for renamed PG secret.
Mon, Mar 30, 3:15 PM
nate updated the summary of D2363: Unset default Dagster image/tag for Helm chart.
Mon, Mar 30, 3:47 AM
nate updated the summary of D2363: Unset default Dagster image/tag for Helm chart.
Mon, Mar 30, 3:46 AM
nate updated the summary of D2363: Unset default Dagster image/tag for Helm chart.
Mon, Mar 30, 3:43 AM
nate created D2363: Unset default Dagster image/tag for Helm chart.
Mon, Mar 30, 12:31 AM

Sun, Mar 29

nate committed R1:bb140ce0e929: Fix AWS ImportError (authored by nate).
Fix AWS ImportError
Sun, Mar 29, 5:41 AM
nate closed D2354: Fix AWS ImportError.
Sun, Mar 29, 5:41 AM
nate added a reviewer for D2354: Fix AWS ImportError: max.
Sun, Mar 29, 5:32 AM

Sat, Mar 28

nate updated the diff for D2354: Fix AWS ImportError.

fix tests

Sat, Mar 28, 4:11 PM
nate added a reviewer for D2354: Fix AWS ImportError: schrockn.
Sat, Mar 28, 3:43 PM
nate updated the diff for D2354: Fix AWS ImportError.

up

Sat, Mar 28, 3:40 PM
nate updated the diff for D2354: Fix AWS ImportError.

up

Sat, Mar 28, 3:36 PM
nate updated the summary of D2354: Fix AWS ImportError.
Sat, Mar 28, 3:19 PM
nate updated the diff for D2354: Fix AWS ImportError.

up

Sat, Mar 28, 3:17 PM
nate created D2354: Fix AWS ImportError.
Sat, Mar 28, 3:15 PM
nate added a comment to D2342: [dagster-aws] Make dagster_aws imports more sane.

yeah saw the github issue, will add back some of the imports until we release again. should be able to make both import from root and import from sub module work. didn’t consider the mixed git clone / PyPI case

Sat, Mar 28, 2:05 PM

Fri, Mar 27

nate committed R1:61fa0a032e4e: Log resource names and resource def names when logging from resources (authored by nate).
Log resource names and resource def names when logging from resources
Fri, Mar 27, 10:51 PM
nate closed D2349: Log resource names and resource def names when logging from resources.
Fri, Mar 27, 10:51 PM
nate updated the diff for D2349: Log resource names and resource def names when logging from resources.

up

Fri, Mar 27, 10:35 PM
nate committed R1:267176a1d204: Add EMR autodocs (authored by nate).
Add EMR autodocs
Fri, Mar 27, 10:20 PM
nate closed D2350: Add EMR autodocs.
Fri, Mar 27, 10:20 PM
nate added a comment to D2350: Add EMR autodocs.

not yet! tbh I'm not even sure what the shape of a future "fake_emr_resource" would be yet, need to start dogfooding this more to get that figured out. I think making progress on D2334 will help

Fri, Mar 27, 10:19 PM
nate created D2350: Add EMR autodocs.
Fri, Mar 27, 9:53 PM
nate committed R1:6261c593d029: [dagster-aws] Make dagster_aws imports more sane (authored by nate).
[dagster-aws] Make dagster_aws imports more sane
Fri, Mar 27, 9:37 PM
nate closed D2342: [dagster-aws] Make dagster_aws imports more sane.
Fri, Mar 27, 9:36 PM
nate updated the diff for D2342: [dagster-aws] Make dagster_aws imports more sane.

comments

Fri, Mar 27, 9:18 PM
nate added reviewers for D2349: Log resource names and resource def names when logging from resources: prha, max.
Fri, Mar 27, 9:06 PM
nate updated the diff for D2349: Log resource names and resource def names when logging from resources.

up

Fri, Mar 27, 9:05 PM
nate updated the diff for D2349: Log resource names and resource def names when logging from resources.

up

Fri, Mar 27, 8:57 PM
nate updated the summary of D2349: Log resource names and resource def names when logging from resources.
Fri, Mar 27, 8:39 PM
nate created D2349: Log resource names and resource def names when logging from resources.
Fri, Mar 27, 8:27 PM
nate accepted D2347: PresetDefinition.with_additional_config.
Fri, Mar 27, 8:00 PM
nate added a comment to D2347: PresetDefinition.with_additional_config.

see one comment

Fri, Mar 27, 8:00 PM
nate accepted D2344: [dagster-k8s] get_celery_engine_config.

cool, this looks reasonable to me—are you thinking we'd also have some way to invoke get_celery_engine_config while loading a preset?

Fri, Mar 27, 5:50 PM
nate added a comment to D2342: [dagster-aws] Make dagster_aws imports more sane.

depending on where they're doing their imports from, it could be, so would want to document / add to changelog / post in #fresh-on-master etc.

Fri, Mar 27, 5:47 PM
nate added a reviewer for D2342: [dagster-aws] Make dagster_aws imports more sane: alangenfeld.
Fri, Mar 27, 5:28 PM
nate committed R1:46b5d605b482: [dagster-k8s] Remove hard-coded resource names (authored by nate).
[dagster-k8s] Remove hard-coded resource names
Fri, Mar 27, 5:02 PM
nate closed D2209: [dagster-k8s] Remove hard-coded resource names.
Fri, Mar 27, 5:02 PM
nate planned changes to D2334: [dagster-aws] EMR pyspark deploy modes.

going to experiment w/ resource factory for this

Fri, Mar 27, 4:48 PM
nate added a comment to D2209: [dagster-k8s] Remove hard-coded resource names.

yes, works and tested for both—instead of relying on the PG secret I provision our own PG secret always so that we have a consistent place to go look for the PG password, regardless of which PG you're using

Fri, Mar 27, 4:43 PM
nate resigned from D2346: [doc] add GA to docs.dagster.io.

LGTM! Will let @sashank do the final sign-off since he's looking at docs stuff right now

Fri, Mar 27, 3:55 PM
nate updated the diff for D2342: [dagster-aws] Make dagster_aws imports more sane.

update autodocs links

Fri, Mar 27, 3:52 PM

Thu, Mar 26

nate updated the diff for D2209: [dagster-k8s] Remove hard-coded resource names.

fix test

Thu, Mar 26, 10:06 PM
nate updated the diff for D2209: [dagster-k8s] Remove hard-coded resource names.

up

Thu, Mar 26, 9:43 PM
nate committed R1:b656835ee30e: [dagster-k8s] Pull in a few Helm chart fixes (authored by nate).
[dagster-k8s] Pull in a few Helm chart fixes
Thu, Mar 26, 3:19 PM
nate closed D2341: [dagster-k8s] Pull in a few Helm chart fixes.
Thu, Mar 26, 3:19 PM
nate added reviewers for D2342: [dagster-aws] Make dagster_aws imports more sane: max, schrockn.
Thu, Mar 26, 12:43 AM
nate updated the summary of D2342: [dagster-aws] Make dagster_aws imports more sane.
Thu, Mar 26, 12:37 AM
nate updated the diff for D2342: [dagster-aws] Make dagster_aws imports more sane.

up

Thu, Mar 26, 12:34 AM
nate created D2342: [dagster-aws] Make dagster_aws imports more sane.
Thu, Mar 26, 12:31 AM

Wed, Mar 25

nate created D2341: [dagster-k8s] Pull in a few Helm chart fixes.
Wed, Mar 25, 11:33 PM
nate added a reviewer for D2334: [dagster-aws] EMR pyspark deploy modes: yuhan.
Wed, Mar 25, 9:37 PM
nate added a reviewer for D2334: [dagster-aws] EMR pyspark deploy modes: max.
Wed, Mar 25, 9:37 PM
nate updated the summary of D2334: [dagster-aws] EMR pyspark deploy modes.
Wed, Mar 25, 9:37 PM
nate updated the diff for D2334: [dagster-aws] EMR pyspark deploy modes.

up

Wed, Mar 25, 9:13 PM
nate accepted D2333: fix viewing logs for a "live" UnknownPipeline.
Wed, Mar 25, 4:29 AM
nate created D2334: [dagster-aws] EMR pyspark deploy modes.
Wed, Mar 25, 12:37 AM

Tue, Mar 24

nate accepted D2325: Revert "(snap-backed-graphql-8) Snaps for mode, resource, and logger; back graphql" and "(snap-backed-graphql-7) Back all dependency-related things via snap".
Tue, Mar 24, 2:05 PM

Mon, Mar 23

nate committed R1:3c57abe9bcc0: Create prod mode and preset for airline demo (authored by nate).
Create prod mode and preset for airline demo
Mon, Mar 23, 11:28 PM
nate closed D2318: Create prod mode and preset for airline demo.
Mon, Mar 23, 11:28 PM
nate updated the diff for D2318: Create prod mode and preset for airline demo.

rebase

Mon, Mar 23, 11:12 PM
nate committed R1:08218f5ea3a1: [dagster-aws] Improve test coverage (authored by nate).
[dagster-aws] Improve test coverage
Mon, Mar 23, 6:26 PM