Page MenuHomePhabricator

nate (Nate Kupp)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Apr 23 2019, 10:22 PM (73 w, 5 d)
Roles
Administrator

Recent Activity

Sat, Sep 19

nate updated the diff for D4487: Add retries to S3 object store client.

rebase

Sat, Sep 19, 8:36 PM
nate updated the diff for D4487: Add retries to S3 object store client.

up

Sat, Sep 19, 4:16 PM
nate added a comment to D4487: Add retries to S3 object store client.

I don't know how best to test this, but see https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html

Sat, Sep 19, 3:38 AM
nate requested review of D4487: Add retries to S3 object store client.
Sat, Sep 19, 3:35 AM

Thu, Sep 17

nate committed R1:44418d67da1b: Add retries to dagster-postgres connections (authored by nate).
Add retries to dagster-postgres connections
Thu, Sep 17, 10:14 PM
nate closed D4449: Add retries to dagster-postgres connections.
Thu, Sep 17, 10:14 PM
nate updated the diff for D4449: Add retries to dagster-postgres connections.

up

Thu, Sep 17, 3:50 PM
nate added inline comments to D4449: Add retries to dagster-postgres connections.
Thu, Sep 17, 3:48 PM
nate added reviewers for D4449: Add retries to dagster-postgres connections: catherinewu, johann.
Thu, Sep 17, 3:02 AM

Wed, Sep 16

nate added inline comments to D4449: Add retries to dagster-postgres connections.
Wed, Sep 16, 10:38 PM
nate added reviewers for D4449: Add retries to dagster-postgres connections: alangenfeld, dgibson, max.
Wed, Sep 16, 10:28 PM
nate updated the diff for D4449: Add retries to dagster-postgres connections.

up

Wed, Sep 16, 10:24 PM
nate updated the diff for D4449: Add retries to dagster-postgres connections.

up

Wed, Sep 16, 10:08 PM
nate updated the diff for D4449: Add retries to dagster-postgres connections.

up

Wed, Sep 16, 9:56 PM
nate updated the summary of D4449: Add retries to dagster-postgres connections.
Wed, Sep 16, 9:54 PM
nate updated the diff for D4449: Add retries to dagster-postgres connections.

up

Wed, Sep 16, 9:40 PM
nate updated the diff for D4449: Add retries to dagster-postgres connections.

up

Wed, Sep 16, 9:39 PM
nate requested review of D4449: Add retries to dagster-postgres connections.
Wed, Sep 16, 9:20 PM

Tue, Sep 15

nate accepted D4429: log link to databricks ui after launching job.
Tue, Sep 15, 11:19 PM
nate accepted D4411: get spark-s3 type storage plugin working with configured storages.
Tue, Sep 15, 8:26 PM

Mon, Sep 14

nate accepted D4423: apidoc for emr_pyspark_step_launcher.
Mon, Sep 14, 7:00 PM

Fri, Sep 11

nate accepted D4403: default name for create_databricks_job_solid.
Fri, Sep 11, 6:18 PM
nate committed R1:dfce9a07ab1d: Make Scala event processing less fancy (authored by nate).
Make Scala event processing less fancy
Fri, Sep 11, 4:00 PM
nate closed D4399: Make Scala event processing less fancy.
Fri, Sep 11, 4:00 PM
nate requested review of D4399: Make Scala event processing less fancy.
Fri, Sep 11, 4:49 AM
nate accepted D4395: fix databricks api call.
Fri, Sep 11, 1:43 AM

Thu, Sep 10

nate accepted D4387: add tests for pyspark resource.
Thu, Sep 10, 7:04 PM

Thu, Aug 27

nate accepted D3972: Use configured in emr pyspark example.

nice! this is sweet

Thu, Aug 27, 11:40 PM

Tue, Aug 25

nate accepted D4238: Drop support for Python 2 in dagster-pyspark.
Tue, Aug 25, 10:19 PM

Aug 21 2020

nate accepted D4163: Turn on auto-formatting for string literals in black.

after a long stint of meticulously switching double quotes to single wherever I found them, given the abstentions, I'm happy to be the one to go ahead and greenlight this. Getting tooling support is super attractive

Aug 21 2020, 8:53 PM

Aug 20 2020

nate added a comment to D4238: Drop support for Python 2 in dagster-pyspark.

sounds good. will also need to remove from buildkite pipeline.py if we still test 27 there

Aug 20 2020, 6:04 PM
nate accepted D4233: update default helm images.
Aug 20 2020, 3:36 PM

Aug 19 2020

nate added a comment to D4216: [docs] Releases and deprecations policy.

cool, this lgtm - will let others chime in before the final sign-off

Aug 19 2020, 4:33 PM
nate accepted D4212: include user configmap in user code deployment by default.
Aug 19 2020, 4:31 PM

Aug 17 2020

nate accepted D4202: add missing escape character in pyspark makefile.
Aug 17 2020, 2:51 PM

Aug 12 2020

nate accepted D4162: fix celery-k8s code example.
Aug 12 2020, 5:46 PM
nate added a comment to D4162: fix celery-k8s code example.

yes, we should make this a literalinclude instead of a code-block - mind taking that on this diff? it'll ensure we don't end up in this state again

Aug 12 2020, 3:21 PM

Aug 11 2020

nate accepted D4158: updage java debian version in bk image.

cool this looks good to me. assume you're on top of this, but you'll have to also update the published integration image to "rebase" on this updated image - I think this is documented in the readme? lmk if you need a hand

Aug 11 2020, 8:41 PM

Aug 10 2020

nate accepted D4155: kind cluster info dump in buildkite.

see one comment, otherwise LGTM! thanks for setting this up

Aug 10 2020, 11:16 PM

Aug 7 2020

nate committed R1:24a576bd26fe: Rename runmaster to run_coordinator (authored by nate).
Rename runmaster to run_coordinator
Aug 7 2020, 4:18 PM
nate closed D4127: Rename runmaster to run_coordinator.
Aug 7 2020, 4:18 PM
nate accepted D4134: add editable image to dagster-image.
Aug 7 2020, 3:25 PM
nate added reviewers for D4127: Rename runmaster to run_coordinator: catherinewu, johann.
Aug 7 2020, 3:11 PM

Aug 6 2020

nate accepted D4065: update protobuf to 3.12.4.

see one comment otherwise lgtm, thanks for figuring this out!

Aug 6 2020, 8:57 PM
nate accepted D4130: GRPC Docs.
Aug 6 2020, 8:45 PM
nate added a comment to D4128: k8s scheduler and k8s user code deployment docs.

LGTM - @johann do you want to take a look before Cat lands this?

Aug 6 2020, 7:05 PM
nate accepted D4126: k8s scheduler api docs.
Aug 6 2020, 5:52 PM
nate requested review of D4127: Rename runmaster to run_coordinator.
Aug 6 2020, 4:49 PM
nate added a comment to D4110: update k8s images.
dagster-image build-all --name k8s-dagit
dagster-image build-all --name k8s-example
dagster-image build-all --name k8s-celery-worker
dagster-image push-dockerhub --name k8s-dagit
dagster-image push-dockerhub --name k8s-example
dagster-image push-dockerhub --name k8s-celery-worker
Aug 6 2020, 4:37 PM
nate accepted D4110: update k8s images.

Looks great, thanks for doing this. after you build and push the dagit image to docker hub, can you also update the helm chart to use the dagit image?

Aug 6 2020, 4:36 PM
nate added inline comments to D4119: [docs] examples/hooks.
Aug 6 2020, 3:22 PM
nate committed R1:3b26a27eed5d: Add retries for flakey integration tests (authored by nate).
Add retries for flakey integration tests
Aug 6 2020, 3:19 PM
nate closed D4114: Add retries for flakey integration tests.
Aug 6 2020, 3:19 PM
nate committed R1:299ee4b1511b: Update language around py38 in docs (authored by nate).
Update language around py38 in docs
Aug 6 2020, 3:15 AM
nate closed D4109: Update language around py38 in docs.
Aug 6 2020, 3:15 AM

Aug 5 2020

nate requested review of D4114: Add retries for flakey integration tests.
Aug 5 2020, 11:17 PM
nate accepted D4113: [docs] dagit overview.

few nits, otherwise lgtm!

Aug 5 2020, 10:38 PM
nate added inline comments to D4110: update k8s images.
Aug 5 2020, 9:12 PM
nate updated the diff for D4109: Update language around py38 in docs.

buildkite is so flaky

Aug 5 2020, 8:25 PM
nate requested review of D4109: Update language around py38 in docs.
Aug 5 2020, 8:25 PM
nate accepted D4041: k8s user code deployments.

This is great. Left a few comments on some small nits throughout, otherwise this looks good to land.

Aug 5 2020, 4:03 AM
nate added a reviewer for D4041: k8s user code deployments: nate.
Aug 5 2020, 3:48 AM

Aug 4 2020

nate committed R1:ef441447aedc: Full support for Python 3.8 (authored by nate).
Full support for Python 3.8
Aug 4 2020, 10:59 PM
nate closed D4091: Full support for Python 3.8.
Aug 4 2020, 10:59 PM
nate requested changes to D4094: Don't load pyspark examples in py38.

I think we can just do D4091 instead

Aug 4 2020, 6:58 PM
nate added inline comments to D4091: Full support for Python 3.8.
Aug 4 2020, 6:49 PM
nate updated the diff for D4091: Full support for Python 3.8.

up

Aug 4 2020, 6:48 PM
nate retitled D4091: Full support for Python 3.8 from Full test suppport for Python 3.8 to Full support for Python 3.8.
Aug 4 2020, 6:47 PM
nate requested review of D4091: Full support for Python 3.8.
Aug 4 2020, 6:30 PM
nate added inline comments to D4041: k8s user code deployments.
Aug 4 2020, 4:46 PM
nate accepted D3987: Initialize docker grpc server with target.

LGTM, rebase should deal w/ the GCP build failurees

Aug 4 2020, 3:55 PM
nate accepted D3984: Test grpc ping inside container.

LGTM pending time.sleep change

Aug 4 2020, 3:52 PM
nate accepted D4082: Support specifying env secrets in k8s scheduler.
Aug 4 2020, 3:13 PM

Aug 3 2020

nate abandoned D3915: Presentation + k8s.
Aug 3 2020, 10:32 PM
nate accepted D4064: remove google bigquery pin.
Aug 3 2020, 3:58 PM

Jul 30 2020

nate committed R1:bd174d0a5050: Fix for release CLI (authored by nate).
Fix for release CLI
Jul 30 2020, 11:38 PM
nate closed D4052: Fix for release CLI.
Jul 30 2020, 11:38 PM
nate published D4052: Fix for release CLI for review.
Jul 30 2020, 11:35 PM

Jul 29 2020

nate committed R1:d374859dc6d1: Turn flower off by default (authored by nate).
Turn flower off by default
Jul 29 2020, 5:54 PM
nate closed D4033: Turn flower off by default.
Jul 29 2020, 5:54 PM
nate requested review of D4033: Turn flower off by default.
Jul 29 2020, 4:31 PM
nate committed R1:07e7f493a798: Removing Dask image build until we're ready to properly support it for Dask… (authored by nate).
Removing Dask image build until we're ready to properly support it for Dask…
Jul 29 2020, 4:13 PM
nate closed D4028: Removing Dask image build until we're ready to properly support it for Dask cluster tests.
Jul 29 2020, 4:13 PM
nate requested review of D4028: Removing Dask image build until we're ready to properly support it for Dask cluster tests.
Jul 29 2020, 12:13 AM

Jul 28 2020

nate accepted D3916: Spin up grpc server in container.
Jul 28 2020, 11:27 PM
nate added a comment to D4027: [celery-k8s] add celery config source to run launcher.

it'd be nice to test this somehow.....maybe file a bug that we should get test coverage on these?

Jul 28 2020, 11:11 PM
nate accepted D4027: [celery-k8s] add celery config source to run launcher.
Jul 28 2020, 11:11 PM
nate committed R1:2481cf3b06f7: Fix for dagster-airflow tags (authored by nate).
Fix for dagster-airflow tags
Jul 28 2020, 5:00 PM
nate closed D4017: Fix for dagster-airflow tags.
Jul 28 2020, 5:00 PM
nate updated the summary of D4017: Fix for dagster-airflow tags.
Jul 28 2020, 3:39 PM
nate updated the diff for D4017: Fix for dagster-airflow tags.

up

Jul 28 2020, 3:05 AM
nate updated the diff for D4017: Fix for dagster-airflow tags.

up

Jul 28 2020, 1:11 AM
nate updated the diff for D4017: Fix for dagster-airflow tags.

up

Jul 28 2020, 12:17 AM

Jul 27 2020

nate updated the diff for D4017: Fix for dagster-airflow tags.

up

Jul 27 2020, 11:51 PM
nate updated the diff for D4017: Fix for dagster-airflow tags.

up

Jul 27 2020, 11:48 PM
nate updated the diff for D4017: Fix for dagster-airflow tags.

add tests and refactor DagsterDockerOperator

Jul 27 2020, 11:43 PM
nate requested review of D4017: Fix for dagster-airflow tags.
Jul 27 2020, 9:53 PM
nate accepted D3996: dockerfile tweaks.

looks reasonable - is there a way to confirm that this will do the right thing despite the lack of --no-cache? would be good to ensure that a code change does properly get propagated to the built image vs. just re-using the last cached version

Jul 27 2020, 5:27 PM
nate accepted D4013: pin google-cloud-storage .
Jul 27 2020, 5:20 PM

Jul 24 2020

nate accepted D3999: release automation check_versions bug.
Jul 24 2020, 3:32 PM