Page MenuHomePhabricator

natekupp (Nate Kupp)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Apr 23 2019, 10:22 PM (29 w, 1 d)
Roles
Administrator

Recent Activity

Yesterday

natekupp accepted D1414: pin google-cloud-storage until they fix deps.
Wed, Nov 13, 6:40 PM
natekupp updated the diff for D1349: pyspark on EMR.

fixes for using Spark configs, etc.

Wed, Nov 13, 5:27 PM
natekupp updated the summary of D1349: pyspark on EMR.
Wed, Nov 13, 1:02 AM
natekupp updated the diff for D1349: pyspark on EMR.

up

Wed, Nov 13, 1:00 AM
natekupp updated the diff for D1349: pyspark on EMR.

update to resource-based implementation

Wed, Nov 13, 12:50 AM
natekupp added inline comments to D1405: Improve airflow docs.
Wed, Nov 13, 12:32 AM

Tue, Nov 12

natekupp planned changes to D1349: pyspark on EMR.

Going to move this to resources

Tue, Nov 12, 8:00 PM
natekupp accepted D1401: Add audit check for versions.
Tue, Nov 12, 6:38 PM

Mon, Nov 11

natekupp updated the diff for D1397: [dagster-airflow] Minimize core dependencies.

comments

Mon, Nov 11, 10:21 PM
natekupp updated the summary of D1349: pyspark on EMR.
Mon, Nov 11, 7:45 PM
natekupp updated the summary of D1349: pyspark on EMR.
Mon, Nov 11, 7:44 PM
natekupp added inline comments to D1349: pyspark on EMR.
Mon, Nov 11, 7:43 PM
natekupp retitled D1349: pyspark on EMR from RFC: pyspark on EMR to pyspark on EMR.
Mon, Nov 11, 5:30 AM
natekupp updated the diff for D1349: pyspark on EMR.

up

Mon, Nov 11, 5:28 AM
natekupp updated the diff for D1349: pyspark on EMR.

up

Mon, Nov 11, 2:49 AM
natekupp updated the diff for D1349: pyspark on EMR.

up

Mon, Nov 11, 2:33 AM

Sun, Nov 10

natekupp updated the diff for D1349: pyspark on EMR.

rebase

Sun, Nov 10, 11:51 PM
natekupp updated the diff for D1349: pyspark on EMR.

Taking another pass and cleaning things up

Sun, Nov 10, 11:50 PM
natekupp updated the summary of D1397: [dagster-airflow] Minimize core dependencies.
Sun, Nov 10, 6:06 AM
natekupp updated the diff for D1397: [dagster-airflow] Minimize core dependencies.

up

Sun, Nov 10, 5:54 AM
natekupp updated the diff for D1397: [dagster-airflow] Minimize core dependencies.

up

Sun, Nov 10, 5:43 AM
natekupp created D1397: [dagster-airflow] Minimize core dependencies.
Sun, Nov 10, 5:34 AM

Thu, Nov 7

natekupp updated the diff for D1359: Add support for using GCS for system storage.

fix pylint

Thu, Nov 7, 11:19 PM
natekupp updated the diff for D1359: Add support for using GCS for system storage.

up - fix rebase issues

Thu, Nov 7, 11:09 PM
natekupp updated the diff for D1359: Add support for using GCS for system storage.

update GCS bucket

Thu, Nov 7, 10:37 PM
natekupp accepted D1378: Revert ecdsa dependency pin.
Thu, Nov 7, 6:01 AM

Wed, Nov 6

natekupp accepted D1373: pin ecdsa (moto dep), which has import resolving issues.

thanks for sleuthing this. sounds reasonable to me

Wed, Nov 6, 9:26 PM
natekupp added a comment to D1370: RFC: chain function for building strings of solids.

TBH I'm kind of partial to the bitshift >> overloading especially since it'll be familiar to users coming from the Airflow world

Wed, Nov 6, 8:58 PM
natekupp accepted D1368: Allow composite solids to have empty config dicts.

LGTM! Thanks for fixing this

Wed, Nov 6, 2:18 AM

Tue, Nov 5

natekupp accepted D1364: Use execute_solid.

ha, I didn't even know execute_isolated_solid existed. this is great, thanks for the cleanup

Tue, Nov 5, 9:47 PM
natekupp updated the summary of D1359: Add support for using GCS for system storage.
Tue, Nov 5, 5:31 AM
natekupp updated the diff for D1359: Add support for using GCS for system storage.

up

Tue, Nov 5, 5:27 AM
natekupp updated the diff for D1359: Add support for using GCS for system storage.

fix one more test

Tue, Nov 5, 5:25 AM
natekupp created D1359: Add support for using GCS for system storage.
Tue, Nov 5, 5:17 AM

Mon, Nov 4

natekupp accepted D1344: bump all buildkite jobs to medium queue.

Can we also remove the micro queue from AWS after this gets deployed? @themissinghlink wanted to learn how to use Pulumi I think, maybe this is a good chance

Mon, Nov 4, 10:57 PM
natekupp planned changes to D1334: [RFC] Add Spark interface.

Will hold on this til 0.7.0

Mon, Nov 4, 9:53 PM
natekupp updated the diff for D1349: pyspark on EMR.

up

Mon, Nov 4, 5:55 PM
natekupp added a comment to D1352: [snowflake] more flexible deps.

see this: https://github.com/dagster-io/dagster/issues/1888 we were hitting issues with 2.0.3

Mon, Nov 4, 4:23 PM

Sun, Nov 3

natekupp updated the summary of D1349: pyspark on EMR.
Sun, Nov 3, 6:39 PM
natekupp updated the diff for D1349: pyspark on EMR.

up

Sun, Nov 3, 5:53 PM
natekupp created D1349: pyspark on EMR.
Sun, Nov 3, 5:51 PM

Sat, Nov 2

natekupp updated the diff for D1348: Add dagster-snowflake dep to examples [full].

up

Sat, Nov 2, 3:31 AM
natekupp updated the diff for D1348: Add dagster-snowflake dep to examples [full].

up

Sat, Nov 2, 3:23 AM
natekupp created D1348: Add dagster-snowflake dep to examples [full].
Sat, Nov 2, 3:13 AM

Fri, Nov 1

natekupp added a reviewer for D1334: [RFC] Add Spark interface: sashank.
Fri, Nov 1, 5:39 PM

Thu, Oct 31

natekupp accepted D1338: bump buildkite queue for dagstermill.
Thu, Oct 31, 5:44 PM
natekupp added a comment to D1336: Remove reexecution_config, add memoization strategy for reexecution.

Amazing. thanks for doing all this cleanup!

Thu, Oct 31, 4:39 PM

Wed, Oct 30

natekupp updated the diff for D1334: [RFC] Add Spark interface.

up

Wed, Oct 30, 12:52 AM
natekupp updated the diff for D1334: [RFC] Add Spark interface.

up

Wed, Oct 30, 12:48 AM
natekupp created D1334: [RFC] Add Spark interface.
Wed, Oct 30, 12:47 AM

Tue, Oct 29

natekupp accepted D1321: [RFC] make implicit OutputDefinition optional.

ah, I really like this, its way simpler than other things we were discussing - @max how do you feel about it?

Tue, Oct 29, 9:46 PM
natekupp abandoned D1292: [spark 1/N] Move Spark logic to a Spark resource.

I'm going to abandon this one in favor of iterating on the more comprehensive RFC diff

Tue, Oct 29, 9:45 PM
natekupp abandoned D1312: [spark 2/N] Clean up EMR job implementation.
Tue, Oct 29, 9:44 PM
natekupp added inline comments to D1276: Add limit to SchedulesRootQuery.
Tue, Oct 29, 9:44 PM
natekupp planned changes to D1330: [RFC] Refactor Spark / EMR Spark.
Tue, Oct 29, 8:00 PM
natekupp added inline comments to D1319: [composition] improve output node misuse errors.
Tue, Oct 29, 5:44 PM
natekupp updated the summary of D1330: [RFC] Refactor Spark / EMR Spark.
Tue, Oct 29, 3:02 AM
natekupp updated the summary of D1330: [RFC] Refactor Spark / EMR Spark.
Tue, Oct 29, 2:59 AM
natekupp updated the summary of D1330: [RFC] Refactor Spark / EMR Spark.
Tue, Oct 29, 2:56 AM
natekupp updated the summary of D1330: [RFC] Refactor Spark / EMR Spark.
Tue, Oct 29, 2:55 AM
natekupp updated the diff for D1330: [RFC] Refactor Spark / EMR Spark.

up

Tue, Oct 29, 2:47 AM
natekupp created D1330: [RFC] Refactor Spark / EMR Spark.
Tue, Oct 29, 2:44 AM

Mon, Oct 28

natekupp abandoned D1270: Rename execution_epoch_time.

I'm going to drop this—after discussing with @prha given the run config cleanup he's doing, I would prefer to rip all of this out entirely.

Mon, Oct 28, 5:23 PM
natekupp updated subscribers of D1329: thread pipeline_run through context.

pretty heroic refactor :) nice! let's sync on the config mapping context tomorrow.

Mon, Oct 28, 5:14 AM

Fri, Oct 25

natekupp accepted D1324: Loosen future dependency in outside of dagster-airflow.
Fri, Oct 25, 2:54 PM

Wed, Oct 23

natekupp accepted D1311: Add example of datetime paritioned schedule.
Wed, Oct 23, 6:37 PM

Tue, Oct 22

natekupp updated the diff for D1312: [spark 2/N] Clean up EMR job implementation.

up

Tue, Oct 22, 12:27 AM
natekupp created D1312: [spark 2/N] Clean up EMR job implementation.
Tue, Oct 22, 12:01 AM

Mon, Oct 21

natekupp retitled D1292: [spark 1/N] Move Spark logic to a Spark resource from Move Spark logic to a Spark resource to [spark 1/N] Move Spark logic to a Spark resource.
Mon, Oct 21, 11:30 PM
natekupp accepted D1311: Add example of datetime paritioned schedule.

love the crypto data example :) see comments

Mon, Oct 21, 10:35 PM
natekupp updated the diff for D1309: Fix bug in Snowflake solid implementation.

up

Mon, Oct 21, 6:10 PM
natekupp updated the diff for D1309: Fix bug in Snowflake solid implementation.

up

Mon, Oct 21, 5:28 PM
natekupp added a reviewer for D1309: Fix bug in Snowflake solid implementation: alangenfeld.
Mon, Oct 21, 4:36 PM
natekupp created D1309: Fix bug in Snowflake solid implementation.
Mon, Oct 21, 4:33 PM

Sun, Oct 20

natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 6:03 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 5:55 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 5:52 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 5:50 AM
natekupp updated the summary of D1292: [spark 1/N] Move Spark logic to a Spark resource.
Sun, Oct 20, 5:47 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 5:42 AM
natekupp retitled D1292: [spark 1/N] Move Spark logic to a Spark resource from [spark 1/N] Move Spark logic to a Spark resource to Move Spark logic to a Spark resource.
Sun, Oct 20, 5:40 AM
natekupp retitled D1292: [spark 1/N] Move Spark logic to a Spark resource from Move Spark logic to a Spark resource to [spark 1/N] Move Spark logic to a Spark resource.
Sun, Oct 20, 5:16 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 5:06 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 4:41 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 4:27 AM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Sun, Oct 20, 4:26 AM

Thu, Oct 17

natekupp updated the summary of D1299: [dagster-snowflake] update snowflake-connector-python version for asn1crypto dep.
Thu, Oct 17, 8:29 PM
natekupp created D1299: [dagster-snowflake] update snowflake-connector-python version for asn1crypto dep.
Thu, Oct 17, 8:19 PM
natekupp added inline comments to D1294: RFC: skip pipeline steps during retry for which we have intermediates.
Thu, Oct 17, 5:42 PM

Wed, Oct 16

natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Wed, Oct 16, 12:01 AM

Tue, Oct 15

natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Tue, Oct 15, 11:45 PM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Tue, Oct 15, 10:04 PM
natekupp updated the diff for D1292: [spark 1/N] Move Spark logic to a Spark resource.

up

Tue, Oct 15, 9:51 PM
natekupp created D1292: [spark 1/N] Move Spark logic to a Spark resource.
Tue, Oct 15, 9:47 PM
natekupp accepted D1288: Used the wrong version of principle (principal) in testing guide.
Tue, Oct 15, 3:15 PM
natekupp accepted D1289: Loosen enum-compat to >=0.0.1.
Tue, Oct 15, 2:40 PM
natekupp updated the diff for D1286: [dagster-aws] Increase timeout on dagster-aws info CLI command.

up

Tue, Oct 15, 3:15 AM
natekupp retitled D1286: [dagster-aws] Increase timeout on dagster-aws info CLI command from [dagster-aws] Increase timeout on dagster-aws info CLI command to 500ms to [dagster-aws] Increase timeout on dagster-aws info CLI command.
Tue, Oct 15, 3:15 AM
natekupp created D1286: [dagster-aws] Increase timeout on dagster-aws info CLI command.
Tue, Oct 15, 3:10 AM

Oct 14 2019

natekupp accepted D1280: fix airline demo docs.
Oct 14 2019, 3:55 PM