Page MenuHomePhabricator

max (Max Gasner)
User

Projects

User does not belong to any projects.

User Details

User Since
Apr 25 2019, 5:20 PM (49 w, 5 d)

Recent Activity

Today

max updated the diff for D2470: GraphQL query for run groupings.

update schema

Wed, Apr 8, 4:42 AM

Yesterday

max accepted D2468: Document use of bash solids (#2345).
Tue, Apr 7, 11:21 PM
max added a comment to D2468: Document use of bash solids (#2345).

seems legit

Tue, Apr 7, 11:16 PM
max added a reviewer for D2470: GraphQL query for run groupings: bengotow.
Tue, Apr 7, 11:13 PM
max created D2470: GraphQL query for run groupings.
Tue, Apr 7, 11:13 PM
max accepted D2447: Thread is_optional into PandasColumn compositions.

This seems fine, I wish we had docstrings on these classes

Tue, Apr 7, 9:49 PM
max accepted D2425: Add pandas version agnostic type checking constraint..

I don't want to block this on docstrings, but I really wish we had docstrings.

Tue, Apr 7, 9:49 PM
max accepted D2457: Make "make dev_install" install deps required by pytest at root.

i wish we had a nice way of keeping this in sync somehow

Tue, Apr 7, 9:47 PM
max accepted D2399: pull out step_key into top-level event_log schema.
Tue, Apr 7, 9:40 PM
max added a comment to D2399: pull out step_key into top-level event_log schema.

i'd prefer we not check in the -shm and -wal files and i would like to see some docs or a long changelog entry explaining to users how to migrate, what the migration will do, and what to do if they do not want to / cannot migrate.

Tue, Apr 7, 9:39 PM
max added a comment to D2405: #2346-1 [RFC] multi-step (arbitrary subset) re-execution through dagit Run.

@schrockn it might make sense for us to do this work against a feature branch rather than master

Tue, Apr 7, 6:47 PM
max added inline comments to D2405: #2346-1 [RFC] multi-step (arbitrary subset) re-execution through dagit Run.
Tue, Apr 7, 6:46 PM

Mon, Apr 6

max added inline comments to D2444: Add query to get runs grouped by root run id.
Mon, Apr 6, 10:35 PM
max created D2454: Fix typo.
Mon, Apr 6, 6:54 PM
max added inline comments to D2444: Add query to get runs grouped by root run id.
Mon, Apr 6, 5:37 PM
max updated the diff for D2444: Add query to get runs grouped by root run id.

Clarify names

Mon, Apr 6, 5:32 PM
max updated the diff for D2444: Add query to get runs grouped by root run id.

blk

Mon, Apr 6, 4:34 AM
max added inline comments to D2444: Add query to get runs grouped by root run id.
Mon, Apr 6, 4:17 AM
max accepted D2443: Add PresetDefinition.from_pkg_resources.
Mon, Apr 6, 3:39 AM
max updated the diff for D2444: Add query to get runs grouped by root run id.

Up

Mon, Apr 6, 3:36 AM

Sun, Apr 5

max added inline comments to D2444: Add query to get runs grouped by root run id.
Sun, Apr 5, 11:57 PM
max added inline comments to D2425: Add pandas version agnostic type checking constraint..
Sun, Apr 5, 4:14 AM
max requested changes to D2443: Add PresetDefinition.from_pkg_resources.

i think there are some transpositions in the docstrings

Sun, Apr 5, 4:11 AM
max accepted D2445: Assert run order in run storage tests.
Sun, Apr 5, 4:05 AM
max added a comment to D2399: pull out step_key into top-level event_log schema.

@prha let's discuss this live on monday

Sun, Apr 5, 4:05 AM
max updated the diff for D2444: Add query to get runs grouped by root run id.

Postgres

Sun, Apr 5, 3:54 AM
max created D2444: Add query to get runs grouped by root run id.
Sun, Apr 5, 1:21 AM

Sat, Apr 4

max committed R1:22b43fac9cc5: Factor out shared query logic (authored by max).
Factor out shared query logic
Sat, Apr 4, 2:46 AM
max closed D2440: Factor out shared query logic.
Sat, Apr 4, 2:46 AM
max added a comment to D2399: pull out step_key into top-level event_log schema.

hmm, this makes me so uncomfortable

Sat, Apr 4, 12:36 AM
max created D2440: Factor out shared query logic.
Sat, Apr 4, 12:02 AM

Fri, Apr 3

max committed R1:bae8a1f5ae09: Add the ability to specify that an executed notebook be yielded as an output… (authored by max).
Add the ability to specify that an executed notebook be yielded as an output…
Fri, Apr 3, 8:20 PM
max closed D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.
Fri, Apr 3, 8:20 PM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

Expand test

Fri, Apr 3, 7:32 PM
max added inline comments to D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.
Fri, Apr 3, 7:23 PM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

Improve test

Fri, Apr 3, 7:11 PM
max added a comment to D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

I don't think we do actually delete the local file manager directories after the run though -- the lifecycle stuff on local_file_manager only deals with the TempfileManager, which i think was an abortive idea about providing tempfile facilities on the context

Fri, Apr 3, 6:44 PM
max added inline comments to D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.
Fri, Apr 3, 6:40 PM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

up

Fri, Apr 3, 7:34 AM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

blk

Fri, Apr 3, 7:17 AM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

Fixup

Fri, Apr 3, 7:14 AM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

py2

Fri, Apr 3, 7:07 AM
max updated the diff for D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.

Fixup

Fri, Apr 3, 5:49 AM
max added a comment to D2334: [dagster-aws] EMR pyspark deploy modes.

Is there a user we can run this by for feedback?

Fri, Apr 3, 5:47 AM
max created D2431: Add the ability to specify that an executed notebook be yielded as an output from a dagstermill solid.
Fri, Apr 3, 5:00 AM
max added a comment to D2399: pull out step_key into top-level event_log schema.

We should at least include an upgrade path or descriptions of what to do to upgrade. I had to blow away my whole schedule storage (without any instructions) this week because of some change and from my perspective as a user it did nothing but eat up an afternoon. I agree that in large production deployments there are scale issues but I don't like the world where scale issues / whatever are available as excuses for us not to ship migrations.

Fri, Apr 3, 12:48 AM
max added inline comments to D2425: Add pandas version agnostic type checking constraint..
Fri, Apr 3, 12:45 AM

Thu, Apr 2

max added inline comments to D2426: Changelog for 0.7.6.
Thu, Apr 2, 11:58 PM
max added a comment to D2424: add utility script to clean resolved phabricator git tags.

I defer to @alangenfeld but would almost rather have a script that deleted and recreated this tag each time so that it bumped to the top: https://github.com/dagster-io/dagster/releases/tag/__placeholder

Thu, Apr 2, 11:27 PM
max closed D2422: Correctly escape quote characters in config over the dagstermill bridge.
Thu, Apr 2, 11:21 PM
max committed R1:241b2081a82c: Correctly escape quote characters in config over the dagstermill bridge (authored by max).
Correctly escape quote characters in config over the dagstermill bridge
Thu, Apr 2, 11:21 PM
max committed R1:714002d91006: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks (authored by max).
Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks
Thu, Apr 2, 11:21 PM
max closed D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks.
Thu, Apr 2, 11:21 PM
max updated the diff for D2422: Correctly escape quote characters in config over the dagstermill bridge.

Fixup

Thu, Apr 2, 10:53 PM
max added inline comments to D2420: Made docker requirement optional and added import warnings.
Thu, Apr 2, 10:34 PM
max added a comment to D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks.

lol uh oh

Thu, Apr 2, 10:30 PM
max updated the diff for D2422: Correctly escape quote characters in config over the dagstermill bridge.

Blk

Thu, Apr 2, 10:23 PM
max updated the diff for D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks.

Up

Thu, Apr 2, 10:22 PM
max added a comment to D2422: Correctly escape quote characters in config over the dagstermill bridge.

I am fighting the pickle fight elsewhere

Thu, Apr 2, 10:21 PM
max updated the diff for D2422: Correctly escape quote characters in config over the dagstermill bridge.

Rebase

Thu, Apr 2, 10:17 PM
max updated the diff for D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks.

Up

Thu, Apr 2, 10:17 PM
max added a reviewer for D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks: nate.
Thu, Apr 2, 10:11 PM
max created D2422: Correctly escape quote characters in config over the dagstermill bridge.
Thu, Apr 2, 10:08 PM
max created D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks.
Thu, Apr 2, 9:24 PM
max requested changes to D2399: pull out step_key into top-level event_log schema.

I think we should add a data migration

Thu, Apr 2, 5:42 PM
max committed R1:7bb3fe5b5ee3: Allow integer types for float config fields (authored by max).
Allow integer types for float config fields
Thu, Apr 2, 5:41 PM
max closed D2389: Allow integer types for float config fields.
Thu, Apr 2, 5:41 PM

Wed, Apr 1

max updated the diff for D2389: Allow integer types for float config fields.

Rebase

Wed, Apr 1, 11:54 PM
max accepted D2401: Update default celery queue in k8s.
Wed, Apr 1, 7:15 PM
max added a comment to D2357: RFC: Add execute_dagstermill_solid utility.

I don't understand what you are proposing here. If we dynamically construct a pipeline, as we do for execute_solid, there is no way to pass it across the serialization boundary. It might be helpful to consider:

Wed, Apr 1, 6:52 PM
max added a comment to D2401: Update default celery queue in k8s.

Does this cause a change in behavior?

Wed, Apr 1, 6:37 PM
max resigned from D2377: add pipeline run tags from dagster CLI.

LGTM, resigning to Sashank.

Wed, Apr 1, 6:37 PM
max committed R1:6ad432cd1692: Schedule docs (authored by max).
Schedule docs
Wed, Apr 1, 6:33 PM
max closed D2397: Schedule docs.
Wed, Apr 1, 6:33 PM
max updated the diff for D2397: Schedule docs.

Feedback

Wed, Apr 1, 6:00 PM

Tue, Mar 31

max updated the diff for D2397: Schedule docs.

blk

Tue, Mar 31, 11:19 PM
max accepted D2398: Fix docs issue.
Tue, Mar 31, 11:18 PM
max updated the diff for D2397: Schedule docs.

Up

Tue, Mar 31, 11:08 PM
max updated the diff for D2397: Schedule docs.

Up

Tue, Mar 31, 11:06 PM
max created D2397: Schedule docs.
Tue, Mar 31, 11:01 PM
max updated the diff for D2389: Allow integer types for float config fields.

blk

Tue, Mar 31, 6:05 PM
max added a comment to D2389: Allow integer types for float config fields.

this should be good to merge after D2388 and D2390

Tue, Mar 31, 5:57 PM
max updated the diff for D2389: Allow integer types for float config fields.

Nits

Tue, Mar 31, 5:56 PM
max updated the diff for D2389: Allow integer types for float config fields.

Blk and remove extraneous check

Tue, Mar 31, 5:51 PM
max accepted D2390: Remove is_config_scalar_valid implementations.

This seems fine to me too

Tue, Mar 31, 5:50 PM
max created D2389: Allow integer types for float config fields.
Tue, Mar 31, 5:44 PM

Mon, Mar 30

max closed D2351: Exit gracefully with informative message when no arguments are passed.
Mon, Mar 30, 9:26 PM
max committed R1:c1eba8eceee4: Exit gracefully with informative message when no arguments are passed (authored by max).
Exit gracefully with informative message when no arguments are passed
Mon, Mar 30, 9:26 PM
max committed R1:378eea225d52: Don't eagerly update schedule status (authored by max).
Don't eagerly update schedule status
Mon, Mar 30, 9:25 PM
max closed D2352: Don't eagerly update schedule status.
Mon, Mar 30, 9:25 PM
max committed R1:374ddad82448: Unquieten make_dev_install by default (authored by max).
Unquieten make_dev_install by default
Mon, Mar 30, 9:25 PM
max closed D2371: Unquieten make_dev_install by default.
Mon, Mar 30, 9:25 PM
max added a comment to D2357: RFC: Add execute_dagstermill_solid utility.

are you asking why vendor cloudpickle in, or why is cloudpickle or equivalent needed here? dynamically constructed pipelines (not loaded through an ExecutionTargetHandle) can't cross the serialization barrier -- so if you construct a pipeline in a pytest test that uses a fixture or in a fixture, there's no way to pass it. our stack-walking tricks, which would work if the pipeline was constructed at the top level in a test file, are fundamentally a partial implementation of cloudpickle.

Mon, Mar 30, 9:24 PM
max accepted D2369: Build next site.
Mon, Mar 30, 9:14 PM
max created D2371: Unquieten make_dev_install by default.
Mon, Mar 30, 7:39 PM

Sun, Mar 29

max abandoned D2359: Temporarily pin dagster-aws dep in examples.
Sun, Mar 29, 5:55 AM
max accepted D2354: Fix AWS ImportError.

this is fine or we can go with sth like https://dagster.phacility.com/D2359

Sun, Mar 29, 5:36 AM
max created D2359: Temporarily pin dagster-aws dep in examples.
Sun, Mar 29, 4:16 AM
max created D2357: RFC: Add execute_dagstermill_solid utility.
Sun, Mar 29, 2:29 AM
max added a comment to D2339: InputDefinition.default_value.

How do we feel about bundling this in with a crack at https://github.com/dagster-io/dagster/issues/1087

Sun, Mar 29, 1:13 AM