Page MenuHomePhabricator

alangenfeld (Alex Langenfeld)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Mar 20 2019, 8:25 PM (54 w, 6 d)
Roles
Administrator

Recent Activity

Sat, Apr 4

alangenfeld updated the diff for D2439: [celery] post process config to handle env.

add test

Sat, Apr 4, 12:34 AM

Fri, Apr 3

alangenfeld added inline comments to D2425: Add pandas version agnostic type checking constraint..
Fri, Apr 3, 11:24 PM
alangenfeld created D2439: [celery] post process config to handle env.
Fri, Apr 3, 11:16 PM
alangenfeld added inline comments to D2425: Add pandas version agnostic type checking constraint..
Fri, Apr 3, 10:38 PM
alangenfeld added inline comments to D2425: Add pandas version agnostic type checking constraint..
Fri, Apr 3, 7:16 PM
alangenfeld closed D2433: [nightly] rm tag cleanup.
Fri, Apr 3, 7:13 PM
alangenfeld committed R1:3cb58ab33701: [nightly] rm tag cleanup (authored by alangenfeld).
[nightly] rm tag cleanup
Fri, Apr 3, 7:13 PM
alangenfeld requested changes to D2424: add utility script to clean resolved phabricator git tags.

alright the staging area change works - this isn't needed

Fri, Apr 3, 6:56 PM
alangenfeld created D2433: [nightly] rm tag cleanup.
Fri, Apr 3, 6:56 PM
alangenfeld accepted D2408: Minor refactor of execute.py.

verygood

Fri, Apr 3, 3:59 PM
alangenfeld added inline comments to D2425: Add pandas version agnostic type checking constraint..
Fri, Apr 3, 3:43 PM

Thu, Apr 2

alangenfeld added inline comments to D2426: Changelog for 0.7.6.
Thu, Apr 2, 11:59 PM
alangenfeld accepted D2426: Changelog for 0.7.6.
Thu, Apr 2, 11:56 PM
alangenfeld added inline comments to D2425: Add pandas version agnostic type checking constraint..
Thu, Apr 2, 11:51 PM
alangenfeld added a comment to D2425: Add pandas version agnostic type checking constraint..

lets take some time to think this one through since it appears doing it in a backwards compat safe and not confusing way may be tricky

Thu, Apr 2, 11:41 PM
alangenfeld accepted D2420: Made docker requirement optional and added import warnings.

mankey

Thu, Apr 2, 11:16 PM
alangenfeld added a comment to D2424: add utility script to clean resolved phabricator git tags.

Buildkite also to run bin/clean.py on every merge to master?

Thu, Apr 2, 11:00 PM
alangenfeld added a comment to D2368: Replace the playground's execution plan preview with a more useful config map.

them empty errors view is a little janky too but not blocking

Thu, Apr 2, 10:55 PM
alangenfeld added inline comments to D2420: Made docker requirement optional and added import warnings.
Thu, Apr 2, 10:47 PM
alangenfeld accepted D2421: Allow users to specify Jupyter kernels when scaffolding dagstermill notebooks.

in max we trust

Thu, Apr 2, 10:24 PM
alangenfeld requested changes to D2420: Made docker requirement optional and added import warnings.

TEST PLAN
na

Thu, Apr 2, 10:19 PM
alangenfeld requested changes to D2408: Minor refactor of execute.py.
Thu, Apr 2, 10:17 PM
alangenfeld committed R1:fc7da90ad671: [celery] mention url encoding passwords (authored by alangenfeld).
[celery] mention url encoding passwords
Thu, Apr 2, 9:44 PM
alangenfeld closed D2419: [celery] mention url encoding passwords.
Thu, Apr 2, 9:44 PM
alangenfeld added inline comments to D2408: Minor refactor of execute.py.
Thu, Apr 2, 9:22 PM
alangenfeld created D2419: [celery] mention url encoding passwords.
Thu, Apr 2, 8:52 PM
alangenfeld added a reviewer for D2408: Minor refactor of execute.py: schrockn.
Thu, Apr 2, 8:31 PM
alangenfeld added a comment to D2357: RFC: Add execute_dagstermill_solid utility.

I would much rather see an appropriate where the module/fn name of the function that defines the solid definition to be put under test is serialized with an appropriate pipeline, repository, etc reconstructed on the other sid, and the intermediates store is used for the input values.

Thu, Apr 2, 6:00 PM
alangenfeld committed R1:fb3d6700adcc: add Nothing input to dbt solid (authored by alangenfeld).
add Nothing input to dbt solid
Thu, Apr 2, 5:20 PM
alangenfeld closed D2415: add Nothing input to dbt solid.
Thu, Apr 2, 5:20 PM
alangenfeld accepted D2411: (refactor-validation-context-2) Change from post_process bool to enum.

soclean

Thu, Apr 2, 4:57 PM
alangenfeld created Image Macro "soclean".
Thu, Apr 2, 4:57 PM
alangenfeld created D2415: add Nothing input to dbt solid.
Thu, Apr 2, 4:44 PM
alangenfeld committed R1:85b8ef5813cf: InputDefinition.default_value (authored by alangenfeld).
InputDefinition.default_value
Thu, Apr 2, 12:06 AM
alangenfeld closed D2339: InputDefinition.default_value.
Thu, Apr 2, 12:06 AM

Wed, Apr 1

alangenfeld accepted D2410: (refactor-validation-context-1) Rename ValidationContext to TraversalContext.

everything old is new again https://dagster.phacility.com/D1671 time is a flat circle

Wed, Apr 1, 11:59 PM
alangenfeld updated the summary of D2339: InputDefinition.default_value.
Wed, Apr 1, 11:57 PM
alangenfeld added a comment to D2399: pull out step_key into top-level event_log schema.

this stuff still scares me but i feel like developing trust in these migrations is pretty key for us

Wed, Apr 1, 11:56 PM
alangenfeld added a reviewer for D2399: pull out step_key into top-level event_log schema: max.

oh shit migrations - want @max in here for that

Wed, Apr 1, 11:54 PM
alangenfeld accepted D2396: (stable-pipeline-snapshot-2) Add snapshot snapshot tests and create_pipeline_snapshot_id.

verygood

Wed, Apr 1, 11:52 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

4th?

Wed, Apr 1, 11:50 PM
alangenfeld accepted D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

rename + enum would be nice but isn't blocking - file away a starter issue if you punt on it

Wed, Apr 1, 11:47 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

3rd times the charm'

Wed, Apr 1, 10:34 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

handle empty

Wed, Apr 1, 8:46 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

handle empty

Wed, Apr 1, 8:41 PM
alangenfeld requested changes to D2393: (stable-pipeline-snapshot-1) Use json rep default value in config type key.

to you q for question

Wed, Apr 1, 7:42 PM
alangenfeld requested changes to D2392: (2-read-only-containers)-link-to-pipelin-cli-print-command.

this diff currently contains all the commits for the diff below as well

Wed, Apr 1, 7:34 PM
alangenfeld requested changes to D2381: (1-read-only-containers)-container-snapshot-provider-scaffolding.

I think the repository.yaml constraint is reasonable, but maybe we can relax the requirement for the entrypoint being set? I think in our other images we haven't set this, that way the end user can choose to run dagster, dagster-graphql, dagit, etc. on the same image depending on use case

Wed, Apr 1, 7:28 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

composite

Wed, Apr 1, 7:26 PM
alangenfeld requested changes to D2362: Incorporate default value in type key as serialized json.

i think we want to have just the value here since its now guaranteed to be string

Wed, Apr 1, 7:25 PM
alangenfeld accepted D2388: Only allow raw config values as default_value.

rhydon

Wed, Apr 1, 7:24 PM
alangenfeld accepted D2377: add pipeline run tags from dagster CLI.

geodude

Wed, Apr 1, 7:22 PM
alangenfeld planned changes to D2339: InputDefinition.default_value.

woah good call almost whiffed on testing composite mappings which need much more care

Wed, Apr 1, 4:41 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

leverage BuiltinScalarDagsterType for early errors

Wed, Apr 1, 12:24 AM

Tue, Mar 31

alangenfeld retitled D2339: InputDefinition.default_value from [RFC] InputDefinition.default_value to InputDefinition.default_value.
Tue, Mar 31, 11:50 PM
alangenfeld resigned from D2389: Allow integer types for float config fields.

abandon

Tue, Mar 31, 8:12 PM
alangenfeld requested changes to D2388: Only allow raw config values as default_value.
Tue, Mar 31, 8:10 PM
alangenfeld accepted D2387: Add utility is_enum_value.

alrightalrightalright

Tue, Mar 31, 8:06 PM
alangenfeld created Image Macro "alrightalrightalright".
Tue, Mar 31, 8:06 PM
alangenfeld added a reviewer for D2381: (1-read-only-containers)-container-snapshot-provider-scaffolding: nate.
Tue, Mar 31, 6:54 PM
alangenfeld updated subscribers of D2381: (1-read-only-containers)-container-snapshot-provider-scaffolding.

@nate since i was bugging you about this with regards to the k8s stuff the other day - what do we think the rules should be for a "dagster image"

Tue, Mar 31, 6:53 PM
alangenfeld accepted D2378: Adds prettier command from Makefile into package.json.

make sure to test make prettier too - I think that might need to get updated to just yarn run prettier.

Tue, Mar 31, 5:54 PM

Mon, Mar 30

alangenfeld accepted D2358: Add repository snapshot CLI.

illallowit

Mon, Mar 30, 10:10 PM
alangenfeld accepted D2374: Find and Replce Execute Tab with Playground.

shipit

Mon, Mar 30, 10:10 PM
alangenfeld requested changes to D2374: Find and Replce Execute Tab with Playground.

requesting changes to fix lint

Mon, Mar 30, 10:00 PM
alangenfeld added a comment to D2368: Replace the playground's execution plan preview with a more useful config map.

code looks reasonable

Mon, Mar 30, 7:45 PM
alangenfeld accepted D2363: Unset default Dagster image/tag for Helm chart.

accepting assuming good error happens

Mon, Mar 30, 7:41 PM
alangenfeld added a comment to D2363: Unset default Dagster image/tag for Helm chart.

you get a good error when you try to install without these values set?

Mon, Mar 30, 7:39 PM
alangenfeld accepted D2361: (stable-hash-2) Make PipelineSnapshot serializable via serdes.

seems fine to land this as is though - unclear from discussion if there are follow ups but those can be other diff

Mon, Mar 30, 7:39 PM
alangenfeld requested changes to D2358: Add repository snapshot CLI.
Mon, Mar 30, 7:37 PM
alangenfeld accepted D2355: GraphQL PipelineSnapshotMixin.

thumbsup

Mon, Mar 30, 7:32 PM
alangenfeld added a comment to D2355: GraphQL PipelineSnapshotMixin.

seems legit to me

Mon, Mar 30, 7:31 PM
alangenfeld added a comment to D2362: Incorporate default value in type key as serialized json.

definitely not tied to using serdes

Mon, Mar 30, 5:13 PM
alangenfeld added a comment to D2339: InputDefinition.default_value.

Why don't we special case default checks for the scalar types? I think that would snag an entire class of errors.

Mon, Mar 30, 4:23 PM
alangenfeld added a comment to D2362: Incorporate default value in type key as serialized json.

my initial reaction is negative - ill think about it some more

Mon, Mar 30, 4:21 PM
alangenfeld added a comment to D2339: InputDefinition.default_value.

use native default values?

Mon, Mar 30, 3:39 PM
alangenfeld accepted D2366: Helm fix for renamed PG secret.
Mon, Mar 30, 3:37 PM
alangenfeld added a comment to D2357: RFC: Add execute_dagstermill_solid utility.

hmm this isnt too crazy - interested what other folks think

Mon, Mar 30, 3:34 PM
alangenfeld added a comment to D2339: InputDefinition.default_value.

How do we feel about bundling this in with a crack at https://github.com/dagster-io/dagster/issues/1087

Mon, Mar 30, 3:27 PM

Fri, Mar 27

alangenfeld committed R1:5d7262053f6b: [dagster-k8s] get_celery_engine_config (authored by alangenfeld).
[dagster-k8s] get_celery_engine_config
Fri, Mar 27, 8:40 PM
alangenfeld closed D2344: [dagster-k8s] get_celery_engine_config.
Fri, Mar 27, 8:40 PM
alangenfeld committed R1:a88b0e4991ed: PresetDefinition.with_additional_config (authored by alangenfeld).
PresetDefinition.with_additional_config
Fri, Mar 27, 8:40 PM
alangenfeld closed D2347: PresetDefinition.with_additional_config.
Fri, Mar 27, 8:40 PM
alangenfeld created D2347: PresetDefinition.with_additional_config.
Fri, Mar 27, 7:33 PM
alangenfeld updated the diff for D2339: InputDefinition.default_value.

handle more cases add more tests

Fri, Mar 27, 6:46 PM
alangenfeld retitled D2339: InputDefinition.default_value from [RFC] InputDefinition.default_value to InputDefinition.default_value.
Fri, Mar 27, 6:46 PM
alangenfeld added a comment to D2344: [dagster-k8s] get_celery_engine_config.

ya normal PresetDefinition construction can take this in with the rest of a manually constructed dict, the from_files case is a little tougher and might justify a with so i can add that in a separate diff

Fri, Mar 27, 5:54 PM
alangenfeld added a comment to D2342: [dagster-aws] Make dagster_aws imports more sane.

is any of this going to be a breaking change for users?

Fri, Mar 27, 5:40 PM
alangenfeld accepted D2336: #2320 add root_run_id to group related runs.
Fri, Mar 27, 4:47 PM
alangenfeld added inline comments to D2344: [dagster-k8s] get_celery_engine_config.
Fri, Mar 27, 4:45 PM
alangenfeld updated the summary of D2344: [dagster-k8s] get_celery_engine_config.
Fri, Mar 27, 4:44 PM
alangenfeld added inline comments to D2336: #2320 add root_run_id to group related runs.
Fri, Mar 27, 4:32 PM
alangenfeld added a comment to D2334: [dagster-aws] EMR pyspark deploy modes.

builder / deployment stuff looks v reasonable - tricky part is definitely where / how these knobs should be turned since having a lot of this stuff in the environment dict feels spooky - at least to me

Fri, Mar 27, 4:28 PM
alangenfeld accepted D2209: [dagster-k8s] Remove hard-coded resource names.

this works for both the us rolling out pg case and the user supplied one i assume? we put the user provided password in a secret?

Fri, Mar 27, 4:23 PM
alangenfeld accepted D2345: Stop backfilling schedule when backfill is complete.

whynot

Fri, Mar 27, 4:21 PM
alangenfeld updated the diff for D2344: [dagster-k8s] get_celery_engine_config.

up

Fri, Mar 27, 4:05 PM
alangenfeld updated the diff for D2344: [dagster-k8s] get_celery_engine_config.

up

Fri, Mar 27, 12:07 AM

Thu, Mar 26

alangenfeld updated the diff for D2344: [dagster-k8s] get_celery_engine_config.

up

Thu, Mar 26, 11:51 PM
alangenfeld added a comment to D2343: [prototype rfc] User Code Container Entrypoints in Dagit.

ExecutionTargetHandle is such a mess already - im skeptical that we want to do this image handling inside that. Especially since what we are loading out of the image is actually going to be a RepositorySnapshot which has a set of PipelineSnapshots and not a real RepositoryDefinition. I have a hunch putting a layer of indirection on top of ExecutionTargetHandle may be a better path.

Thu, Mar 26, 9:23 PM
alangenfeld updated the diff for D2344: [dagster-k8s] get_celery_engine_config.

up

Thu, Mar 26, 7:34 PM