Page MenuHomePhabricator

alangenfeld (Alex Langenfeld)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Mar 20 2019, 8:25 PM (96 w, 4 d)
Roles
Administrator

Recent Activity

Fri, Jan 22

alangenfeld closed D6076: [mypy][core] execution/plan.
Fri, Jan 22, 10:35 PM
alangenfeld committed R1:6b5f4e846913: [mypy][core] execution/plan (authored by alangenfeld).
[mypy][core] execution/plan
Fri, Jan 22, 10:35 PM
alangenfeld added a comment to D6076: [mypy][core] execution/plan.

It would be nice to have a better story for these named tuples. Unfortunately, it might require mucking around with metaclasses to support overriding init if we want to do runtime type checking.

Fri, Jan 22, 10:35 PM
alangenfeld resigned from D6048: [Memoized Dev Example 2/2] Memoized development example.

others have more context on this than I

Fri, Jan 22, 10:34 PM
alangenfeld requested changes to D6095: [10/n]Make StepOutput user-code-free.
Fri, Jan 22, 10:33 PM
alangenfeld requested changes to D6094: [9/n] Make StepInputSource use snapshots.
Fri, Jan 22, 10:27 PM
alangenfeld requested changes to D6093: [8/n] Make StepInput.dagster_type a snapshot instead (no user code).
Fri, Jan 22, 10:15 PM
alangenfeld accepted D6088: [6and7/n] remove solid from ExecutionStep.
Fri, Jan 22, 4:59 PM
alangenfeld accepted D6086: [5/n] Wait to determine step compute_fn until you're actually in step execution.
Fri, Jan 22, 4:46 PM
alangenfeld accepted D6084: [4/n] Remove mode_def from SystemExecutionContext, replace with mode_name.
Fri, Jan 22, 4:40 PM
alangenfeld accepted D6083: [3/n] Include pipeline snapshot and execution plan snapshot on system execution context.

mostly just concerned about leaving stuff around we don't need - lets not add the execution plan snapshot if we dont need it

Fri, Jan 22, 4:23 PM
alangenfeld closed D6106: pin mypy.
Fri, Jan 22, 4:20 PM
alangenfeld committed R1:cea931bad634: pin mypy (authored by alangenfeld).
pin mypy
Fri, Jan 22, 4:20 PM
alangenfeld requested changes to D6082: [mypy][core] mypy typing for definitions/dependency.py.

back to you to parameterize dict/list

Fri, Jan 22, 4:10 PM
alangenfeld requested review of D6106: pin mypy.
Fri, Jan 22, 3:58 PM

Thu, Jan 21

alangenfeld requested review of D6076: [mypy][core] execution/plan.
Thu, Jan 21, 9:19 PM
alangenfeld accepted D6069: Add Helm schema for dagsterDaemon.
Thu, Jan 21, 9:09 PM
alangenfeld accepted D6068: [2/2] Fix Helm schema for env, envConfigMaps, and envSecrets.
Thu, Jan 21, 9:08 PM
alangenfeld accepted D6063: [1/2] Fix Helm templating for env, envConfigMaps, and envSecrets.
Thu, Jan 21, 9:08 PM
alangenfeld closed D6085: get_output_manager -> get_io_manager.
Thu, Jan 21, 7:31 PM
alangenfeld committed R1:3397d9c1eda1: get_output_manager -> get_io_manager (authored by alangenfeld).
get_output_manager -> get_io_manager
Thu, Jan 21, 7:31 PM
alangenfeld requested review of D6085: get_output_manager -> get_io_manager.
Thu, Jan 21, 6:24 PM
alangenfeld accepted D6081: Remove pipeline_def from top-level SystemExecutionContext, only use in step execution contexts.

solids still have access to context.pipeline_def right? just making sure this isnt a breaking change

Thu, Jan 21, 6:05 PM
alangenfeld added inline comments to D6058: [mypy][core] definitions/decorators/composite_solid.py mypy.
Thu, Jan 21, 12:05 AM
alangenfeld accepted D6058: [mypy][core] definitions/decorators/composite_solid.py mypy.

should be good after the ... changes

Thu, Jan 21, 12:04 AM
alangenfeld added a comment to D6033: include system framing message in user errors.

Do you have something in mind for how we could improve the display?

Thu, Jan 21, 12:02 AM

Tue, Jan 19

alangenfeld accepted D6040: Unpin alembic.

redfordnod

Tue, Jan 19, 9:42 PM
alangenfeld added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

first reactions:

  • concerns around hiding errors / confusion for intermediate user getting Any when trying to actually use types
  • throwing away information by dropping all the way to Any
Tue, Jan 19, 8:56 PM
alangenfeld added a comment to D6033: include system framing message in user errors.

what led away from the approach in https://dagster.phacility.com/D5784 ? Just updating test call sites?

Tue, Jan 19, 8:23 PM
alangenfeld added a comment to D6036: Make type_check_fn optional on DagsterType.

I think I'm ok with this - don't feel that strong in either direction. Curious for others thoughts.

Tue, Jan 19, 7:48 PM
alangenfeld accepted D5978: remove define_solid_config_cls, which is only used by tests.

cleanup

Tue, Jan 19, 6:51 PM
alangenfeld planned changes to D2879: RFC: Composite solid subselection.
Tue, Jan 19, 6:34 PM
alangenfeld commandeered D2879: RFC: Composite solid subselection.
Tue, Jan 19, 6:34 PM
alangenfeld edited reviewers for D6035: Remove dev-requirements.txt files, added: max; removed: alangenfeld.
Tue, Jan 19, 4:44 PM

Fri, Jan 15

alangenfeld closed D6032: [mypy][core] execution/context.
Fri, Jan 15, 9:10 PM
alangenfeld committed R1:4dc65088b03c: [mypy][core] execution/context (authored by alangenfeld).
[mypy][core] execution/context
Fri, Jan 15, 9:10 PM
alangenfeld updated the diff for D6032: [mypy][core] execution/context.

feedbcak

Fri, Jan 15, 7:44 PM
alangenfeld added inline comments to D6032: [mypy][core] execution/context.
Fri, Jan 15, 7:44 PM
alangenfeld closed D6020: [mypy][core] execution api.py and active.py.
Fri, Jan 15, 7:16 PM
alangenfeld committed R1:9184970fc2e6: [mypy][core] execution api.py and active.py (authored by alangenfeld).
[mypy][core] execution api.py and active.py
Fri, Jan 15, 7:16 PM
alangenfeld planned changes to D6032: [mypy][core] execution/context.
Fri, Jan 15, 7:09 PM
alangenfeld added a comment to D6032: [mypy][core] execution/context.

nice thanks for the close review, will address inlines

Fri, Jan 15, 7:08 PM
alangenfeld requested review of D6032: [mypy][core] execution/context.
Fri, Jan 15, 5:34 PM
alangenfeld requested review of D6020: [mypy][core] execution api.py and active.py.
Fri, Jan 15, 4:29 PM
alangenfeld closed D6005: redo yarn.lock for /docs.
Fri, Jan 15, 3:34 PM
alangenfeld committed R1:a01b3335171f: redo yarn.lock for /docs (authored by alangenfeld).
redo yarn.lock for /docs
Fri, Jan 15, 3:34 PM
alangenfeld requested review of D6017: initial async solids support.
Fri, Jan 15, 12:15 AM

Thu, Jan 14

alangenfeld requested review of D6005: redo yarn.lock for /docs.
Thu, Jan 14, 8:08 PM
alangenfeld closed D6003: [docs] dynamic graphs.
Thu, Jan 14, 8:05 PM
alangenfeld committed R1:2fe99ab5d371: [docs] dynamic graphs (authored by alangenfeld).
[docs] dynamic graphs
Thu, Jan 14, 8:04 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

drop the for

Thu, Jan 14, 7:57 PM
alangenfeld updated the test plan for D6003: [docs] dynamic graphs.
Thu, Jan 14, 7:57 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

now tested

Thu, Jan 14, 7:56 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

side bar entry

Thu, Jan 14, 7:45 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

doc blocks

Thu, Jan 14, 7:43 PM
alangenfeld requested review of D6003: [docs] dynamic graphs.
Thu, Jan 14, 7:27 PM
alangenfeld accepted D6002: mark input and output metadata experimental.

ponyta

Thu, Jan 14, 6:49 PM
alangenfeld published D6002: mark input and output metadata experimental for review.
Thu, Jan 14, 6:49 PM
alangenfeld accepted D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

does the watch thread need to get turned back on?

Thu, Jan 14, 6:07 PM
alangenfeld requested changes to D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

to your queue for mentioned test and additional comments

Thu, Jan 14, 5:28 PM
alangenfeld added a comment to D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

nice

Thu, Jan 14, 5:27 PM
alangenfeld closed D5996: only connect on init when needed.
Thu, Jan 14, 4:25 PM
alangenfeld committed R1:b14a5d08d927: only connect on init when needed (authored by alangenfeld).
only connect on init when needed
Thu, Jan 14, 4:25 PM
alangenfeld requested review of D5996: only connect on init when needed.
Thu, Jan 14, 4:20 PM
alangenfeld requested changes to D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

zoom debugging / review

Thu, Jan 14, 3:54 PM
alangenfeld accepted D5993: Make instance migration error clearer.

victreebel

Thu, Jan 14, 3:30 PM

Wed, Jan 13

alangenfeld updated the diff for D5126: RFC use custom yaml loader.

rebase

Wed, Jan 13, 10:39 PM
alangenfeld requested review of D5126: RFC use custom yaml loader.
Wed, Jan 13, 10:36 PM
alangenfeld added a comment to D5126: RFC use custom yaml loader.

the current status is having one of these date yaml entries in a preset file like

Wed, Jan 13, 10:32 PM
alangenfeld closed D5961: have DagsterExecutionInterruptedError inherit from BaseException.
Wed, Jan 13, 10:06 PM
alangenfeld committed R1:21faf5c9b7a4: have DagsterExecutionInterruptedError inherit from BaseException (authored by alangenfeld).
have DagsterExecutionInterruptedError inherit from BaseException
Wed, Jan 13, 10:06 PM
alangenfeld closed D5632: DynamicOutput re-execution.
Wed, Jan 13, 8:59 PM
alangenfeld committed R1:24f6b309277e: DynamicOutput re-execution (authored by alangenfeld).
DynamicOutput re-execution
Wed, Jan 13, 8:59 PM
alangenfeld closed D5721: DynamicOutput DSL map approach.
Wed, Jan 13, 8:58 PM
alangenfeld committed R1:09d68569fe4d: DynamicOutput DSL map approach (authored by alangenfeld).
DynamicOutput DSL map approach
Wed, Jan 13, 8:58 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase

Wed, Jan 13, 8:39 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

rebase

Wed, Jan 13, 8:38 PM
alangenfeld added inline comments to D5721: DynamicOutput DSL map approach.
Wed, Jan 13, 8:18 PM
alangenfeld closed D5960: close input load user boundary before return.
Wed, Jan 13, 8:12 PM
alangenfeld committed R1:3a81a2c6b350: close input load user boundary before return (authored by alangenfeld).
close input load user boundary before return
Wed, Jan 13, 8:11 PM
alangenfeld closed D5955: 0.10.0 API removals.
Wed, Jan 13, 8:11 PM
alangenfeld committed R1:63b28851daf6: 0.10.0 API removals (authored by alangenfeld).
0.10.0 API removals
Wed, Jan 13, 8:11 PM
alangenfeld requested review of D5961: have DagsterExecutionInterruptedError inherit from BaseException.
Wed, Jan 13, 7:40 PM
alangenfeld requested review of D5955: 0.10.0 API removals.
Wed, Jan 13, 7:37 PM
alangenfeld requested review of D5960: close input load user boundary before return.

^ flake

Wed, Jan 13, 7:23 PM
alangenfeld accepted D5956: Move not_graphql_context_test_suite to root tox file.
Wed, Jan 13, 7:21 PM
alangenfeld added a reviewer for D5721: DynamicOutput DSL map approach: prha.

+ @prha for lint shenanigans

Wed, Jan 13, 6:56 PM
alangenfeld accepted D5958: Fix build failure from six cleanup.
Wed, Jan 13, 6:43 PM
alangenfeld added a comment to D5941: RFC: Make default sharded sqlite implementation asset-aware.

Mechanically this is a pretty clever way to get this feature working in the sharded sqlite scenario but the implementation is some real "inheritance is the root of all evil" shit. I think it makes it clear that the way the run sharding was managed between the child/parent class is pretty fragile foundation.

Wed, Jan 13, 5:14 PM
alangenfeld accepted D5944: Remove usage of six.
Wed, Jan 13, 4:59 PM

Tue, Jan 12

alangenfeld accepted D5938: Add insturctions to run dagster instance migrate to the MIGRATION.md, plus instructions for how to turn on the new scheduler.

wartortle

Tue, Jan 12, 11:11 PM
alangenfeld added a reviewer for D5926: Raise a migration required exception when you load a pre-0.10.0 schedule storage: max.
Tue, Jan 12, 11:09 PM
alangenfeld added a comment to D5926: Raise a migration required exception when you load a pre-0.10.0 schedule storage.

I do think it would be good to start stamping postgres alembic revisions, just so we have the flexibility to know for sure what revision a given user is on

Tue, Jan 12, 11:09 PM
alangenfeld added inline comments to D5829: Modernize dagstermill file handling.
Tue, Jan 12, 11:05 PM
alangenfeld added a comment to D5916: create run partition data migration, tie to `dagster instance migrate`.

i dont know, I think building missing indexes on migration seems reasonable, the tracking of indexes in each storage also makes sense. I think dropping all this stuff will have to come with some big change to the instance set up

Tue, Jan 12, 10:26 PM
alangenfeld accepted D5925: InputManager -> RootInputManager.

pidgeot

Tue, Jan 12, 9:55 PM
alangenfeld accepted D5916: create run partition data migration, tie to `dagster instance migrate`.

i don't love this complexity burden but i think its the right call for end user experience

Tue, Jan 12, 9:54 PM
alangenfeld accepted D5924: unexpose output managers.

cloyster

Tue, Jan 12, 9:35 PM
alangenfeld requested changes to D5780: eliminate Persistable.

lets go for the consolidation while we're making changes, should just be easy enough

Tue, Jan 12, 9:34 PM
alangenfeld accepted D5829: Modernize dagstermill file handling.

some complex stuff but i think the comments and cleanup make it parseable at least, thanks for that.

Tue, Jan 12, 9:31 PM