Page MenuHomePhabricator
Feed Advanced Search

Today

alangenfeld closed D6085: get_output_manager -> get_io_manager.
Thu, Jan 21, 7:31 PM
alangenfeld committed R1:3397d9c1eda1: get_output_manager -> get_io_manager (authored by alangenfeld).
get_output_manager -> get_io_manager
Thu, Jan 21, 7:31 PM
alangenfeld requested review of D6085: get_output_manager -> get_io_manager.
Thu, Jan 21, 6:24 PM
alangenfeld accepted D6081: Remove pipeline_def from top-level SystemExecutionContext, only use in step execution contexts.

solids still have access to context.pipeline_def right? just making sure this isnt a breaking change

Thu, Jan 21, 6:05 PM
alangenfeld added inline comments to D6058: [mypy][core] definitions/decorators/composite_solid.py mypy.
Thu, Jan 21, 12:05 AM
alangenfeld accepted D6058: [mypy][core] definitions/decorators/composite_solid.py mypy.

should be good after the ... changes

Thu, Jan 21, 12:04 AM
alangenfeld added a comment to D6033: include system framing message in user errors.

Do you have something in mind for how we could improve the display?

Thu, Jan 21, 12:02 AM

Tue, Jan 19

alangenfeld accepted D6040: Unpin alembic.

redfordnod

Tue, Jan 19, 9:42 PM
alangenfeld added a comment to D5115: don't error on solid type annotations that don't resolve to dagster types.

first reactions:

  • concerns around hiding errors / confusion for intermediate user getting Any when trying to actually use types
  • throwing away information by dropping all the way to Any
Tue, Jan 19, 8:56 PM
alangenfeld added a comment to D6033: include system framing message in user errors.

what led away from the approach in https://dagster.phacility.com/D5784 ? Just updating test call sites?

Tue, Jan 19, 8:23 PM
alangenfeld added a comment to D6036: Make type_check_fn optional on DagsterType.

I think I'm ok with this - don't feel that strong in either direction. Curious for others thoughts.

Tue, Jan 19, 7:48 PM
alangenfeld accepted D5978: remove define_solid_config_cls, which is only used by tests.

cleanup

Tue, Jan 19, 6:51 PM
alangenfeld planned changes to D2879: RFC: Composite solid subselection.
Tue, Jan 19, 6:34 PM
alangenfeld commandeered D2879: RFC: Composite solid subselection.
Tue, Jan 19, 6:34 PM
alangenfeld edited reviewers for D6035: Remove dev-requirements.txt files, added: max; removed: alangenfeld.
Tue, Jan 19, 4:44 PM

Fri, Jan 15

alangenfeld closed D6032: [mypy][core] execution/context.
Fri, Jan 15, 9:10 PM
alangenfeld committed R1:4dc65088b03c: [mypy][core] execution/context (authored by alangenfeld).
[mypy][core] execution/context
Fri, Jan 15, 9:10 PM
alangenfeld updated the diff for D6032: [mypy][core] execution/context.

feedbcak

Fri, Jan 15, 7:44 PM
alangenfeld added inline comments to D6032: [mypy][core] execution/context.
Fri, Jan 15, 7:44 PM
alangenfeld closed D6020: [mypy][core] execution api.py and active.py.
Fri, Jan 15, 7:16 PM
alangenfeld committed R1:9184970fc2e6: [mypy][core] execution api.py and active.py (authored by alangenfeld).
[mypy][core] execution api.py and active.py
Fri, Jan 15, 7:16 PM
alangenfeld planned changes to D6032: [mypy][core] execution/context.
Fri, Jan 15, 7:09 PM
alangenfeld added a comment to D6032: [mypy][core] execution/context.

nice thanks for the close review, will address inlines

Fri, Jan 15, 7:08 PM
alangenfeld requested review of D6032: [mypy][core] execution/context.
Fri, Jan 15, 5:34 PM
alangenfeld requested review of D6020: [mypy][core] execution api.py and active.py.
Fri, Jan 15, 4:29 PM
alangenfeld closed D6005: redo yarn.lock for /docs.
Fri, Jan 15, 3:34 PM
alangenfeld committed R1:a01b3335171f: redo yarn.lock for /docs (authored by alangenfeld).
redo yarn.lock for /docs
Fri, Jan 15, 3:34 PM
alangenfeld requested review of D6017: initial async solids support.
Fri, Jan 15, 12:15 AM

Thu, Jan 14

alangenfeld requested review of D6005: redo yarn.lock for /docs.
Thu, Jan 14, 8:08 PM
alangenfeld closed D6003: [docs] dynamic graphs.
Thu, Jan 14, 8:05 PM
alangenfeld committed R1:2fe99ab5d371: [docs] dynamic graphs (authored by alangenfeld).
[docs] dynamic graphs
Thu, Jan 14, 8:04 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

drop the for

Thu, Jan 14, 7:57 PM
alangenfeld updated the test plan for D6003: [docs] dynamic graphs.
Thu, Jan 14, 7:57 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

now tested

Thu, Jan 14, 7:56 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

side bar entry

Thu, Jan 14, 7:45 PM
alangenfeld updated the diff for D6003: [docs] dynamic graphs.

doc blocks

Thu, Jan 14, 7:43 PM
alangenfeld requested review of D6003: [docs] dynamic graphs.
Thu, Jan 14, 7:27 PM
alangenfeld accepted D6002: mark input and output metadata experimental.

ponyta

Thu, Jan 14, 6:49 PM
alangenfeld published D6002: mark input and output metadata experimental for review.
Thu, Jan 14, 6:49 PM
alangenfeld accepted D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

does the watch thread need to get turned back on?

Thu, Jan 14, 6:07 PM
alangenfeld requested changes to D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

to your queue for mentioned test and additional comments

Thu, Jan 14, 5:28 PM
alangenfeld added a comment to D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

nice

Thu, Jan 14, 5:27 PM
alangenfeld closed D5996: only connect on init when needed.
Thu, Jan 14, 4:25 PM
alangenfeld committed R1:b14a5d08d927: only connect on init when needed (authored by alangenfeld).
only connect on init when needed
Thu, Jan 14, 4:25 PM
alangenfeld requested review of D5996: only connect on init when needed.
Thu, Jan 14, 4:20 PM
alangenfeld requested changes to D5953: Split DagterGraphQLContext into ProcessContext and RequestContext.

zoom debugging / review

Thu, Jan 14, 3:54 PM
alangenfeld accepted D5993: Make instance migration error clearer.

victreebel

Thu, Jan 14, 3:30 PM

Wed, Jan 13

alangenfeld updated the diff for D5126: RFC use custom yaml loader.

rebase

Wed, Jan 13, 10:39 PM
alangenfeld requested review of D5126: RFC use custom yaml loader.
Wed, Jan 13, 10:36 PM
alangenfeld added a comment to D5126: RFC use custom yaml loader.

the current status is having one of these date yaml entries in a preset file like

Wed, Jan 13, 10:32 PM
alangenfeld closed D5961: have DagsterExecutionInterruptedError inherit from BaseException.
Wed, Jan 13, 10:06 PM
alangenfeld committed R1:21faf5c9b7a4: have DagsterExecutionInterruptedError inherit from BaseException (authored by alangenfeld).
have DagsterExecutionInterruptedError inherit from BaseException
Wed, Jan 13, 10:06 PM
alangenfeld closed D5632: DynamicOutput re-execution.
Wed, Jan 13, 8:59 PM
alangenfeld committed R1:24f6b309277e: DynamicOutput re-execution (authored by alangenfeld).
DynamicOutput re-execution
Wed, Jan 13, 8:59 PM
alangenfeld closed D5721: DynamicOutput DSL map approach.
Wed, Jan 13, 8:58 PM
alangenfeld committed R1:09d68569fe4d: DynamicOutput DSL map approach (authored by alangenfeld).
DynamicOutput DSL map approach
Wed, Jan 13, 8:58 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase

Wed, Jan 13, 8:39 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

rebase

Wed, Jan 13, 8:38 PM
alangenfeld added inline comments to D5721: DynamicOutput DSL map approach.
Wed, Jan 13, 8:18 PM
alangenfeld closed D5960: close input load user boundary before return.
Wed, Jan 13, 8:12 PM
alangenfeld committed R1:3a81a2c6b350: close input load user boundary before return (authored by alangenfeld).
close input load user boundary before return
Wed, Jan 13, 8:11 PM
alangenfeld closed D5955: 0.10.0 API removals.
Wed, Jan 13, 8:11 PM
alangenfeld committed R1:63b28851daf6: 0.10.0 API removals (authored by alangenfeld).
0.10.0 API removals
Wed, Jan 13, 8:11 PM
alangenfeld requested review of D5961: have DagsterExecutionInterruptedError inherit from BaseException.
Wed, Jan 13, 7:40 PM
alangenfeld requested review of D5955: 0.10.0 API removals.
Wed, Jan 13, 7:37 PM
alangenfeld requested review of D5960: close input load user boundary before return.

^ flake

Wed, Jan 13, 7:23 PM
alangenfeld accepted D5956: Move not_graphql_context_test_suite to root tox file.
Wed, Jan 13, 7:21 PM
alangenfeld added a reviewer for D5721: DynamicOutput DSL map approach: prha.

+ @prha for lint shenanigans

Wed, Jan 13, 6:56 PM
alangenfeld accepted D5958: Fix build failure from six cleanup.
Wed, Jan 13, 6:43 PM
alangenfeld added a comment to D5941: RFC: Make default sharded sqlite implementation asset-aware.

Mechanically this is a pretty clever way to get this feature working in the sharded sqlite scenario but the implementation is some real "inheritance is the root of all evil" shit. I think it makes it clear that the way the run sharding was managed between the child/parent class is pretty fragile foundation.

Wed, Jan 13, 5:14 PM
alangenfeld accepted D5944: Remove usage of six.
Wed, Jan 13, 4:59 PM

Tue, Jan 12

alangenfeld accepted D5938: Add insturctions to run dagster instance migrate to the MIGRATION.md, plus instructions for how to turn on the new scheduler.

wartortle

Tue, Jan 12, 11:11 PM
alangenfeld added a reviewer for D5926: Raise a migration required exception when you load a pre-0.10.0 schedule storage: max.
Tue, Jan 12, 11:09 PM
alangenfeld added a comment to D5926: Raise a migration required exception when you load a pre-0.10.0 schedule storage.

I do think it would be good to start stamping postgres alembic revisions, just so we have the flexibility to know for sure what revision a given user is on

Tue, Jan 12, 11:09 PM
alangenfeld added inline comments to D5829: Modernize dagstermill file handling.
Tue, Jan 12, 11:05 PM
alangenfeld added a comment to D5916: create run partition data migration, tie to `dagster instance migrate`.

i dont know, I think building missing indexes on migration seems reasonable, the tracking of indexes in each storage also makes sense. I think dropping all this stuff will have to come with some big change to the instance set up

Tue, Jan 12, 10:26 PM
alangenfeld accepted D5925: InputManager -> RootInputManager.

pidgeot

Tue, Jan 12, 9:55 PM
alangenfeld accepted D5916: create run partition data migration, tie to `dagster instance migrate`.

i don't love this complexity burden but i think its the right call for end user experience

Tue, Jan 12, 9:54 PM
alangenfeld accepted D5924: unexpose output managers.

cloyster

Tue, Jan 12, 9:35 PM
alangenfeld requested changes to D5780: eliminate Persistable.

lets go for the consolidation while we're making changes, should just be easy enough

Tue, Jan 12, 9:34 PM
alangenfeld accepted D5829: Modernize dagstermill file handling.

some complex stuff but i think the comments and cleanup make it parseable at least, thanks for that.

Tue, Jan 12, 9:31 PM
alangenfeld added a comment to D5474: ObjectStoreOperation/AssetStoreOperation -> HandledOutput/LoadedInput.

oops had some old inlines i forgot to submit too

Tue, Jan 12, 7:38 PM
alangenfeld accepted D5928: Remove py2 test skip markers that are now unused.

cleanup

Tue, Jan 12, 7:22 PM
alangenfeld accepted D5927: Remove misc py2 compat stuff.

verynice

Tue, Jan 12, 7:22 PM
alangenfeld closed D5907: python 2 docs cleanup.
Tue, Jan 12, 5:03 PM
alangenfeld committed R1:4fba0845aca1: python 2 docs cleanup (authored by alangenfeld).
python 2 docs cleanup
Tue, Jan 12, 5:03 PM
alangenfeld closed D5920: remove legacy repository.yaml support.
Tue, Jan 12, 4:59 PM
alangenfeld committed R1:827e2791ce48: remove legacy repository.yaml support (authored by alangenfeld).
remove legacy repository.yaml support
Tue, Jan 12, 4:59 PM
alangenfeld added a comment to D5780: eliminate Persistable.

typing.NamedTuples don't play well with double inheritance

Tue, Jan 12, 4:57 PM
alangenfeld added a reviewer for D5780: eliminate Persistable: prha.
Tue, Jan 12, 4:57 PM
alangenfeld requested review of D5920: remove legacy repository.yaml support.
Tue, Jan 12, 4:56 PM
alangenfeld closed D5910: [cli] add --run-id to dagster pipeline launch.
Tue, Jan 12, 4:42 PM
alangenfeld committed R1:c1a670e29004: [cli] add --run-id to dagster pipeline launch (authored by alangenfeld).
[cli] add --run-id to dagster pipeline launch
Tue, Jan 12, 4:42 PM

Mon, Jan 11

alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase

Mon, Jan 11, 11:06 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

rebase

Mon, Jan 11, 11:04 PM
alangenfeld retitled D5721: DynamicOutput DSL map approach from RFC DynamicOutput DSL map approach to DynamicOutput DSL map approach.
Mon, Jan 11, 11:04 PM
alangenfeld accepted D5843: root input manager.

finallydone

Mon, Jan 11, 10:40 PM
alangenfeld added a comment to D5916: create run partition data migration, tie to `dagster instance migrate`.

probably makes sense, lets just get a test in place before landing

Mon, Jan 11, 10:38 PM
alangenfeld requested changes to D5767: (buildkite 10/n) Add Windows tests.

to your queue for now - between these tests failing and the total run time im not sure this gets us in to much better of a spot than we are now on azure. Lets revisit after release

Mon, Jan 11, 10:08 PM
alangenfeld accepted D5890: Expose custom run launcher option in helm values.

dratini

Mon, Jan 11, 10:07 PM