Page MenuHomePhabricator
Feed Advanced Search

Jan 11 2021

alangenfeld accepted D5889: Expose custom scheduler option in helm values.
Jan 11 2021, 10:06 PM
alangenfeld resigned from D5891: Update helm defaults for user code deployments, run launcher, and rabbitmq.

defer to @catherinewu for final accept to make sure docs are aligned

Jan 11 2021, 10:06 PM
alangenfeld added a comment to D5767: (buildkite 10/n) Add Windows tests.

woof those times are insane

Jan 11 2021, 10:00 PM
alangenfeld updated the diff for D5910: [cli] add --run-id to dagster pipeline launch.

add test case to show re-use of run_id fails with good error

Jan 11 2021, 8:34 PM
alangenfeld closed D5612: DynamicOutput enforce constraints.
Jan 11 2021, 8:25 PM
alangenfeld committed R1:2f9ca0165b05: DynamicOutput enforce constraints (authored by alangenfeld).
DynamicOutput enforce constraints
Jan 11 2021, 8:25 PM
alangenfeld closed D5542: DynamicOutput execution support.
Jan 11 2021, 8:24 PM
alangenfeld committed R1:7da6eee9b486: DynamicOutput execution support (authored by alangenfeld).
DynamicOutput execution support
Jan 11 2021, 8:24 PM
alangenfeld closed D5755: DagsterEvent step_handle.
Jan 11 2021, 8:24 PM
alangenfeld committed R1:5aaebe6e226d: DagsterEvent step_handle (authored by alangenfeld).
DagsterEvent step_handle
Jan 11 2021, 8:24 PM
alangenfeld requested review of D5910: [cli] add --run-id to dagster pipeline launch.
Jan 11 2021, 6:10 PM
alangenfeld requested changes to D5821: Add additional validation for start date and end date timezones.

I think this is in a slightly confusing state, I would guess due to how it started vs the latest approach. Sending to you for one more spin for some mix of more tests or renaming / rewording

Jan 11 2021, 6:08 PM
alangenfeld accepted D5819: Remove default partition selector from PartitionSetDefinition.create_schedule_definition.

dugtrio

Jan 11 2021, 5:56 PM
alangenfeld requested review of D5907: python 2 docs cleanup.
Jan 11 2021, 5:27 PM
alangenfeld accepted D5855: RFC: fix io manager optional fan in.

thanks!

Jan 11 2021, 4:59 PM
alangenfeld added inline comments to D5721: DynamicOutput DSL map approach.
Jan 11 2021, 4:54 PM
alangenfeld added inline comments to D5891: Update helm defaults for user code deployments, run launcher, and rabbitmq.
Jan 11 2021, 4:51 PM

Jan 9 2021

alangenfeld added inline comments to D5721: DynamicOutput DSL map approach.
Jan 9 2021, 12:22 AM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

more clean up

Jan 9 2021, 12:22 AM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

pylint hax engage

Jan 9 2021, 12:19 AM

Jan 8 2021

alangenfeld requested changes to D5855: RFC: fix io manager optional fan in.

I have a good feeling that this will feel better all pulled in to one spot

Jan 8 2021, 11:00 PM
alangenfeld added a comment to D5855: RFC: fix io manager optional fan in.

comments inline - I think this will be much clearer if all of this complexity is brought in to one spot with some comments

Jan 8 2021, 10:58 PM
alangenfeld accepted D5873: [2/2] Add helm schema for dagster run launcher.
Jan 8 2021, 10:52 PM
alangenfeld accepted D5872: [1/2] Add helm configuration for dagster run launcher.

i take it the new defaults will be in a followup diff?

Jan 8 2021, 10:49 PM
alangenfeld requested changes to D5843: root input manager.

based on our earlier conversation it seems we may punt on this for release due to the complexity around:

  • type based required resource keys
  • interactions with default value loading
Jan 8 2021, 10:45 PM
alangenfeld accepted D5857: Refresh workspaces overview.

vulpix

Jan 8 2021, 10:43 PM
alangenfeld added inline comments to D5824: Revamp deploying locally and deploying in docker docs to reference dagster-daemon and new deploy_docker example.
Jan 8 2021, 10:39 PM
alangenfeld accepted D5824: Revamp deploying locally and deploying in docker docs to reference dagster-daemon and new deploy_docker example.
Jan 8 2021, 10:39 PM
alangenfeld requested changes to D5829: Modernize dagstermill file handling.

to your q

Jan 8 2021, 10:34 PM
alangenfeld added inline comments to D5829: Modernize dagstermill file handling.
Jan 8 2021, 10:34 PM
alangenfeld accepted D5756: docs for schedules for 0.10.0.

growlithe

Jan 8 2021, 10:11 PM
alangenfeld accepted D5748: [2/2] Add helm schema for dagster compute log manager.
Jan 8 2021, 10:09 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase on fixes

Jan 8 2021, 8:54 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

some fixes, still looking in to lint

Jan 8 2021, 8:51 PM
alangenfeld accepted D4920: Update Scheduler helm schema to use enum types.
Jan 8 2021, 7:31 PM
alangenfeld added inline comments to D5855: RFC: fix io manager optional fan in.
Jan 8 2021, 6:44 PM
alangenfeld requested changes to D5855: RFC: fix io manager optional fan in.

ooof this sucks

Jan 8 2021, 6:41 PM
alangenfeld abandoned D5613: RFC DynamicOutput DSL __iter__ approach.
Jan 8 2021, 6:19 PM
alangenfeld added a comment to D5721: DynamicOutput DSL map approach.

What if we added a map method to InvokedSolidOutputHandle that raises a NotImplementedError?

Jan 8 2021, 4:49 PM

Jan 7 2021

alangenfeld requested review of D5721: DynamicOutput DSL map approach.

can we add some magic to help with the lint warnings?

Jan 7 2021, 11:46 PM
alangenfeld updated the summary of D5721: DynamicOutput DSL map approach.
Jan 7 2021, 11:30 PM
alangenfeld retitled D5721: DynamicOutput DSL map approach from RFC DynamicOutput DSL map & forEach approach to RFC DynamicOutput DSL map approach.
Jan 7 2021, 11:29 PM
alangenfeld updated the diff for D5542: DynamicOutput execution support.

feedback

Jan 7 2021, 11:27 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

forEach -> map

Jan 7 2021, 10:36 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

drop forEach

Jan 7 2021, 10:34 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

fixup

Jan 7 2021, 10:24 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

lint

Jan 7 2021, 7:51 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase

Jan 7 2021, 7:44 PM
alangenfeld accepted D5860: str -> StringSource for azure compute log manager config.

goodcatch

Jan 7 2021, 7:43 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

rebase

Jan 7 2021, 7:41 PM
alangenfeld updated the diff for D5612: DynamicOutput enforce constraints.

rebase

Jan 7 2021, 7:40 PM
alangenfeld updated the diff for D5542: DynamicOutput execution support.

rebase

Jan 7 2021, 7:40 PM
alangenfeld added a comment to D5721: DynamicOutput DSL map approach.

If we have map, do we also need forEach?

Jan 7 2021, 7:34 PM
alangenfeld updated the diff for D5755: DagsterEvent step_handle.

attempt 2

Jan 7 2021, 7:27 PM
alangenfeld updated the diff for D5755: DagsterEvent step_handle.

fix celery

Jan 7 2021, 6:37 PM
alangenfeld retitled D5755: DagsterEvent step_handle from RFC DagsterEvent step_handle to DagsterEvent step_handle.
Jan 7 2021, 6:32 PM
alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase

Jan 7 2021, 6:01 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

rebase

Jan 7 2021, 5:59 PM
alangenfeld updated the diff for D5612: DynamicOutput enforce constraints.

rebase

Jan 7 2021, 5:59 PM
alangenfeld updated the diff for D5542: DynamicOutput execution support.

rebase

Jan 7 2021, 5:58 PM
alangenfeld updated the diff for D5755: DagsterEvent step_handle.

rebase

Jan 7 2021, 5:55 PM
alangenfeld accepted D5823: add lazy migration for the secondary indexed asset keys.

push_n_pray

Jan 7 2021, 5:27 PM
alangenfeld requested changes to D5744: RFC: handle type-based loading with input managers.

can you clarify what you want feedback on at this stage? if there isn't any RFCish stuff left prep this for final review

Jan 7 2021, 5:07 PM
alangenfeld added a comment to D5748: [2/2] Add helm schema for dagster compute log manager.

TEST PLAN
integration
bk

Jan 7 2021, 4:59 PM
alangenfeld accepted D5745: [1/2] Add helm configuration for dagster compute log manager.

im still worried about keeping these config fields up to date - but ill let you make the final call

Jan 7 2021, 4:46 PM
alangenfeld accepted D5854: update repository tutorial to use new executable_path key.
Jan 7 2021, 4:30 PM

Jan 6 2021

alangenfeld updated the diff for D5632: DynamicOutput re-execution.

rebase, adding step_handle to DagsterEvent shown in D5755

Jan 6 2021, 10:15 PM
alangenfeld closed D5850: Revert "Change default sqlite watchdog on mac so that it works in tox environments".
Jan 6 2021, 10:13 PM
alangenfeld committed R1:ea792e5f0fc0: Revert "Change default sqlite watchdog on mac so that it works in tox… (authored by alangenfeld).
Revert "Change default sqlite watchdog on mac so that it works in tox…
Jan 6 2021, 10:13 PM
alangenfeld updated the test plan for D5850: Revert "Change default sqlite watchdog on mac so that it works in tox environments".
Jan 6 2021, 8:21 PM
alangenfeld updated the diff for D5755: DagsterEvent step_handle.

update

Jan 6 2021, 5:17 PM

Jan 5 2021

alangenfeld added a comment to D5745: [1/2] Add helm configuration for dagster compute log manager.

the full custom alternative path is less of a concern since that is unlikely - but I do worry that the config setup does not actually work if attempted to be loaded given the provided test plan

Jan 5 2021, 10:34 PM
alangenfeld added inline comments to D5745: [1/2] Add helm configuration for dagster compute log manager.
Jan 5 2021, 10:29 PM
alangenfeld added a comment to D5745: [1/2] Add helm configuration for dagster compute log manager.

is there an escape hatch here for providing a custom one?

Jan 5 2021, 10:23 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

rebase

Jan 5 2021, 10:13 PM
alangenfeld retitled D5721: DynamicOutput DSL map approach from RFC DynamicOutput DSL forEach approach to RFC DynamicOutput DSL map & forEach approach.
Jan 5 2021, 10:04 PM
alangenfeld updated the diff for D5721: DynamicOutput DSL map approach.

add map

Jan 5 2021, 9:49 PM
alangenfeld accepted D5796: Modernize dagstermill test setup.

cleanup

Jan 5 2021, 8:07 PM
alangenfeld accepted D5750: Add deploy_docker example that builds from pypi.

take a note in some form to test this once we publish a prerelease

Jan 5 2021, 8:03 PM
alangenfeld requested changes to D5744: RFC: handle type-based loading with input managers.

to your queue post discussion, summary was roughly trying to simplify things a bit by not having overlapping responsibilities between input and object managers, and moving towards a RootInputManager set up where it is only responsible for root loads

Jan 5 2021, 7:42 PM
alangenfeld resigned from D5651: [Github Community] Add a PR template.

ill defer to @max to make the final call

Jan 5 2021, 4:30 PM
alangenfeld requested changes to D5381: RFC (for discussion): Make executor definitions resource definitions.

clearing queue for 0.10.0 burn down - request review post release if there is more discussion this can generate

Jan 5 2021, 4:10 PM
alangenfeld requested changes to D5428: [RFC] Resource keys re-mapping.

to your queue

Jan 5 2021, 4:09 PM
alangenfeld added a comment to D5428: [RFC] Resource keys re-mapping.

Do we need this in for 0.10.0? If yes we should figure out a path forward, likely punting on solving the name issue. If not lets revisit post release

Jan 5 2021, 4:08 PM
alangenfeld accepted D5809: fix build after race condition with removal of seven.

gogogo

Jan 5 2021, 12:44 AM
alangenfeld published D5809: fix build after race condition with removal of seven for review.
Jan 5 2021, 12:44 AM

Jan 4 2021

alangenfeld added inline comments to D5790: [dagit] Create copy of DagsterGraphQLContext for each request.
Jan 4 2021, 7:42 PM
alangenfeld added inline comments to D5790: [dagit] Create copy of DagsterGraphQLContext for each request.
Jan 4 2021, 6:54 PM
alangenfeld accepted D5757: Make DagsterDaemonScheduler the default scheduler.

alright thats an acceptable enough base line i think - and chances are it will improve a little before launch

Jan 4 2021, 5:25 PM
alangenfeld added a comment to D5757: Make DagsterDaemonScheduler the default scheduler.

We should think about what kind of affordances we might want to put into dagit that prompt people to go deploy the daemon though.

Jan 4 2021, 5:08 PM
alangenfeld accepted D5778: Replace namedtuple with typing.NamedTuple in outputs.py.

from what i can tell there are no differences to this approach besides the additional typing information on __annotations__
https://docs.python.org/3/library/typing.html

Jan 4 2021, 5:06 PM
alangenfeld created Image Macro "absolute-win".
Jan 4 2021, 5:05 PM
alangenfeld accepted D5761: Clean up legacy 2/3 compat.

thanks for cleaning up

Jan 4 2021, 4:39 PM
alangenfeld resigned from D5764: Run mypy on dagster core.

will let @sandyryza resolve this and D5768

Jan 4 2021, 4:33 PM
alangenfeld resigned from D5768: mypy.

will let @max resolve this and D5764

Jan 4 2021, 4:33 PM
alangenfeld accepted D5747: Type hints for check.

D5768 / D5764

Jan 4 2021, 4:24 PM

Dec 22 2020

alangenfeld published D5755: DagsterEvent step_handle for review.

some more test call sites to patch up but this gives a gist of what this could look like

Dec 22 2020, 10:44 PM
alangenfeld added a comment to D5747: Type hints for check.

does the our existing mypy check in buildkite exercise this? should we add a step?

Dec 22 2020, 9:30 PM
alangenfeld resigned from D5742: Remove DagstermillExecutionError altogether.

I am pretty sure this is legit given how a lot of the dagstermill impl predates so much of the shifting that has happened around the core execution pathway but I will defer to @max to be sure

Dec 22 2020, 7:49 PM
alangenfeld accepted D5731: re-enable dagstermill tests with an ipykernel pin.
Dec 22 2020, 5:53 PM