Page MenuHomePhabricator

alangenfeld (Alex Langenfeld)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Mar 20 2019, 8:25 PM (22 w, 1 d)
Roles
Administrator

Recent Activity

Yesterday

alangenfeld accepted D877: Set environment variables on ScheduleDefinition.

thumbsup

Thu, Aug 22, 9:57 PM
alangenfeld committed R1:a08b07d94c42: fix bad rebase (authored by alangenfeld).
fix bad rebase
Thu, Aug 22, 8:43 PM
alangenfeld committed R1:bb5ae923c591: [storage] handle missing event log files (authored by alangenfeld).
[storage] handle missing event log files
Thu, Aug 22, 8:19 PM
alangenfeld closed D870: [storage] handle missing event log files.
Thu, Aug 22, 8:19 PM
alangenfeld updated the diff for D870: [storage] handle missing event log files.

verify_event_log_exists

Thu, Aug 22, 8:04 PM
alangenfeld added a comment to D874: RFC: Remove lambda_solid from tutorial.

hmmmm

Thu, Aug 22, 7:53 PM
alangenfeld accepted D851: RFC: Add support for specifying Dask resources.

shipit

Thu, Aug 22, 4:40 PM
alangenfeld committed R1:cae2fa5a4db7: [RFC] simpler user defined types section (authored by alangenfeld).
[RFC] simpler user defined types section
Thu, Aug 22, 4:25 PM
alangenfeld closed D861: [RFC] simpler user defined types section.
Thu, Aug 22, 4:25 PM
alangenfeld created D870: [storage] handle missing event log files.
Thu, Aug 22, 3:25 PM
alangenfeld accepted D868: Add a more prominent Slack button to the readme.

illallowit

Thu, Aug 22, 2:40 PM

Wed, Aug 21

alangenfeld commandeered D862: Add naive filesystem log watchers.
Wed, Aug 21, 10:43 PM
alangenfeld requested changes to D858: Overhauled run history view.
Wed, Aug 21, 10:27 PM
alangenfeld requested changes to D862: Add naive filesystem log watchers.

I will commandeer this (or the origin diff)

Wed, Aug 21, 9:50 PM
alangenfeld added inline comments to D862: Add naive filesystem log watchers.
Wed, Aug 21, 9:40 PM
alangenfeld accepted D859: Break log storage out from run storage.

move eventlog stuff to new file please

Wed, Aug 21, 9:35 PM
alangenfeld added a comment to D851: RFC: Add support for specifying Dask resources.

some other options to consider:

Wed, Aug 21, 9:10 PM
alangenfeld updated the summary of D861: [RFC] simpler user defined types section.
Wed, Aug 21, 8:59 PM
alangenfeld created D861: [RFC] simpler user defined types section.
Wed, Aug 21, 8:57 PM
alangenfeld accepted D853: Use __slots__ for PipelineRun.

no real cost to this though so whatevs

Wed, Aug 21, 3:20 PM
alangenfeld added a comment to D853: Use __slots__ for PipelineRun.

whats the point of slots? just speculative memory optimization?

Wed, Aug 21, 3:19 PM
alangenfeld accepted D852: Bring pyproject.toml black settings in line with CLI.
Wed, Aug 21, 3:10 PM
alangenfeld accepted D841: Consolidate double implementation of run storage.
Wed, Aug 21, 3:10 PM
alangenfeld accepted D850: Refactor PipelineRun.

the only thing to consider is moving some stuff out to separate files

Wed, Aug 21, 2:51 PM
alangenfeld committed R1:32f8b7f63ac5: [dagster] fix composition bug (authored by alangenfeld).
[dagster] fix composition bug
Wed, Aug 21, 2:47 PM
alangenfeld closed D847: [dagster] fix composition bug.
Wed, Aug 21, 2:47 PM

Tue, Aug 20

alangenfeld updated the diff for D847: [dagster] fix composition bug.

reuse solid handle to and use that to index for compute step

Tue, Aug 20, 9:00 PM
alangenfeld added a comment to D841: Consolidate double implementation of run storage.

just looking for answers to [1]

Tue, Aug 20, 8:14 PM
alangenfeld added a comment to D844: Initial pass at custom modification of the Dagre layout for wide pipelines.

do we have any tests for dagre?

Tue, Aug 20, 8:13 PM
alangenfeld accepted D845: Add dagster-aws up EC2 syncing.
Tue, Aug 20, 7:46 PM
alangenfeld requested changes to D842: Create ScheduleDefinition and dagster CLI scheduler commands.

dagster schedule list
dagster schedule running

Tue, Aug 20, 6:58 PM
alangenfeld updated the summary of D847: [dagster] fix composition bug.
Tue, Aug 20, 6:41 PM
alangenfeld updated the diff for D847: [dagster] fix composition bug.

cleamn

Tue, Aug 20, 6:41 PM
alangenfeld created D847: [dagster] fix composition bug.
Tue, Aug 20, 6:39 PM
alangenfeld created P17 (An Untitled Masterwork).
Tue, Aug 20, 3:14 PM
alangenfeld accepted D843: Break up AWS CLI and add tests.
Tue, Aug 20, 12:13 AM
alangenfeld added a comment to D843: Break up AWS CLI and add tests.

so this takes care of everything so its just pip install dagster-aws; dagster-aws; (include in test plan)?

Tue, Aug 20, 12:13 AM
alangenfeld accepted D826: Implement start_schedule and end_schedule for SystemCronTabScheduler.

thumbsup

Tue, Aug 20, 12:10 AM

Mon, Aug 19

alangenfeld added a comment to D841: Consolidate double implementation of run storage.

would love to see a stacked diff on top of this that takes a pass at cleaning up the weird class hierarchy / poor naming

Mon, Aug 19, 9:59 PM
alangenfeld added inline comments to D841: Consolidate double implementation of run storage.
Mon, Aug 19, 9:56 PM
alangenfeld changed the image for Image Macro "grumpycat".
Mon, Aug 19, 9:54 PM
alangenfeld created Image Macro "grumpycat".
Mon, Aug 19, 9:52 PM
alangenfeld committed R1:0de98e61b1d8: [dagster] whitelisted namedtuple serdes (authored by alangenfeld).
[dagster] whitelisted namedtuple serdes
Mon, Aug 19, 9:25 PM
alangenfeld closed D835: [dagster] whitelisted namedtuple serdes.
Mon, Aug 19, 9:25 PM
alangenfeld committed R1:c8b42fae28f9: [dagster] remove event_callback from RunConfig (authored by alangenfeld).
[dagster] remove event_callback from RunConfig
Mon, Aug 19, 9:24 PM
alangenfeld closed D829: [dagster] remove event_callback from RunConfig.
Mon, Aug 19, 9:24 PM
alangenfeld committed R1:8911ca4db53d: [dagster] EventSink (authored by alangenfeld).
[dagster] EventSink
Mon, Aug 19, 9:23 PM
alangenfeld closed D824: [dagster] EventSink.
Mon, Aug 19, 9:23 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

rebase

Mon, Aug 19, 9:15 PM
alangenfeld accepted D840: Temporarily pin ge to 0.7.6 or less.
Mon, Aug 19, 8:57 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

comment

Mon, Aug 19, 8:54 PM
alangenfeld accepted D832: Add EC2 setup script.

youve got enough code here to warrant breaking things out to a few files, now or in a subsequent diff

Mon, Aug 19, 7:48 PM
alangenfeld added a reviewer for D829: [dagster] remove event_callback from RunConfig: Restricted Project.

oops forgot to add reviewers

Mon, Aug 19, 6:44 PM
alangenfeld requested changes to D789: Naive watcher for filesystem log storage.

to your queue for restacking on some lower level clean ups

Mon, Aug 19, 6:40 PM
alangenfeld accepted D838: Re-enable coverage.

illallowit

Mon, Aug 19, 4:18 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

rebase

Mon, Aug 19, 3:47 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

up

Mon, Aug 19, 3:45 PM
alangenfeld accepted D830: Add initial sqllite run storage.
Mon, Aug 19, 3:30 PM
alangenfeld requested changes to D826: Implement start_schedule and end_schedule for SystemCronTabScheduler.
Mon, Aug 19, 3:29 PM
alangenfeld added inline comments to D826: Implement start_schedule and end_schedule for SystemCronTabScheduler.
Mon, Aug 19, 3:26 PM

Fri, Aug 16

alangenfeld added a comment to D789: Naive watcher for filesystem log storage.

ideally we also wouldn't even watch the log files for external runs unless we had an active subscription on them

Fri, Aug 16, 11:35 PM
alangenfeld added a comment to D789: Naive watcher for filesystem log storage.

we could land this after a bit of cleanup - but i think im tempted to just have this feed in to a more systemic refactor of all this PipelineRun stuff.

Fri, Aug 16, 11:13 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

open and close connections every time

Fri, Aug 16, 9:52 PM
alangenfeld added inline comments to D830: Add initial sqllite run storage.
Fri, Aug 16, 9:39 PM
alangenfeld resigned from D831: RFC: push config obj instantiation into `reconstitute_pipeline_context`.

defering to max
cleanup

Fri, Aug 16, 9:36 PM
alangenfeld added a comment to D832: Add EC2 setup script.

should put this somewhere other than bin?

Fri, Aug 16, 9:35 PM
alangenfeld updated subscribers of D826: Implement start_schedule and end_schedule for SystemCronTabScheduler.

looks pretty solid to me

Fri, Aug 16, 9:30 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

hold connection

Fri, Aug 16, 9:22 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

rebase

Fri, Aug 16, 8:54 PM
alangenfeld updated the diff for D829: [dagster] remove event_callback from RunConfig.

rebase

Fri, Aug 16, 8:52 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

rebase

Fri, Aug 16, 8:52 PM
alangenfeld committed R1:981cb1cb36ec: [dagster-graphql] use obj for required resource (authored by alangenfeld).
[dagster-graphql] use obj for required resource
Fri, Aug 16, 8:44 PM
alangenfeld closed D836: [dagster-graphql] use obj for required resource.
Fri, Aug 16, 8:44 PM
alangenfeld accepted D821: Add dbt test solid.

might be worth moving from init to a couple smaller files

Fri, Aug 16, 8:38 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

rebase w/ fixes

Fri, Aug 16, 8:31 PM
alangenfeld created D836: [dagster-graphql] use obj for required resource.
Fri, Aug 16, 8:30 PM
alangenfeld updated the diff for D835: [dagster] whitelisted namedtuple serdes.

use decorator

Fri, Aug 16, 6:50 PM
alangenfeld updated subscribers of D835: [dagster] whitelisted namedtuple serdes.
Fri, Aug 16, 6:06 PM
alangenfeld planned changes to D835: [dagster] whitelisted namedtuple serdes.
Fri, Aug 16, 5:46 PM
alangenfeld created D835: [dagster] whitelisted namedtuple serdes.
Fri, Aug 16, 5:30 PM

Thu, Aug 15

alangenfeld updated the diff for D829: [dagster] remove event_callback from RunConfig.

fixes

Thu, Aug 15, 11:26 PM
alangenfeld created D829: [dagster] remove event_callback from RunConfig.
Thu, Aug 15, 10:17 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

comments

Thu, Aug 15, 9:24 PM
alangenfeld added inline comments to D824: [dagster] EventSink.
Thu, Aug 15, 9:06 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

rebase

Thu, Aug 15, 9:00 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

fixe

Thu, Aug 15, 8:26 PM
alangenfeld planned changes to D824: [dagster] EventSink.
Thu, Aug 15, 7:05 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

add on_pipeline_init and on_pipeline_teardown methods

Thu, Aug 15, 6:56 PM
alangenfeld updated the summary of D824: [dagster] EventSink.
Thu, Aug 15, 6:40 PM
alangenfeld updated the diff for D824: [dagster] EventSink.

clean up

Thu, Aug 15, 6:36 PM
alangenfeld created D824: [dagster] EventSink.
Thu, Aug 15, 6:25 PM
alangenfeld added a comment to D819: Add custom behavior to the CellTruncationProvider.

this looks good to me but ill give @bengotow time to look

Thu, Aug 15, 3:55 PM

Wed, Aug 14

alangenfeld committed R1:fb384ad3b262: [log sink] multiprocess executor (authored by alangenfeld).
[log sink] multiprocess executor
Wed, Aug 14, 8:41 PM
alangenfeld closed D812: [log sink] multiprocess executor.
Wed, Aug 14, 8:41 PM
alangenfeld added inline comments to D815: Promote some step events to INFO.
Wed, Aug 14, 8:28 PM
alangenfeld updated the diff for D812: [log sink] multiprocess executor.

cant remove the log_step_event call

Wed, Aug 14, 8:12 PM
alangenfeld updated the diff for D812: [log sink] multiprocess executor.

fixes

Wed, Aug 14, 7:22 PM
alangenfeld added a comment to D815: Promote some step events to INFO.

what does dagster cli output look like on hello world?

Wed, Aug 14, 6:49 PM
alangenfeld accepted D813: Throwing up example yaml-driven deps.

can you throw a big comment block somewhere in here indicating exactly what this is? I could see some one hitting this too early and thinking its "the way" to do things

Wed, Aug 14, 6:48 PM
alangenfeld accepted D804: Create Scheduler GraphQL interface.
Wed, Aug 14, 6:46 PM