Page MenuHomeElementl

alangenfeld (Alex Langenfeld)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Mar 20 2019, 8:25 PM (117 w, 6 d)
Roles
Administrator

Recent Activity

Mon, Jun 21

alangenfeld closed D8476: mypy and prune RunConfigSchema.
Mon, Jun 21, 7:15 PM
alangenfeld committed R1:19b3071d5730: mypy and prune RunConfigSchema (authored by alangenfeld).
mypy and prune RunConfigSchema
Mon, Jun 21, 7:15 PM
alangenfeld closed D8463: tidy up create_run_for_pipeline.
Mon, Jun 21, 7:13 PM
alangenfeld committed R1:3d74dffacbac: tidy up create_run_for_pipeline (authored by alangenfeld).
tidy up create_run_for_pipeline
Mon, Jun 21, 7:13 PM

Fri, Jun 18

alangenfeld requested review of D8464: get rid of 'environment config' phrasing.
Fri, Jun 18, 10:45 PM
alangenfeld requested review of D8476: mypy and prune RunConfigSchema.
Fri, Jun 18, 10:45 PM
alangenfeld requested review of D8463: tidy up create_run_for_pipeline.
Fri, Jun 18, 10:43 PM
alangenfeld added a comment to D8468: feat: add support for dict of ins and outs.

unclear if inheritance is the way to go with In and Out once there is this behavior difference. Might be best to have a simple value object and interpret it in to the underlying def, maybe with a scheme like [1]

Fri, Jun 18, 7:32 PM
alangenfeld accepted D8365: correctly process configs of io managers on solids inside composite.

check inlines before landing - i think its worth cleaning up

Fri, Jun 18, 4:30 PM
alangenfeld edited reviewers for D8455: Pin dagster-XXX libraries to the same version as dagster, added: owen, sandyryza, yuhan; removed: alangenfeld.

i thiiink this is fine but will swap out for folks who have done practitioner-ing to weigh in

Fri, Jun 18, 4:10 PM
alangenfeld accepted D8272: ban io_manager_key and root_manager_key on composite solids.

thumbsup

Fri, Jun 18, 4:08 PM
alangenfeld accepted D8435: In and Out.

lets defer dict style support to a follow up

Fri, Jun 18, 3:59 PM
alangenfeld added inline comments to D8435: In and Out.
Fri, Jun 18, 3:52 PM
alangenfeld added a comment to D8435: In and Out.

Accept a dict of Ins instead of a List. I went in thinking I would do this, but came out not thinking it was particularly better. The number of characters a user needs to type is identical. It doesn't work for Outs, because, if we want the tuple-stuff mentioned above to work, the Outs need to be ordered, and supplying an OrderedDict is a pain.

Fri, Jun 18, 3:43 PM
alangenfeld accepted D8426: op.
Fri, Jun 18, 3:41 PM
alangenfeld accepted D8425: add graph to top-level imports.
Fri, Jun 18, 3:40 PM
alangenfeld accepted D8275: Make solid invocation directly call the underlying decorated fxn.
Fri, Jun 18, 2:56 PM
alangenfeld added a comment to D8435: In and Out.

the Outs need to be ordered, and supplying an OrderedDict is a pain

Fri, Jun 18, 1:57 PM

Tue, Jun 15

alangenfeld added inline comments to D8275: Make solid invocation directly call the underlying decorated fxn.
Tue, Jun 15, 8:58 PM
alangenfeld requested changes to D8275: Make solid invocation directly call the underlying decorated fxn.
Tue, Jun 15, 7:25 PM
alangenfeld added inline comments to D8272: ban io_manager_key and root_manager_key on composite solids.
Tue, Jun 15, 2:39 PM
alangenfeld added inline comments to D8272: ban io_manager_key and root_manager_key on composite solids.
Tue, Jun 15, 2:37 PM
alangenfeld added inline comments to D8365: correctly process configs of io managers on solids inside composite.
Tue, Jun 15, 2:37 PM
alangenfeld added inline comments to D8365: correctly process configs of io managers on solids inside composite.
Tue, Jun 15, 2:29 PM
alangenfeld accepted D8371: fix images on Pipelines page.
Tue, Jun 15, 2:22 PM

Mon, Jun 14

alangenfeld added inline comments to D8356: don't require context arg when solid has required resources.
Mon, Jun 14, 9:30 PM
alangenfeld added inline comments to D8324: get_stored_events and get_stored_runs.
Mon, Jun 14, 5:43 PM
alangenfeld added inline comments to D8272: ban io_manager_key and root_manager_key on composite solids.
Mon, Jun 14, 4:37 PM
alangenfeld accepted D8283: [crag] partitions on modes.

should add a test for lazy job construction with partition too

Mon, Jun 14, 4:35 PM
alangenfeld accepted D8355: mark to_job as experimental.
Mon, Jun 14, 3:32 PM
alangenfeld accepted D8354: add option to skip uploading empty files to improve perf with minio.

works for me

Mon, Jun 14, 3:32 PM

Fri, Jun 11

alangenfeld updated the diff for D6648: [RFC] pipeline contains a graph.

lint

Fri, Jun 11, 7:42 PM
alangenfeld added a comment to D8344: [crag] job name.

It seems odd to provide this and not just a name param

Fri, Jun 11, 6:09 PM
alangenfeld updated the diff for D6648: [RFC] pipeline contains a graph.

test, lint, mypy

Fri, Jun 11, 5:53 PM
alangenfeld added reviewers for D8324: get_stored_events and get_stored_runs: owen, prha.

Add some other reviewers so that hopefully this is the last rename diff

Fri, Jun 11, 5:42 PM
alangenfeld accepted D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.

maybe mention "crag" in the repository location that loads these repos if you add it to the root workspace (Which i think makes sense to do)

Fri, Jun 11, 5:18 PM
alangenfeld published D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of for review.
Fri, Jun 11, 5:17 PM
alangenfeld accepted D8344: [crag] job name.

wigglytuff

Fri, Jun 11, 5:15 PM
alangenfeld published D8344: [crag] job name for review.
Fri, Jun 11, 5:15 PM
alangenfeld updated the diff for D6648: [RFC] pipeline contains a graph.

rebas

Fri, Jun 11, 4:59 PM

Thu, Jun 10

alangenfeld closed D8318: [crag] default_config.
Thu, Jun 10, 9:58 PM
alangenfeld closed D8261: [crag] config mapping via mode.
Thu, Jun 10, 9:57 PM
alangenfeld committed R1:cd69e4679a0b: [crag] default_config (authored by alangenfeld).
[crag] default_config
Thu, Jun 10, 9:57 PM
alangenfeld committed R1:cd9826793481: [crag] config mapping via mode (authored by alangenfeld).
[crag] config mapping via mode
Thu, Jun 10, 9:57 PM
alangenfeld added a comment to D8283: [crag] partitions on modes.

would like @prha or someone who has spent more time thinking about partitions to weigh in on if this is the right API for the future

Thu, Jun 10, 9:49 PM
alangenfeld added a comment to D8283: [crag] partitions on modes.

where do we do partition set look ups? Is there a way to store a different type of targeting on the object that refers to it, like JobScoped instead of RepoScoped so you go get the Job and fetch it of there instead of getting it out of the repo?

Thu, Jun 10, 7:33 PM
alangenfeld added a comment to D8318: [crag] default_config.

Will this config be used when invoking the job from a schedule or sensor? I had been imagining that it would.

Thu, Jun 10, 5:53 PM
alangenfeld accepted D8275: Make solid invocation directly call the underlying decorated fxn.

welldone

Thu, Jun 10, 4:04 PM
alangenfeld added a comment to D8321: Add CLI command to reset migration state.

would be nice to have a mapping alembic version to dagster version

Thu, Jun 10, 2:48 PM
alangenfeld added a comment to D8321: Add CLI command to reset migration state.

well we should at least test for sqlite and PG reseting various snapshots back to base and then doing an instance migrate.

Thu, Jun 10, 2:39 PM
alangenfeld added a comment to D8326: chore: remove pointer.py.

thanks

Thu, Jun 10, 2:32 PM
alangenfeld accepted D8326: chore: remove pointer.py.
Thu, Jun 10, 2:32 PM
alangenfeld added a comment to D8321: Add CLI command to reset migration state.

is this safe? are all of our migrations idempotent? I faintly remember doing this manually and was getting errors on various migrations so I had to find the actual version right version to start from

Thu, Jun 10, 2:27 PM

Wed, Jun 9

alangenfeld requested review of D8318: [crag] default_config.
Wed, Jun 9, 10:35 PM
alangenfeld added a comment to D8317: [dagster-graphql-client] Fix `get_run_status` return type.

flexible typing got us

Wed, Jun 9, 8:56 PM
alangenfeld updated the diff for D8261: [crag] config mapping via mode.

some fixes for dagit

Wed, Jun 9, 7:39 PM
alangenfeld closed D8311: fix dagster-test.
Wed, Jun 9, 5:47 PM
alangenfeld committed R1:1a7452ffdf1e: fix dagster-test (authored by alangenfeld).
fix dagster-test
Wed, Jun 9, 5:46 PM
alangenfeld updated the diff for D8311: fix dagster-test.

dagster-slack instead

Wed, Jun 9, 5:42 PM
alangenfeld closed D8298: run dagster-test in BK.
Wed, Jun 9, 4:35 PM
alangenfeld committed R1:b4abd39ae243: run dagster-test in BK (authored by alangenfeld).
run dagster-test in BK
Wed, Jun 9, 4:35 PM
alangenfeld requested changes to D8275: Make solid invocation directly call the underlying decorated fxn.

q mgmt

Wed, Jun 9, 4:10 PM
alangenfeld updated the diff for D8298: run dagster-test in BK.

alright should be good to go

Wed, Jun 9, 4:09 PM
alangenfeld accepted D8306: Resolve Versions fix.
Wed, Jun 9, 4:07 PM
alangenfeld requested review of D8298: run dagster-test in BK.
Wed, Jun 9, 4:07 PM
alangenfeld added a comment to D8275: Make solid invocation directly call the underlying decorated fxn.

I think yes, we are still validating inputs right?

Wed, Jun 9, 3:41 PM
alangenfeld accepted D8300: Add force flag to skip prompt when deleting runs from cli.

graveler

Wed, Jun 9, 3:06 PM
alangenfeld accepted D8299: intro RunRecord and get_run_rows -> get_run_records.

I think this is a bit cleaner, thanks for the quick turn around

Wed, Jun 9, 2:20 PM

Tue, Jun 8

alangenfeld added a comment to D8261: [crag] config mapping via mode.

we could obfuscate or add a layer of indirection if we wanted

Tue, Jun 8, 10:18 PM
alangenfeld accepted D8296: add specific error message for pipeline invocation.

coolbeans

Tue, Jun 8, 10:16 PM
alangenfeld updated the diff for D8261: [crag] config mapping via mode.

feedback

Tue, Jun 8, 10:03 PM
alangenfeld accepted D8284: Simplify how we specify the list of available executors to execute_run_host_mode.

whynot

Tue, Jun 8, 9:56 PM
alangenfeld updated the test plan for D8261: [crag] config mapping via mode.
Tue, Jun 8, 9:51 PM
alangenfeld added a comment to D8261: [crag] config mapping via mode.

I think the main thing worth discussing here is that this shifts ModeDefinitions from objects that can live inside multiple pipelines to objects that are specific to a single pipeline
In legacy non-crag world, do we think its legitimate for a user to use the config_mapping arg on ModeDefinition, or is this just a hack to get crag working that doesn't require adding JobDefinition, etc.?

Tue, Jun 8, 9:44 PM
alangenfeld added inline comments to D8275: Make solid invocation directly call the underlying decorated fxn.
Tue, Jun 8, 9:06 PM
alangenfeld requested changes to D8272: ban io_manager_key and root_manager_key on composite solids.

request review if you disagree with proposed alternative approach

Tue, Jun 8, 8:51 PM
alangenfeld added a comment to D8209: [RFC] send python logging messages to the DagsterLogManager.

if we did this we would want it behind an opt-in gating of some kind right?

Tue, Jun 8, 8:47 PM
alangenfeld requested changes to D7945: RFC: add new api hooks for compute log manager to capture logs outside of a pipeline context, support process-based capture.

rebase and take a pass on initial feedback?

Tue, Jun 8, 8:17 PM
alangenfeld accepted D8225: Move solid error boundary closer to user code invocation to clean up stack traces.

welldone

Tue, Jun 8, 8:13 PM
alangenfeld added a reviewer for D8275: Make solid invocation directly call the underlying decorated fxn: owen.

I guess if the decorated fxn isn't a generator we can validate, and then if it is a generator, we wrap it in another generator that performs the validation?

Tue, Jun 8, 8:10 PM
alangenfeld closed D8245: expose retry attempts on SoildExecutionContext.
Tue, Jun 8, 8:01 PM
alangenfeld committed R1:085034a8c065: expose retry attempts on SoildExecutionContext (authored by alangenfeld).
expose retry attempts on SoildExecutionContext
Tue, Jun 8, 8:01 PM
alangenfeld accepted D8262: Create interface for determining python type of all DagsterTypes.

add a test case for the latest tuple update

Tue, Jun 8, 7:25 PM
alangenfeld added a comment to D8276: type annotations for _CacheingDefinitionIndex.

nice!

Tue, Jun 8, 4:48 PM
alangenfeld accepted D8239: Sensor invocation, remove evaluate_tick from API docs, change testing story.

follow up diff tho

Tue, Jun 8, 4:47 PM
alangenfeld added a comment to D8272: ban io_manager_key and root_manager_key on composite solids.

we would ban them on composite solids for consistency

Tue, Jun 8, 4:29 PM
alangenfeld requested review of D8261: [crag] config mapping via mode.
Tue, Jun 8, 4:16 PM
alangenfeld added a comment to D8213: pipeline sensor - fetch events without run id + sqlite trick.

nit: mypy the new functions

Tue, Jun 8, 4:12 PM
alangenfeld accepted D8276: type annotations for _CacheingDefinitionIndex.
Tue, Jun 8, 1:18 AM
alangenfeld added a comment to D8275: Make solid invocation directly call the underlying decorated fxn.

its a bit complex but we *could* do output validation if we wanted

Tue, Jun 8, 1:16 AM
alangenfeld updated the diff for D8245: expose retry attempts on SoildExecutionContext.

drop the attempt

Tue, Jun 8, 1:09 AM
alangenfeld accepted D8231: [RFC] remove PIPELINE_INIT_FAILURE event.

welldone

Tue, Jun 8, 12:51 AM
alangenfeld created Image Macro "welldone".
Tue, Jun 8, 12:51 AM

Mon, Jun 7

alangenfeld added a comment to D8225: Move solid error boundary closer to user code invocation to clean up stack traces.

I understand this is a bit of a spicy change, but I thought I would at least put it out there.

Mon, Jun 7, 4:20 PM
alangenfeld updated the diff for D8245: expose retry attempts on SoildExecutionContext.

see how retry_attempt_number feels

Mon, Jun 7, 4:18 PM

Fri, Jun 4

alangenfeld closed D8257: EnvironmentConfig -> ResolvedRunConfig.
Fri, Jun 4, 10:10 PM
alangenfeld committed R1:53e4005ab82e: EnvironmentConfig -> ResolvedRunConfig (authored by alangenfeld).
EnvironmentConfig -> ResolvedRunConfig
Fri, Jun 4, 10:10 PM
alangenfeld closed D8256: environment -> run_config cleanups.
Fri, Jun 4, 10:09 PM
alangenfeld committed R1:38df6afa4048: environment -> run_config cleanups (authored by alangenfeld).
environment -> run_config cleanups
Fri, Jun 4, 10:09 PM
alangenfeld requested review of D8257: EnvironmentConfig -> ResolvedRunConfig.
Fri, Jun 4, 9:25 PM