Page MenuHomePhabricator

alangenfeld (Alex Langenfeld)
UserAdministrator

Projects

User does not belong to any projects.

User Details

User Since
Mar 20 2019, 8:25 PM (72 w, 6 d)
Roles
Administrator

Recent Activity

Wed, Aug 5

alangenfeld added a comment to D4074: rm gevent from dagster.

if someone could commandeer this that would be great πŸ˜„

Wed, Aug 5, 5:36 PM
alangenfeld added a reviewer for D4065: update protobuf to 3.12.4: nate.
Wed, Aug 5, 5:33 PM
alangenfeld updated the summary of D4065: update protobuf to 3.12.4.
Wed, Aug 5, 5:33 PM
alangenfeld published D4065: update protobuf to 3.12.4 for review.
Wed, Aug 5, 5:33 PM
alangenfeld published D4074: rm gevent from dagster for review.
Wed, Aug 5, 5:32 PM
alangenfeld committed R1:56a07d071966: remove flytekit pin (authored by alangenfeld).
remove flytekit pin
Wed, Aug 5, 5:31 PM
alangenfeld closed D4063: remove flytekit pin.
Wed, Aug 5, 5:31 PM

Mon, Aug 3

alangenfeld added a comment to D4041: k8s user code deployments.

I think the last thing we need to figure out is how to toggle between the new set up and the old set up - assuming we don't want to do a hard cut-over immediately

Mon, Aug 3, 11:44 PM
alangenfeld requested changes to D4061: Move launch_pipeline CLI commands to not run any user code.

can you split out the execute pipeline change to its own diff? would be good to land the launch pipeline fix ASAP

Mon, Aug 3, 10:04 PM
alangenfeld added a comment to D4061: Move launch_pipeline CLI commands to not run any user code.

ya in my head dagster pipeline execute is for "just execute this pipeline right here in my terminal so i can debug it / see what happens", which might not be *right* but is what gives me pause here

Mon, Aug 3, 9:36 PM
alangenfeld added a comment to D4061: Move launch_pipeline CLI commands to not run any user code.

curious if @schrockn has thoughts here

Mon, Aug 3, 9:17 PM
alangenfeld added a comment to D4061: Move launch_pipeline CLI commands to not run any user code.

hmm dagster pipeline execute firing a pipeline execution in an un-managed grpc server is a bit worrisome. In our deployed cases the assumption is the grpc server acts as metadata-reads-only and all executions happen via run launcher. I'm not certain that dagster pipeline execute shouldn't boot up in a user process context directly and execute right there. I guess its a little weird if you are choosing out of a workspace that declares a different python env for that pipeline

Mon, Aug 3, 9:16 PM
alangenfeld accepted D4058: Make schedule launch command work with grpc server origins.

error message not your fault but worth improving

Mon, Aug 3, 8:54 PM
alangenfeld accepted D4046: Move dagster-celery-tests functions to utils and pipelines to repo.py.

movingshitaround

Mon, Aug 3, 8:37 PM
alangenfeld committed R1:83b16402e964: narrow tqdm pin (authored by alangenfeld).
narrow tqdm pin
Mon, Aug 3, 5:29 PM
alangenfeld accepted D4055: remove deprecated "config" args that have been replaced with "config_schema".

seadra

Mon, Aug 3, 4:32 PM
alangenfeld committed R1:9396d6daa24b: remove google bigquery pin (authored by alangenfeld).
remove google bigquery pin
Mon, Aug 3, 4:19 PM
alangenfeld closed D4064: remove google bigquery pin.
Mon, Aug 3, 4:19 PM
alangenfeld committed R1:197267271e07: tqdm pin to fix lint (authored by alangenfeld).
tqdm pin to fix lint
Mon, Aug 3, 4:19 PM
alangenfeld closed D4066: tqdm pin to fix lint.
Mon, Aug 3, 4:18 PM
alangenfeld requested review of D4063: remove flytekit pin.

flake failure

Mon, Aug 3, 4:12 PM
alangenfeld added reviewers for D4066: tqdm pin to fix lint: max, nate.
Mon, Aug 3, 4:11 PM
alangenfeld requested review of D4066: tqdm pin to fix lint.

^ passed

Mon, Aug 3, 4:10 PM
alangenfeld requested review of D4064: remove google bigquery pin.
Mon, Aug 3, 3:56 PM

Fri, Jul 31

alangenfeld committed R1:c5f3ec31a98c: delete toys out of legacy_examples (authored by alangenfeld).
delete toys out of legacy_examples
Fri, Jul 31, 9:45 PM
alangenfeld closed D4057: delete toys out of legacy_examples.
Fri, Jul 31, 9:44 PM
alangenfeld requested review of D4057: delete toys out of legacy_examples.
Fri, Jul 31, 8:48 PM
alangenfeld added inline comments to D3977: solid-hook-1 @hook and HookDefinition.
Fri, Jul 31, 8:27 PM
alangenfeld committed R1:f04894c412b8: flytkit pin (authored by alangenfeld).
flytkit pin
Fri, Jul 31, 8:22 PM
alangenfeld closed D4059: flytkit pin.
Fri, Jul 31, 8:22 PM
alangenfeld requested review of D4059: flytkit pin.

bk failure was from flakey test

Fri, Jul 31, 8:20 PM
alangenfeld accepted D4029: Add new params to get_external_schedule_execution_data to support using it in the scheduler.
Fri, Jul 31, 5:47 PM
alangenfeld accepted D4031: Remove places in scheduler execution where we call user code.
Fri, Jul 31, 5:46 PM
alangenfeld added a comment to D4053: disable flaky retry test on py38.

I wonder if there are any bugfixes in the more recent release of 3.8 - maybe worth updating the minor version of that we are pinned to

Fri, Jul 31, 5:39 PM
alangenfeld added inline comments to D4053: disable flaky retry test on py38.
Fri, Jul 31, 5:37 PM
alangenfeld added inline comments to D4043: Attempt to figure out why py38-sqlite_instance_hosted_user_process_env is consistently hanging on BK runs (no plans to land).
Fri, Jul 31, 4:37 PM

Thu, Jul 30

alangenfeld added inline comments to D4031: Remove places in scheduler execution where we call user code.
Thu, Jul 30, 10:01 PM
alangenfeld added inline comments to D4029: Add new params to get_external_schedule_execution_data to support using it in the scheduler.
Thu, Jul 30, 8:16 PM
alangenfeld accepted D4037: Remove dagit settings from instance.

cleanup

Thu, Jul 30, 8:09 PM
alangenfeld accepted D4048: Use working directory argument in PythonFileTarget / cli args.

add a note to CHANGES.md breaking changes section

Thu, Jul 30, 7:00 PM

Wed, Jul 29

alangenfeld accepted D4015: Make flag to turn on managed grpc server a property of DagsterInstance rather than loaded via yaml.
Wed, Jul 29, 8:58 PM
alangenfeld accepted D3983: [run-repository-2] hide re-execution button based on the currently loaded repository.

thumbsup

Wed, Jul 29, 8:41 PM
alangenfeld accepted D4003: Persistent GRPC run launcher.
Wed, Jul 29, 7:49 PM
alangenfeld added a comment to D4003: Persistent GRPC run launcher.

Macro limecat:  PROCEED

Wed, Jul 29, 7:49 PM
alangenfeld accepted D4032: replace grapqhl in raw k8s run launcher.

coolbeans

Wed, Jul 29, 7:47 PM
alangenfeld added inline comments to D4015: Make flag to turn on managed grpc server a property of DagsterInstance rather than loaded via yaml.
Wed, Jul 29, 5:22 PM
alangenfeld added a comment to D4003: Persistent GRPC run launcher.

LocalRunLauncher is a good name

Wed, Jul 29, 5:16 PM
alangenfeld accepted D4025: EventMetadataEntry: set defaults for some types of entry_data.

illallowit

Wed, Jul 29, 5:08 PM
alangenfeld added inline comments to D4015: Make flag to turn on managed grpc server a property of DagsterInstance rather than loaded via yaml.
Wed, Jul 29, 5:07 PM
alangenfeld added inline comments to D4015: Make flag to turn on managed grpc server a property of DagsterInstance rather than loaded via yaml.
Wed, Jul 29, 5:06 PM
alangenfeld committed R1:3d7a048c518c: fix issue with skipable output results 2 (authored by alangenfeld).
fix issue with skipable output results 2
Wed, Jul 29, 5:02 PM
alangenfeld closed D4026: fix issue with skipable output results 2.
Wed, Jul 29, 5:02 PM
alangenfeld added a comment to D4015: Make flag to turn on managed grpc server a property of DagsterInstance rather than loaded via yaml.

we can drop dagit_settings but can't break telemtry_settings

Wed, Jul 29, 4:19 PM
alangenfeld accepted D3871: step execution cli command.
Wed, Jul 29, 3:59 PM
alangenfeld added inline comments to D4003: Persistent GRPC run launcher.
Wed, Jul 29, 3:54 PM
alangenfeld accepted D4012: consolidate repository location loading logic, remove last vestiges of user code in repository location loader.
Wed, Jul 29, 3:44 PM

Tue, Jul 28

alangenfeld requested review of D4026: fix issue with skipable output results 2.
Tue, Jul 28, 10:22 PM
alangenfeld accepted D3648: Add execute run API to dagster cli for launcher to stop using dagster-graphql.

maybe hold off on landing this till after the weekly on Thursday. We can pile up all the diffs that move off the dagster-graphql stuff and land them together.

Tue, Jul 28, 8:57 PM
alangenfeld accepted D4011: Make pipeline list command work with grpc repo locations.

slamdunk

Tue, Jul 28, 7:38 PM
alangenfeld added a comment to D4010: Ensure the repository_code_pointer_dict is always keyed by repository name, eliminate RepositoryHandle.repository_key.

coolbeans

Tue, Jul 28, 7:31 PM
alangenfeld accepted D4016: [celery-k8s] schedule job name only once.
Tue, Jul 28, 7:31 PM
alangenfeld accepted D4019: celery docker exec, handle stderr is None.

guess this can't hurt to land

Tue, Jul 28, 7:29 PM
alangenfeld added a comment to D4019: celery docker exec, handle stderr is None.

Future: It’s pretty common to get unexpected values when working with data. We should probably handle unexpected values (like None) in EventMetadataEntry without crashing

Tue, Jul 28, 7:29 PM
alangenfeld accepted D4010: Ensure the repository_code_pointer_dict is always keyed by repository name, eliminate RepositoryHandle.repository_key.

The only place where repository_key and repository_name are ever different is the specific case where you specify an attribute in your load_from

Tue, Jul 28, 7:27 PM
alangenfeld requested changes to D3983: [run-repository-2] hide re-execution button based on the currently loaded repository.

as discussed in person - lets try a pipeline snapshot id resolution approach and see how that goes before mucking with storage schema

Tue, Jul 28, 7:12 PM
alangenfeld committed R1:8c027f26d872: dockerfile tweaks (authored by alangenfeld).
dockerfile tweaks
Tue, Jul 28, 7:12 PM
alangenfeld closed D3996: dockerfile tweaks.
Tue, Jul 28, 7:12 PM
alangenfeld requested changes to D3982: [run-repository-1] add repository_name to pipeline run.
Tue, Jul 28, 7:11 PM
alangenfeld accepted D4009: Separate location handles for 'managed' grpc servers vs. external grpc servers.
Tue, Jul 28, 6:50 PM
alangenfeld accepted D4007: Add k8s scheduler to helm.
Tue, Jul 28, 6:47 PM
alangenfeld resigned from D3848: lakehouse generate asset materializations.

Maybe better for me to retire this change until we have some better entity to attach this metadata to?

Tue, Jul 28, 6:41 PM
alangenfeld updated the diff for D3996: dockerfile tweaks.

up

Tue, Jul 28, 6:40 PM
alangenfeld added a comment to D3996: dockerfile tweaks.

I did some manual tests locally and if any of the files change the docker build context changes and the layer gets re-run so I think its legit - will have to keep an eye out

Tue, Jul 28, 6:38 PM

Mon, Jul 27

alangenfeld added inline comments to D4011: Make pipeline list command work with grpc repo locations.
Mon, Jul 27, 6:30 PM
alangenfeld committed R1:3b1c5f5944e8: pin google-cloud-storage (authored by alangenfeld).
pin google-cloud-storage
Mon, Jul 27, 5:21 PM
alangenfeld closed D4013: pin google-cloud-storage .
Mon, Jul 27, 5:21 PM
alangenfeld retitled D4013: pin google-cloud-storage from remove google-cloud-biquery workaround pin to pin google-cloud-storage .
Mon, Jul 27, 4:54 PM
alangenfeld requested review of D4013: pin google-cloud-storage .
Mon, Jul 27, 4:41 PM

Fri, Jul 24

alangenfeld accepted D4008: [docs] Cover "configured" in configuration overview.

noice

Fri, Jul 24, 8:29 PM
alangenfeld accepted D4005: module doc for simple lakehouse example.

whynot

Fri, Jul 24, 8:27 PM
alangenfeld accepted D3940: k8s cron job scheduler.

yeahfuckyeah

Fri, Jul 24, 8:25 PM
alangenfeld accepted D3991: Don't require code pointer dict in grpc server origin.

whynot

Fri, Jul 24, 8:09 PM
alangenfeld updated subscribers of D3991: Don't require code pointer dict in grpc server origin.
Fri, Jul 24, 8:07 PM
alangenfeld accepted D3990: added utility functions to load config from yaml files, strings, and package resources..

alrightalrightalright

Fri, Jul 24, 7:57 PM
alangenfeld accepted D4006: new pipeline origin for grpc servers.

im guessing this will end up shifting around a bit with regards to recent conversation, but will still be needed for externally managed grpc server

Fri, Jul 24, 7:55 PM
alangenfeld requested changes to D3940: k8s cron job scheduler.

generally worth going over K8sScheduler carefully and trimming it down to just what is actually needed in the k8s context

Fri, Jul 24, 6:31 PM
alangenfeld added inline comments to D3990: added utility functions to load config from yaml files, strings, and package resources..
Fri, Jul 24, 6:17 PM
alangenfeld committed R1:b77510af2b51: [celery docker] add a test (authored by alangenfeld).
[celery docker] add a test
Fri, Jul 24, 6:14 PM
alangenfeld closed D3669: [celery docker] add a test.
Fri, Jul 24, 6:14 PM
alangenfeld updated the diff for D3996: dockerfile tweaks.

up

Fri, Jul 24, 6:05 PM
alangenfeld updated the diff for D3996: dockerfile tweaks.

up

Fri, Jul 24, 5:15 PM
alangenfeld updated the diff for D3669: [celery docker] add a test.

up

Fri, Jul 24, 5:13 PM
alangenfeld updated the diff for D3996: dockerfile tweaks.

up

Fri, Jul 24, 4:33 PM
alangenfeld updated the diff for D3669: [celery docker] add a test.

up

Fri, Jul 24, 4:31 PM
alangenfeld updated the diff for D3669: [celery docker] add a test.

up

Fri, Jul 24, 3:57 PM
alangenfeld added inline comments to D3892: use configured in simple_lakehouse example.
Fri, Jul 24, 3:54 PM
alangenfeld added inline comments to D3954: Support soft run termination over GRPC.
Fri, Jul 24, 3:54 PM
alangenfeld updated the diff for D3996: dockerfile tweaks.

more tweaks

Fri, Jul 24, 3:53 PM
alangenfeld updated the diff for D3669: [celery docker] add a test.

up

Fri, Jul 24, 3:32 PM
alangenfeld accepted D4002: (0.9.0-deprecation-3) Fully deprecate --env flag.

just wait to land til we know which week we are rolling 0.9.0

Fri, Jul 24, 3:09 PM