Page MenuHomeElementl
Feed Advanced Search

Thu, May 13

alangenfeld added inline comments to D7907: 0.11.9 changelog.
Thu, May 13, 8:58 PM
alangenfeld accepted D7907: 0.11.9 changelog.
Thu, May 13, 8:57 PM
alangenfeld accepted D7526: [RFC] Directly invoke solids.

i think this is a good place to start

Thu, May 13, 7:53 PM
alangenfeld closed D7855: [mypy] system_config/objects.py.
Thu, May 13, 3:02 PM
alangenfeld committed R1:aa5d02eee9a2: [mypy] system_config/objects.py (authored by alangenfeld).
[mypy] system_config/objects.py
Thu, May 13, 3:02 PM
alangenfeld added inline comments to D7855: [mypy] system_config/objects.py.
Thu, May 13, 3:02 PM
alangenfeld requested changes to D7526: [RFC] Directly invoke solids.

maybe add a DagsterInvalidInvocationError for when the wrong info is passed?

Thu, May 13, 3:00 PM
alangenfeld committed R1:72d98d17d3b1: docs snapshot (authored by alangenfeld).
docs snapshot
Thu, May 13, 2:54 PM
alangenfeld accepted D7891: Allow dagster/priority tag on definitions.
Thu, May 13, 2:28 PM

Wed, May 12

alangenfeld added inline comments to D7526: [RFC] Directly invoke solids.
Wed, May 12, 9:27 PM
alangenfeld added inline comments to D7526: [RFC] Directly invoke solids.
Wed, May 12, 9:25 PM
alangenfeld accepted D7887: Pip install twice.

thumbsup

Wed, May 12, 6:11 PM
alangenfeld closed D7875: [dagit] fix composites in sidebar.
Wed, May 12, 4:57 PM
alangenfeld committed R1:55b822b40c23: [dagit] fix composites in sidebar (authored by alangenfeld).
[dagit] fix composites in sidebar
Wed, May 12, 4:57 PM
alangenfeld requested review of D7875: [dagit] fix composites in sidebar.
Wed, May 12, 4:10 PM
alangenfeld requested review of D7855: [mypy] system_config/objects.py.
Wed, May 12, 3:32 PM

Tue, May 11

alangenfeld closed D7876: pin click.
Tue, May 11, 11:05 PM
alangenfeld committed R1:dadf6d5c8d16: pin click (authored by alangenfeld).
pin click
Tue, May 11, 11:04 PM
alangenfeld added inline comments to D7876: pin click.
Tue, May 11, 10:26 PM
alangenfeld updated the diff for D7876: pin click.

try again

Tue, May 11, 10:25 PM
alangenfeld published D7876: pin click for review.
Tue, May 11, 10:20 PM
alangenfeld closed D7862: fix sql event log cursors.
Tue, May 11, 10:03 PM
alangenfeld committed R1:7cc340082804: fix sql event log cursors (authored by alangenfeld).
fix sql event log cursors
Tue, May 11, 10:03 PM
alangenfeld added inline comments to D7862: fix sql event log cursors.
Tue, May 11, 10:01 PM
alangenfeld requested review of D7862: fix sql event log cursors.
Tue, May 11, 8:55 PM
alangenfeld accepted D7848: add metadata types for dagster pipeline runs, assets.

these pretty safe to me - hard to imagine how theyll be a compat / maintenance issue

Tue, May 11, 8:55 PM
alangenfeld added a comment to D7526: [RFC] Directly invoke solids.

would be good to have the multi output test have direct invoke right next to execute_pipeline / execute_solid to demonstrate their alignment (or lack thereof)

Tue, May 11, 8:50 PM
alangenfeld added inline comments to D1471: Refactor SQL-based event log storages to use SQLAlchemy.
Tue, May 11, 3:24 PM

Mon, May 10

alangenfeld requested changes to D7526: [RFC] Directly invoke solids.

++ sandy's feedback

  • more tests
  • naming
  • outputs
Mon, May 10, 4:59 PM

Fri, May 7

alangenfeld accepted D7783: new arg of_type to EventLogStorage.get_logs_for_run.
Fri, May 7, 7:55 PM
alangenfeld added a reviewer for D7783: new arg of_type to EventLogStorage.get_logs_for_run: prha.
Fri, May 7, 3:50 PM
alangenfeld accepted D7774: D7767 followup: check run_id in downstream.
Fri, May 7, 3:42 PM
alangenfeld resigned from D7677: [Depends on D7674] Refactor lambda_solid as a simple wrapper around _Solid.

il defer to @sandyryza here on how exactly to frame whether this is deprecated, legacy, or whatever

Fri, May 7, 3:33 PM
alangenfeld accepted D7771: Add a flag for whether resources object contains a generator.
Fri, May 7, 2:55 PM
alangenfeld added inline comments to D7677: [Depends on D7674] Refactor lambda_solid as a simple wrapper around _Solid.
Fri, May 7, 2:25 PM
alangenfeld accepted D7688: [dagit] Make Non-Subscription GraphQL ops use HTTP.

I support landing this to maximize soak time til next release

Fri, May 7, 2:14 PM
alangenfeld committed R1:95999815bd40: Automation: versioned docs for 0.11.8 (authored by alangenfeld).
Automation: versioned docs for 0.11.8
Fri, May 7, 3:53 AM
alangenfeld committed R1:d95a5bfc7c83: Automation: versioned docs for 0.11.8rc5 (authored by alangenfeld).
Automation: versioned docs for 0.11.8rc5
Fri, May 7, 3:11 AM

Thu, May 6

alangenfeld committed R1:fcab472e1a1f: Automation: versioned docs for 0.11.8rc4 (authored by alangenfeld).
Automation: versioned docs for 0.11.8rc4
Thu, May 6, 11:49 PM
alangenfeld closed D7772: 0.11.8 changelog.
Thu, May 6, 7:31 PM
alangenfeld committed R1:2ae96e904e8a: 0.11.8 changelog (authored by alangenfeld).
0.11.8 changelog
Thu, May 6, 7:31 PM
alangenfeld updated the diff for D7772: 0.11.8 changelog.

improve

Thu, May 6, 7:07 PM
alangenfeld updated the diff for D7772: 0.11.8 changelog.

improve

Thu, May 6, 7:03 PM
alangenfeld updated the diff for D7772: 0.11.8 changelog.

improvements

Thu, May 6, 7:02 PM
alangenfeld requested review of D7772: 0.11.8 changelog.
Thu, May 6, 7:00 PM
alangenfeld added a comment to D7476: monitor sensor prototype.

How would this work in a world where we encourage people to print (or whatever) instead of using context.log?

Thu, May 6, 6:51 PM
alangenfeld closed D7768: [dagstermill] fix aliased bug.
Thu, May 6, 6:18 PM
alangenfeld committed R1:08e1760a18af: [dagstermill] fix aliased bug (authored by alangenfeld).
[dagstermill] fix aliased bug
Thu, May 6, 6:17 PM
alangenfeld accepted D7767: add run group tags to backfill jobs.

can resolve my error concerns in a follow up - your call

Thu, May 6, 6:17 PM
alangenfeld added inline comments to D7767: add run group tags to backfill jobs.
Thu, May 6, 6:10 PM
alangenfeld requested review of D7768: [dagstermill] fix aliased bug.
Thu, May 6, 6:09 PM
alangenfeld added inline comments to D7762: [crag] unbound sensors.
Thu, May 6, 6:01 PM
alangenfeld added inline comments to D7767: add run group tags to backfill jobs.
Thu, May 6, 5:53 PM
alangenfeld added a comment to D7767: add run group tags to backfill jobs.

would like @prha to sign off on this too

Thu, May 6, 5:39 PM
alangenfeld added a comment to D7476: monitor sensor prototype.

That seemed much worse than what I think you're proposing, which is broadening what the outputs of sensors are and relaxing the constraints of what they can kick off (which would strictly be a pipeline run)

Thu, May 6, 4:26 PM
alangenfeld closed D7725: required resources fixes.
Thu, May 6, 4:08 PM
alangenfeld committed R1:ed15dbb4aed5: required resources fixes (authored by alangenfeld).
required resources fixes
Thu, May 6, 4:08 PM
alangenfeld added a comment to D7725: required resources fixes.

would love to get this in the release today

Thu, May 6, 3:51 PM
alangenfeld added a comment to D7688: [dagit] Make Non-Subscription GraphQL ops use HTTP.

This is behavior that is due to the difference between websockets & http - the websocket response does not have an 'error' field on the JS side so the components render differently. Not sure if throwing away the error in this case (to conform to the old behavior) makes sense though -- feels like errors are important to pay attention to!

Thu, May 6, 2:54 PM

Wed, May 5

alangenfeld closed D7744: dynamic execution bug-fix.
Wed, May 5, 10:15 PM
alangenfeld committed R1:965fe2bd44ed: dynamic execution bug-fix (authored by alangenfeld).
dynamic execution bug-fix
Wed, May 5, 10:15 PM
alangenfeld added reviewers for D7725: required resources fixes: prha, owen.
Wed, May 5, 10:15 PM
alangenfeld accepted D7674: [Depends on D7670] Allow solid to wrap fxn without context arg.

yeahfuckyeah

Wed, May 5, 10:12 PM
alangenfeld added a comment to D7476: monitor sensor prototype.

I think we should strongly consider making hooks standalone objects on the repository, like sensor defs, rather than live on pipeline definitions.

Wed, May 5, 10:08 PM
alangenfeld requested review of D7744: dynamic execution bug-fix.
Wed, May 5, 7:48 PM

Tue, May 4

alangenfeld planned changes to D7594: Allow the set of default executors to use when none are specified to be customized.
Tue, May 4, 9:31 PM
alangenfeld commandeered D7594: Allow the set of default executors to use when none are specified to be customized.

ill put this in my queue for now accordingly

Tue, May 4, 9:31 PM
alangenfeld updated the diff for D7725: required resources fixes.

clean up

Tue, May 4, 9:30 PM
alangenfeld updated the summary of D7725: required resources fixes.
Tue, May 4, 9:24 PM
alangenfeld updated the summary of D7725: required resources fixes.
Tue, May 4, 9:24 PM
alangenfeld requested review of D7725: required resources fixes.
Tue, May 4, 9:16 PM
alangenfeld accepted D7719: Allow TestEventLogStorage subclasses that cannot wipe storage.

whynot

Tue, May 4, 9:00 PM
alangenfeld accepted D7607: [mypy] resource definition.
Tue, May 4, 8:10 PM
alangenfeld requested changes to D7674: [Depends on D7670] Allow solid to wrap fxn without context arg.

to your queue for docs/error message pass

Tue, May 4, 8:09 PM
alangenfeld added a comment to D7674: [Depends on D7670] Allow solid to wrap fxn without context arg.

_context and context_ are such a minor corner case that im not worried - I think this is basically good to go

Tue, May 4, 8:07 PM
alangenfeld accepted D7596: make build_resources usable as a function..

I think im cool with this once you can get the tests passing

Tue, May 4, 8:06 PM
alangenfeld added inline comments to D7596: make build_resources usable as a function..
Tue, May 4, 7:42 PM
alangenfeld added a comment to D7624: [RFC] Celery-less k8s executor.

would be good to include mypy return types as well

Tue, May 4, 6:57 PM
alangenfeld accepted D7693: only suggest config for missing entries.

zubat

Tue, May 4, 3:36 PM

Mon, May 3

alangenfeld added a comment to D7594: Allow the set of default executors to use when none are specified to be customized.

the big thing you lose there is config playground typeahead support in dagit right?

Mon, May 3, 9:35 PM
alangenfeld added a comment to D7594: Allow the set of default executors to use when none are specified to be customized.

heres a quick prototype to illustrate the idea from previous comment https://dagster.phacility.com/differential/diff/36671/

Mon, May 3, 9:27 PM
alangenfeld added reviewers for D7594: Allow the set of default executors to use when none are specified to be customized: prha, sandyryza.

@sandyryza brought up an interesting kernel of an idea about just making the execution block of config Permissive - at least from the pipeline perspective.
So you could drop threading the default executors to the snapshot and env configs and let those just have empty executors and default to a Permissive section. Then only at the "real" execution call sites resolve the instance executors and maybe do a separate config validation/processing pass for the execution subsection

Mon, May 3, 7:03 PM
alangenfeld accepted D7670: Refactor the validation step for positional args in decorators.

pikachu

Mon, May 3, 4:22 PM
alangenfeld closed D7691: fix Dict & Set inner_types.
Mon, May 3, 4:21 PM
alangenfeld committed R1:bdb59cf0d9ed: fix Dict & Set inner_types (authored by alangenfeld).
fix Dict & Set inner_types
Mon, May 3, 4:21 PM
alangenfeld requested review of D7691: fix Dict & Set inner_types.
Mon, May 3, 4:09 PM
alangenfeld closed D7673: test solid_selection and manual partition schedule.
Mon, May 3, 4:04 PM
alangenfeld committed R1:95369f809f6d: test solid_selection and manual partition schedule (authored by alangenfeld).
test solid_selection and manual partition schedule
Mon, May 3, 4:04 PM
alangenfeld added a comment to D7594: Allow the set of default executors to use when none are specified to be customized.

One case i thought of that I think communicates the cost of this change well is if we had a function for just doing run config validation against a target - it would now need to take the instance as an argument

check_run_config(my_pipeline, run_config, instance=DagsterInstance.get())

which gets a little hairy if one wanted to put said function in their test suite to ensure their production runs config was expected to be valid. My assumption here is that generally the test environment will not be setup with the same instance. There are a few different ways to deal with this so it's not a hard blocker - just an awkward case that makes me feel like this isn't the cleanest modeling of this complexity.

Mon, May 3, 2:59 PM

Fri, Apr 30

alangenfeld requested changes to D7674: [Depends on D7670] Allow solid to wrap fxn without context arg.

to your queue for test suite

Fri, Apr 30, 4:26 PM
alangenfeld added a comment to D7674: [Depends on D7670] Allow solid to wrap fxn without context arg.

I think it would be useful to add a block of tests that clearly communicate the new setup by having the different permutations all next to each other

Fri, Apr 30, 3:58 PM
alangenfeld added inline comments to D7670: Refactor the validation step for positional args in decorators.
Fri, Apr 30, 3:44 PM
alangenfeld added a comment to D7670: Refactor the validation step for positional args in decorators.

some ideas inline - I think this can use a second pass. Its a little better but not a clear step up imo

Fri, Apr 30, 3:41 PM
alangenfeld added inline comments to D7646: RFC solid RetryPolicy.
Fri, Apr 30, 3:28 PM

Thu, Apr 29

alangenfeld accepted D7671: Fix bug with enum values and config snapshots.
Thu, Apr 29, 11:04 PM
alangenfeld added a comment to D7646: RFC solid RetryPolicy.

open questions / thinking out loud:

Thu, Apr 29, 11:00 PM
alangenfeld requested review of D7673: test solid_selection and manual partition schedule.
Thu, Apr 29, 10:43 PM
alangenfeld requested review of D7646: RFC solid RetryPolicy.
Thu, Apr 29, 10:15 PM
alangenfeld accepted D7655: [mypy] repository_location.py.

marowak

Thu, Apr 29, 9:14 PM
alangenfeld added inline comments to D7642: [1/n] Add exhaustive check for mypy.
Thu, Apr 29, 9:11 PM