I like SensorEvaluationContext and ScheduleEvaluationContext too.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed All Stories
All Stories
All Stories
Today
Today
dgibson requested changes to D8211: feat: retrieve partition set names from external partition set in schedule based partitions.
overall this makes sense to me - tweak some names and remove some extra timezone fields and i think we're good
sandyryza committed R1:6f0520630762: [crag] a set of pipelines and schedules to prototype cragit UI on top of (authored by sandyryza).
[crag] a set of pipelines and schedules to prototype cragit UI on top of
sandyryza added inline comments to D8356: don't require context arg when solid has required resources.
[crag] job name
yuhan committed R1:99c6d4e27dc1: monitor sensor 4/[dagit] show origin runs on the sensor's page (authored by yuhan).
monitor sensor 4/[dagit] show origin runs on the sensor's page
requesting changes for q management
Yesterday
Yesterday
yuhan added inline comments to D7895: monitor sensor 4/[dagit] show origin runs on the sensor's page.
up
Ideally we'll just lift the valuable things off those pages onto the Job overview.
yuhan requested review of D8365: correctly process configs of io managers on solids inside composite.
alangenfeld added inline comments to D8356: don't require context arg when solid has required resources.
Yea this makes sense to me. The step launcher case already made me a bit uneasy.
This looks great! I think the iconography + colors are nice and I'm a fan of not having the icons click through to the sensor / schedule pages. Ideally we'll just lift the valuable things off those pages onto the Job overview.
move _check_io_managers_on_composite_solid to _CompositeSolid so we error as soon as the composite solid is defined
mark to_job as experimental
In D8362#218480, @prha wrote:+1 on moving way from ExecutionContext.
I think SensorContext is fine... did you also consider SensorEvaluationContext/ ScheduleEvaluationContext? I think we've been pretty good in the docs about "evaluating" sensors & schedules and "executing" pipelines.
+1 on moving way from ExecutionContext.
GitHub <noreply@github.com> committed R1:1582b6e99ae2: Fix pyspark_resource example (#4290) (authored by Andrew-Crosby <37139900+Andrew-Crosby@users.noreply.github.com>).
Fix pyspark_resource example (#4290)
dish committed R1:e7997d675c43: [dagit] More optimistic switch behavior for schedules/sensors (authored by dish).
[dagit] More optimistic switch behavior for schedules/sensors
rexledesma committed R1:a9bf380fa399: feat: add flag to disable all warnings when running dagit (authored by rexledesma).
feat: add flag to disable all warnings when running dagit
alangenfeld added inline comments to D8272: error when solids inside composite have root_manager_key.
should add a test for lazy job construction with partition too
Harbormaster failed remote builds in B32042: Diff 39477 for D8275: Make solid invocation directly call the underlying decorated fxn!
Harbormaster failed remote builds in B32041: Diff 39475 for D8275: Make solid invocation directly call the underlying decorated fxn!
Harbormaster failed remote builds in B32040: Diff 39474 for D8275: Make solid invocation directly call the underlying decorated fxn!
cdecarolis updated the diff for D8275: Make solid invocation directly call the underlying decorated fxn.
Up
prha committed R1:288c4c072ea4: add option to skip uploading empty files to improve perf with minio (authored by prha).
add option to skip uploading empty files to improve perf with minio
cdecarolis updated the diff for D8275: Make solid invocation directly call the underlying decorated fxn.
up
cdecarolis updated the diff for D8275: Make solid invocation directly call the underlying decorated fxn.
Rearrange implementation around flag for decorators requiring context arguments, fix tests
works for me
GitHub <noreply@github.com> committed R1:e646625a687d: docs clean up 0.11.13rcx (#4288) (authored by yuhan).
docs clean up 0.11.13rcx (#4288)
Sat, Jun 12
Sat, Jun 12
remove 0.11.14 RCs from docs
elementl-devtools committed R1:2df4b60f29a5: Automation: versioned docs for 0.11.14rc7 (authored by elementl-devtools).
Automation: versioned docs for 0.11.14rc7
Fri, Jun 11
Fri, Jun 11
prha requested review of D8354: add option to skip uploading empty files to improve perf with minio.
sandyryza committed R1:632895150a62: make input and output definition metadata unexperimental (authored by sandyryza).
make input and output definition metadata unexperimental
sandyryza updated the diff for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.
up
up
error on root_manager_key and io_manager_key on composite solids
sandyryza retitled D8283: [crag] partitions on modes from [RFC] [crag] partitions on modes to [crag] partitions on modes.
elementl-devtools committed R1:e71da6bcea13: Automation: versioned docs for 0.11.14rc5 (authored by elementl-devtools).
Automation: versioned docs for 0.11.14rc5
Harbormaster failed remote builds in B32015: Diff 39443 for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of!
I think the ideal behavior is to ban io_manager_key and root_input_manager on @composite_solid but allow these to be set on the inner @solids (which is quite the opposite of this diff). reasons are:
- users can still specify io managers for solids inside composite - imo composite is a way to better organizing solids but limit existing functionalities of its inner solids.
- we can keep solids' reusability - a solid that requires custom io managers can still be included in a composite.
- the dagster machinery can still treat composite as a container - meaning a composite has nothing to do with how io works and it would just defer to its inner solids' set up.
elementl-devtools committed R1:d8b0055fde57: Automation: versioned docs for 0.11.14rc4 (authored by elementl-devtools).
Automation: versioned docs for 0.11.14rc4
sandyryza updated the diff for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of.
up
I had a conversation with @schrockn and I am receptive to his argument that it's more elegant for jobs to have names than suffixes. Here's a re-spin that does that.
dgibson committed R1:7c9a752e9a82: Be less strict in dagster image build CLI about the environment matching the… (authored by dgibson).
Be less strict in dagster image build CLI about the environment matching the…
lgtm!
elementl-devtools committed R1:967a1bcd5036: Automation: versioned docs for 0.11.14rc3 (authored by elementl-devtools).
Automation: versioned docs for 0.11.14rc3
+sandy
IMO if the purpose to support "pipeline_name: mode_name" in dagit displays we could do that at the dagit/graphql layer exclusively rather than inject the notion of a suffix into the greenfield job concept
It seems odd to provide this and not just a name param
Harbormaster failed remote builds in B31996: Diff 39420 for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of!
test, lint, mypy
Harbormaster failed remote builds in B31994: Diff 39418 for D8347: [crag] a set of pipelines and schedules to prototype cragit UI on top of!
[dagit] Upgrade to Typescript 4.3.2
[dagit] Clean up package.jsons