Page MenuHomePhabricator

Support for pipeline tags
ClosedPublic

Authored by prha on Wed, Mar 18, 7:45 PM.

Details

Summary

Allows users to specify tags on pipeline definitions.

These will resolve in the following cases:

  1. Loading in the playground view will pre-populate the tag container
  2. Loading partition sets from the preset/config picker will pre-populate the tag container with the union of pipeline tags and partition tags, with partition tags taking precedence
  3. Executing from the CLI will generate runs with the pipeline tags
  4. Executing programmatically using the execute_pipeline api will create a run with the union of pipeline tags and RunConfig tags, with RunConfig tags taking precedence
  5. Scheduled runs (both launched and executed) will have the union of pipeline tags and the schedule tags function, with the schedule tags taking precedence
Test Plan

Added tests for both default and overrides for execute_pipeline and the schedule graphql endpoints

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

prha created this revision.Wed, Mar 18, 7:45 PM

hmmmMmmmmm

I think I lean towards treating these as "base tags" (maybe even naming them as such to be clear) and allowing them to be overridden

prha updated this revision to Diff 10707.Thu, Mar 19, 12:37 AM
  • frontend
  • fix execute pipeline overrides
  • tests
prha edited the summary of this revision. (Show Details)Thu, Mar 19, 12:42 AM
prha edited the test plan for this revision. (Show Details)
prha retitled this revision from RFC: initial support for pipeline tags to Support for pipeline tags.
alangenfeld accepted this revision.Thu, Mar 19, 4:14 PM

maybe get a second opinion - but i think this is very reasonable&intuitive behavior

python_modules/dagster/dagster/core/definitions/decorators.py
843–845

call out here that these tags may be over-written at invocation time

python_modules/dagster/dagster/core/definitions/pipeline.py
88

add docs here too

This revision is now accepted and ready to land.Thu, Mar 19, 4:14 PM
prha updated this revision to Diff 10712.Thu, Mar 19, 4:43 PM
prha edited the summary of this revision. (Show Details)
  • updates, fix tests, add docs
This revision was automatically updated to reflect the committed changes.