Page MenuHomePhabricator

Add metadata entries to DagsterEvent
ClosedPublic

Authored by sashank on Jul 24 2019, 2:59 AM.

Details

Reviewers
max
Group Reviewers
Restricted Project
Commits
R1:22e127e12c33: Add metadata entries to DagsterEvent
Summary

Pull up any metadata entries from fields in DagsterEvent.event_specific_data into DagsterEvent.metadata_entries, and expose the metadata entries in GrqphQL directly on DagsterEvent

Test Plan

unit

Example Queries:

Here, the expectationResult metadata_entries can be accessed under expectationResult, or directly on the StepExpectationResultEvent

{
  pipelineRuns {
    logs {
      nodes {
        ... on StepExpectationResultEvent {
          expectationResult{
            metadataEntries{
              ...on EventJsonMetadataEntry{
                jsonString
              }
            }
          }
          metadataEntries {
            label
            description
            ... on EventJsonMetadataEntry {
              jsonString
            }
          }
        }
      }
    }
  }
}

Here, PipeProcessStartedEvent has a field named process_id on event_specific_data. We can move that into the top level metadata_entries, and query it using:

{
  pipelineRuns {
    logs {
      nodes {
        ... on PipelineProcessStartedEvent {
          processId
          metadataEntries {
            label
            ... on EventTextMetadataEntry {
              text
            }
          }
        }
      }
    }
  }
}

Response:

...
   {
      "processId": 36294,
      "metadataEntries": [
	{
	  "label": "process_id",
	  "text": "36294"
	}
      ]
    },
...

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

sashank updated this revision to Diff 3116.Jul 24 2019, 2:59 AM
sashank created this revision.

Remove run_id from PIPELINE_PROCESS_STARTED metadata_entries

max requested changes to this revision.Jul 24 2019, 5:40 PM
max added a subscriber: max.

update snapshot and revert debug [1] value then gtg

python_modules/dagster-graphql/dagster_graphql/schema/runs.py
340

this looks like a debug change which should be reverted

535

any reason not to just collapse {'metadata_entries_: dagster_event.metadata_entries} into basic_params?

This revision now requires changes to proceed.Jul 24 2019, 5:40 PM
sashank updated this revision to Diff 3118.Jul 24 2019, 6:04 PM
sashank marked 2 inline comments as done.

Feedback and rename process_id -> processId

sashank updated this revision to Diff 3120.Jul 24 2019, 6:09 PM

run generate-types

max accepted this revision.Jul 24 2019, 6:37 PM
This revision is now accepted and ready to land.Jul 24 2019, 6:37 PM