Page MenuHomePhabricator

Add an example of how to trigger a pipeline by graphQL
AbandonedPublic

Authored by yichendai on Jan 13 2021, 6:19 AM.

Details

Summary

Use a script to kick off a pipeline run.

Test Plan

Added test_trigger_pipleline.py

Diff Detail

Repository
R1 dagster
Branch
gql
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 6:40 AM
Harbormaster failed remote builds in B24232: Diff 29480!

Add test of testing executing pipeline by graphql

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 14 2021, 3:40 AM
Harbormaster failed remote builds in B24363: Diff 29653!
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 14 2021, 5:22 AM
Harbormaster failed remote builds in B24365: Diff 29656!
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 14 2021, 5:45 AM
Harbormaster failed remote builds in B24366: Diff 29657!

Change relative import path to abs path

Rename to dev-requirements.txt

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 14 2021, 6:57 AM
Harbormaster failed remote builds in B24371: Diff 29662!
yichendai added reviewers: catherinewu, max.
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 14 2021, 9:08 PM
Harbormaster failed remote builds in B24446: Diff 29759!
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 15 2021, 12:11 AM
Harbormaster failed remote builds in B24472: Diff 29795!
Harbormaster returned this revision to the author for changes because remote builds failed.Jan 15 2021, 1:05 AM
Harbormaster failed remote builds in B24483: Diff 29810!

Use abs path for import statements

examples/trigger_pipeline/tests/test_trigger_pipeline.py
9

what are some ways that the trigger.py script could break that this test suite is not currently testing?

Add docs content. Add executionMetadata to pipeline run and modify tests

  1. This is the commit message #5:

Use a public workspace attribute

yichendai retitled this revision from [Draft] Add an example of how to trigger a pipeline by graphQL to Add an example of how to trigger a pipeline by graphQL.Jan 19 2021, 6:28 PM
examples/trigger_pipeline/tests/test_trigger_pipeline.py
9

May need to check if the workspace is empty and if the file path exists.

Improve test coverage line by line

Are we exposing or documenting stuff under dagster.core.host_representation like ManagedGrpcPythonEnvRepositoryLocationOrigin, LoadableTargetOrigin, and PipelineSelector, which have been mentioned in this example?
Should we? Without linking to detailed API docs or further explanation, it could be a bit hard for users to digest what's going on when using these suggested APIs. cc @dgibson: as you may have more context in this corner of the codebase.

Besides, I dont think we have documented any of the dagster-graphql apis? Shall we start documenting dagster-graphql? - AFAIK we've been treating it as more of an internal-only package

docs/next/src/pages/examples/examples.json
86–110

plz rebase master and make sure you've picked up the latest changes. i think this has been addressed in D5824

Yeah, this is fine in that it works but this is not the right way to run a pipeline over GraphQL. The right way to run a pipeline over graphql is to start a dagit server and to query the server.

yichendai added inline comments.
docs/next/src/pages/examples/examples.json
86–110

I rebased master. D5824 added the correct one but forgot to delete line 103-107.

yichendai marked an inline comment as done.EditedJan 25 2021, 6:06 PM
In D5949#165010, @max wrote:

Yeah, this is fine in that it works but this is not the right way to run a pipeline over GraphQL. The right way to run a pipeline over graphql is to start a dagit server and to query the server.

I'll look at the new solution during our meeting this afternoon

Use python-gql to execute pipelines and add corresponding tests

examples/trigger_pipeline/trigger_pipeline/trigger.py
2

better to have the mutation be explicit

Write the new documentation