Page MenuHomePhabricator

(run-launcher-handle-aware-2) Add InProcessOrigin (pointer and yaml path). Add to EnvironmentHandle and make it accessible to ExternalRepository
ClosedPublic

Authored by schrockn on Thu, May 21, 2:14 AM.

Details

Summary

This stashes the code pointer and yaml path in
InProcessOrigin, puts that in EnvironmentHandle. Primary goal of this is to be able to thread an
ExternalPipeline down to the run_launcher, and this now contains enough
information for run launcher to know what invoke. The ExternalPipeline
will be able to get what it needs by going up the handle chain

Depends on D3026

Test Plan

BK

Diff Detail

Repository
R1 dagster
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

schrockn created this revision.Thu, May 21, 2:14 AM
schrockn requested review of this revision.Thu, May 21, 2:29 AM
schrockn updated this revision to Diff 14417.Thu, May 21, 5:10 PM
schrockn retitled this revision from (repo-source-1) Add RepositorySource and add it to ExternalRepository to (run-launcher-handle-aware-2) Add pointer (and yaml path) to RepositoryHandle and add it to ExternalRepository.
schrockn edited the summary of this revision. (Show Details)

up

schrockn updated this revision to Diff 14430.Thu, May 21, 6:22 PM
schrockn added reviewers: max, alangenfeld.

up

alangenfeld added inline comments.Thu, May 21, 6:35 PM
python_modules/dagster/dagster/core/host_representation/handle.py
21–28

I think the more correct - and should be basically the same code - is to put it here on EnvironmentHandle as source / origin and to potentially add InProcessOrigin as a wrapper for code_pointer + repository.yaml

alangenfeld accepted this revision.Thu, May 21, 6:38 PM

but its not that much code were shuffling around so ill let you make the call

This revision is now accepted and ready to land.Thu, May 21, 6:38 PM
schrockn updated this revision to Diff 14437.Thu, May 21, 6:57 PM
schrockn retitled this revision from (run-launcher-handle-aware-2) Add pointer (and yaml path) to RepositoryHandle and add it to ExternalRepository to (run-launcher-handle-aware-2) Add InProcessOrigin (pointer and yaml path). Add to EnvironmentHandle and make it accessible to ExternalRepository.
schrockn edited the summary of this revision. (Show Details)

up

alangenfeld added inline comments.Thu, May 21, 6:58 PM
python_modules/dagster/dagster/core/host_representation/handle.py
24–28

this should be short lived legacy_from_yaml maybe more appropriate

This revision was landed with ongoing or failed builds.Thu, May 21, 7:02 PM
This revision was automatically updated to reflect the committed changes.