Add a RepositoryLocationOrigin class that separates out the loading…

Unpublished Commit · Learn More

Not On Permanent Ref: This commit is not an ancestor of any permanent ref.
This commit has been deleted in the repository: it is no longer reachable from any branch, tag, or ref.


Add a RepositoryLocationOrigin class that separates out the loading instructions for a repo location from the actual repository data that it loads

The high level goal here is to support dagit being able to better handle failures while loading repository locations.

Right now, if there's an issue loading a repo location, we don't have a great way to keep the parts of the repo location that could be used to reload it - most of the repo location handles, for example, assume that you have already successfully loaded the code pointers. This diff adds a separate RepositoryLocationOrigin class hierarchy that just captures the origin of the repo location (much like RepositoryOrigin or PipelineOrigin) - this is all information that we can count on being there before we load any user code. Now, the workspace load path just loads a list of these origins, then passes them into the Workspace (which creates handles from them, doing all the loading then). This paves the way for us to be able to reload repo locations that never loaded successfully in the first place.

As part of this work, we also need to make it so that default repo location names don't depend on the load being successful (otherwise there would be no name for repo locations that fail to load).

This diff doesn't have any actual functionality changes (other than the repo location name changes) - it just splits out handles from origins to lay groundwork for making those changes next.

Test Plan: BK

Reviewers: alangenfeld, sashank, prha, schrockn

Differential Revision: https://dagster.phacility.com/D4916


dgibsonAuthored on Oct 26 2020, 8:38 PM
R1:180e1aa8525e: Differentiate between ModuleCodePoitner and PackageCodePointer on the gRPC…

Event Timeline