Introduces @repository as shorthand for constructing RepositoryDefinitions directly.
ima file a starter task to break this big file up in to directory
I think this is an interesting decision - in my head the way this would work is we would take list of heterogenous types: PipelineDefinition, ScheduleDefinition, PartitionSetDefinition, and tuple[str, Callable[, PipelineDefinition]]
but i could see merits to being less dynamic / magical. Curious to hear yalls thoughts
So building on my inline comment. I would suggest two versions of this API.
One the "starter" version, where the repository returns an array of definitions.
Then an "advanced" version that returns a more structured object that is designed for lazy evaluation at definition granularity. Alternatively this could be a wholly separate top-level decorator. However this would return an object like a DefinitionFactory which has an interface with get_pipeline(), get_schedule(), and so forth, providing maximal flexibility for people who want to do stuff like build definitions on demand that are generated from DSLs (e.g. like yaml-driven pipeline definitions).
I think it's a bit strange for the repository decorator to take partition_set_defs and schedule_defs as arguments, but return pipeline defs. This will almost certainly expand to include standalone solid definitions in the future and the who knows?
IMO, one of the goals of this new API should be potentially 100% deferred evaluation. Imagine a huge monorepo with dozens or hundreds of repos.
I would recommend having the easy version of this API return an array of defs. I think we should have the optionality of making the evaluation of that function lazy, and maybe even do that now.