We've gotten feedback from a few different directions that it's difficult to understand what Dagster is/does from our top-level messaging. Here's one example:
I guess the description under that header -- "Pipelines are built from ... data processing and infrastructure components. Dagster's web interface lets anyone inspect these objects and discover how to use them," -- is kind of unclear to me as I don't really understand what these components are? I think I'm confused on how you would have a component that is used to build the pipeline and process data, but then can be used for something else, which is what the last sentence there is kind of implying to me. Like, wouldn't "how you use them" just be to process data?
On a re-read, I definitely sympathize. Here's an attempt to update the three headlines on our website to be more specific.
In particular, I sought to:
- Specifically talk about "pipelines" of "computations" rather than "components".
- Connect each of the headlines to one of (my view of) our core value props: (1) orchestration of diverse compute, (2) testability, (3) linking assets with compute.
Maybe this is already slated for bulldozing as part of a larger messaging push?