Some uses -
- My pipeline includes a step that processes a big dataset and then a step that joins it with a smaller dataset. I want the second step to run every hour, but I only need the first step to run once a day.
- My pipeline has a step for each file in a directory. I want to run that step each time the file changes.
This essentially offers a path to memoization that doesn't require adding new abstractions.