Yeah I think with this one might need some special rule for "logger resources" that they are automatically added to every solid in an execution

Yeah I think with this one might need some special rule for "logger resources" that they are automatically added to every solid in an execution

@schrockn that brings up the question of whether we'd want to make that a generic resource capability. I'm sure there are some users who would love to have a global config resource that doesn't require setting required_resource_keys on each solid that wants to access it.

I'm a bit more wary of allowing users to do that. This is "special" since it is a "system" resource that is not directly reference by a context. (i.e., the user always goes through the intermediate logging API rather than direct against the resource)

I'm a bit more wary of allowing users to do that. This is "special" since it is a "system" resource that is not directly reference by a context. (i.e., the user always goes through the intermediate logging API rather than direct against the resource)

Backing up, my understanding of the requirement is essentially that we want someone to be able to set a logging policy for in-process logging inside their solids, without needing to require a resource key per solid.

What if someone wanted to be able to do the same for code that executes out-of-process? E.g. maybe all their solids invoke Java programs, and they have some particular Log4J config that they want to apply for production runs of their job. I suspect it would be useful to support that as well.

Which is to say that I think the need we're observing here points to a requirement that's more general than "configure these particular logging libraries", and I'm wary of adding a narrow abstraction if there's a more general one that's likely to subsume it. If we don't want to enable the config use case I described above, perhaps we could offer a class of global resources that run pre/post-step, but that solid bodies don't have access to?