Differential D8684 Diff 40976 docs/content/concepts/logging/loggers.mdx

Changeset View

Standalone View

docs/content/concepts/logging/loggers.mdx

Show First 20 Lines • Show All 165 Lines • ▼ Show 20 Lines	def json_console_logger(init_context):
return logger_		return logger_


@solid		@solid
def hello_logs(context):		def hello_logs(context):
context.log.info("Hello, world!")		context.log.info("Hello, world!")


@pipeline(mode_defs=[ModeDefinition(logger_defs={"my_json_logger": json_console_logger})])		@pipeline(
		mode_defs=[
		ModeDefinition(logger_defs={"my_json_logger": json_console_logger})
		]
		)
def demo_pipeline():		def demo_pipeline():
hello_logs()		hello_logs()
```		```

As you can see, you can specify the logger name in the run config. It also takes a `config` argument, representing the config that users can pass to the logger, for example:		As you can see, you can specify the logger name in the run config. It also takes a `config` argument, representing the config that users can pass to the logger, for example:

```yaml file=/concepts/logging/config_custom_logger.yaml		```yaml file=/concepts/logging/config_custom_logger.yaml
loggers:		loggers:
Show All 24 Lines

If you need to provide config to the initialization of your logger, you can use the <PyObject object="build_init_logger_context" /> function to do so.		If you need to provide config to the initialization of your logger, you can use the <PyObject object="build_init_logger_context" /> function to do so.

```python file=/concepts/logging/custom_logger.py startafter=start_custom_logger_testing_context endbefore=end_custom_logger_testing_context		```python file=/concepts/logging/custom_logger.py startafter=start_custom_logger_testing_context endbefore=end_custom_logger_testing_context
from dagster import build_init_logger_context		from dagster import build_init_logger_context


def test_init_json_console_logger_with_context():		def test_init_json_console_logger_with_context():
logger_ = json_console_logger(build_init_logger_context(logger_config={"name": "my_logger"}))		logger_ = json_console_logger(
		build_init_logger_context(logger_config={"name": "my_logger"})
		)
assert logger_.level == 20		assert logger_.level == 20
assert logger_.name == "my_logger"		assert logger_.name == "my_logger"
```		```

## Patterns		## Patterns

### <Check/> Environment-specific logging using modes		### <Check/> Environment-specific logging using modes

Logging is environment-specific: you don't want messages generated by data scientists' local development loops to be aggregated with production messages; on the other hand, you may find that in production console logging is irrelevant or even counterproductive.		Logging is environment-specific: you don't want messages generated by data scientists' local development loops to be aggregated with production messages; on the other hand, you may find that in production console logging is irrelevant or even counterproductive.

Dagster recognizes this by attaching loggers to [modes](/concepts/modes-resources) so that you can seamlessly switch from, e.g., Cloudwatch logging in production to console logging in development and test, without changing any of your code.		Dagster recognizes this by attaching loggers to [modes](/concepts/modes-resources) so that you can seamlessly switch from, e.g., Cloudwatch logging in production to console logging in development and test, without changing any of your code.

```python file=/concepts/logging/logging_modes.py startafter=start_logging_mode_marker_0 endbefore=end_logging_mode_marker_0		```python file=/concepts/logging/logging_modes.py startafter=start_logging_mode_marker_0 endbefore=end_logging_mode_marker_0
@solid		@solid
def hello_logs(context):		def hello_logs(context):
context.log.info("Hello, world!")		context.log.info("Hello, world!")


@pipeline(		@pipeline(
mode_defs=[		mode_defs=[
ModeDefinition(name="local", logger_defs={"console": colored_console_logger}),		ModeDefinition(
ModeDefinition(name="prod", logger_defs={"cloudwatch": cloudwatch_logger}),		name="local", logger_defs={"console": colored_console_logger}
		),
		ModeDefinition(
		name="prod", logger_defs={"cloudwatch": cloudwatch_logger}
		),
]		]
)		)
def hello_modes():		def hello_modes():
hello_logs()		hello_logs()
```		```

From Dagit, you can switch your pipeline mode to 'prod' and edit config in order to use the new Cloudwatch logger, for example:		From Dagit, you can switch your pipeline mode to 'prod' and edit config in order to use the new Cloudwatch logger, for example:

```yaml file=/concepts/logging/config_modes.yaml		```yaml file=/concepts/logging/config_modes.yaml
loggers:		loggers:
cloudwatch:		cloudwatch:
config:		config:
log_level: ERROR		log_level: ERROR
log_group_name: /my/cool/cloudwatch/log/group		log_group_name: /my/cool/cloudwatch/log/group
log_stream_name: very_good_log_stream		log_stream_name: very_good_log_stream
```		```