Page MenuHomePhabricator

Fix bug with config defaults
ClosedPublic

Authored by sashank on Aug 1 2019, 9:28 PM.

Details

Reviewers
nate
Group Reviewers
Restricted Project
Commits
R1:3d2f3d1911c3: Fix bug with config defaults
Summary

See https://github.com/dagster-io/dagster/issues/1608

Also included small utility function print_schema() to make it easier to debug config_types

Example:

{
  solids: {
    print_value?: {
      outputs?: {
        result?: {
          |–json: {
          |  path: [Path]
          |}
          |–pickle: {
          |  path: [Path]
          |}
        }
      }
    } default={}
    prefix_id: {
      inputs: {
        id: {
          |–value: [ANY]
          |–json: {
          |  path: [Path]
          |}
          |–pickle: {
          |  path: [Path]
          |}
        }
      }
      outputs?: {
        result?: {
          |–json: {
          |  path: [Path]
          |}
          |–pickle: {
          |  path: [Path]
          |}
        }
      }
      config?: {
        prefix?: [String]  default=_id_
      } default={'prefix': '_id_'}
    }
  }
  expectations?: {
    evaluate?: [Bool]  default=True
  } default={'evaluate': True}
  storage?: {
    |–in_memory?: {
    |
    |} default={}
    |–filesystem?: {
    |  config?: {
    |    base_dir?: [String]
    |  } default={}
    |} default={'config': {}}
  }
  execution?: {

  } default={}
  loggers?: {
    console?: {
      config?: {
        log_level?: [String]  default=INFO
        name?: [String]  default=dagster
      } default={'log_level': 'INFO', 'name': 'dagster'}
    }
  } default={}
  resources?: {

  } default={}
}
Test Plan

unit

Diff Detail

Repository
R1 dagster
Branch
fix-evaluation-defaults
Lint
Lint OK
Unit
No Unit Test Coverage

Event Timeline

sashank created this revision.Aug 1 2019, 9:28 PM
sashank edited the summary of this revision. (Show Details)Aug 1 2019, 9:29 PM
sashank updated this revision to Diff 3365.Aug 1 2019, 10:05 PM

Including print_schema

sashank edited the summary of this revision. (Show Details)Aug 1 2019, 10:13 PM
sashank updated this revision to Diff 3366.Aug 1 2019, 10:16 PM

Change – to - due to encoding error

sashank updated this revision to Diff 3367.Aug 1 2019, 10:24 PM

use .format

alangenfeld added inline comments.
python_modules/dagster/dagster/core/types/evaluator/evaluation.py
251

this was ultimately all that was needed to fix?

python_modules/dagster/dagster/core/types/field_utils.py
202

call debug_str since it doesn't print it self

sashank updated this revision to Diff 3368.Aug 1 2019, 11:07 PM

Move tests and rename to debug_str

sashank updated this revision to Diff 3371.Aug 1 2019, 11:27 PM

ANY -> Any

nate accepted this revision.Aug 1 2019, 11:29 PM
nate added a subscriber: nate.
nate added inline comments.
python_modules/dagster/dagster/core/types/field_utils.py
202

thanks for implementing this!

This revision is now accepted and ready to land.Aug 1 2019, 11:29 PM
This revision was automatically updated to reflect the committed changes.