HomePhabricator

add markdown descriptions for dagster pandas dataframe types

Authored by themissinghlink on Jan 2 2020, 6:50 PM.

Description

add markdown descriptions for dagster pandas dataframe types

Summary:
This revision is tackling the addition of markdown descriptions for pandas dataframe types to enable useful understanding of data in dagit.

More specifically, this revision attempts to implement cases 1-3 of the following spec: https://hackmd.io/pqpg7dy9RjKlNa6CU8niVQ.

I am not handling 4 because that requires some changes to the base column api and I don't want to complicate this revision too much.

changelog

  • Added markdown description generation and hooked it into create_dagster_pandas_dataframe_type.
  • Added unit tests for create_dagster_pandas_dataframe_type description generation.
  • Established the concept of error_descriptions and markdown_descriptions so that error messaging during development was decoupled from the markdown syntax requirements in dagit.

Proof this works

Test Plan: unit

Reviewers: alangenfeld, max, schrockn

Reviewed By: alangenfeld

Differential Revision: https://dagster.phacility.com/D1753

Details