HomePhabricator

[dagster] whitelisted namedtuple serdes

Authored by alangenfeld on Aug 16 2019, 5:02 PM.

Description

[dagster] whitelisted namedtuple serdes

Summary:
A scheme for serializing & deserializing dagster event data to and from json.

These are all namedtuples so by default you end up with lists if you just do default json
serialization. Even if you manage to encode them as dicts, you still dont know the classes
to use at deserialization time unless we manually maintain a bunch of code.

To resolve this, this serdes encodes the class name as __class__ then uses a predfined
whitelist to grab the actual class object using the name. I think this scheme prevents this
from being a security problem.

Test Plan: Unit test that round trips everything from the many_events pipeline.

Reviewers: #ft, schrockn

Reviewed By: #ft, schrockn

Subscribers: schrockn, prha

Differential Revision: https://dagster.phacility.com/D835

Details

Committed
alangenfeldMon, Aug 19, 9:25 PM
Reviewer
Restricted Project
Differential Revision
D835: [dagster] whitelisted namedtuple serdes
Parents
R1:c8b42fae28f9: [dagster] remove event_callback from RunConfig
Branches
Unknown
Tags
Unknown