up pickle protocol
Summary:
User was getting "OSError: [Errno 22] Invalid argument" when the default intermediate storage called pickle.dump with PICKLE_PROTOCOL = 2 which can happen for very large objects. protocol 4 adds support for pickling very large objects: https://docs.python.org/3/library/pickle.html#pickle-protocols
there are potential issues if the pickle writer gets ahead of the pickle reader. I think there are some rare edge cases where this is unsafe, such as if you use python2 to re-execute a pipeline run that was originally executed in python3
separately "OSError: [Errno 22] Invalid argument" from the default intermediate storage is a pretty confusing error and it's not clear for the user what they should do next (set up a diff intermediate storage)
Test Plan: bk
Reviewers: alangenfeld, max, sandyryza, yuhan
Reviewed By: yuhan
Differential Revision: https://dagster.phacility.com/D4937