HomeElementl

Use tz_localize to get naive local timestamp

Description

Use tz_localize to get naive local timestamp

Summary:
Pandas 1.3.0 introduces a bunch of changes and deprecations around how
timestamps are handled. With the change, calling
Timestamp.min.replace() or Timestamp.max.replace() causes out of
bounds errors:

pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1677-09-21 00:12:43

This is presumably the same kind of error that this code is trying to
avoid - by converting min and max timestamps to something slightly
inside the true bounds.

It seems that Timestamp.min.replace(tzinfo=None) and
Timestamp.min.tz_localize(None) are semantically equivalent - and as
an added bonus, the latter is more performant and doesn't hit our out of
bounds errors:

https://stackoverflow.com/a/34687479

I believe the risk of this causing issues is low - if I understand the
surrounding code correctly, it only matters when trying to validate
DataFrames that have Timestamps that are very close to the actual
min/max valid Timestamp.

Test Plan: unit

Reviewers: alangenfeld, catherinewu

Reviewed By: alangenfeld

Differential Revision: https://dagster.phacility.com/D8694

Details

Provenance
jordansandersAuthored on Jul 2 2021, 8:55 PM
Reviewer
alangenfeld
Differential Revision
D8694: Use tz_localize to get naive local timestamp
Parents
R1:825836167c3e: event log / instance api for event_records
Branches
Unknown
Tags
Unknown