Page MenuHomePhabricator

Fix weather dataset config and validation runner code so that code works and tests catch future bugs.

Authored by themissinghlink on Nov 26 2019, 12:35 AM.




The config has indeterminate ordering because it's a dictionary. It turns out that my example is failing validation for a different reason (datetime validation was too strict -M8 and datetime64[ns]-) but because I am just checking to see if the type checking fails, the test "succeeds" before we can actually validate the bucket. It just so happened that the test validated the columns in a different order which surfaced the bug!

Solution: Fixed the datetime validation failure. Fixed the offending category code which was masked.

Test Plan


Diff Detail

R1 dagster
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

what was the actual bug here and why was it 35 only?

schrockn requested changes to this revision.Nov 26 2019, 12:41 AM

Update diff summary with whatever is being typed in slack right now

This revision now requires changes to proceed.Nov 26 2019, 12:41 AM
themissinghlink edited the summary of this revision. (Show Details)Nov 26 2019, 12:44 AM
schrockn accepted this revision.Nov 26 2019, 12:44 AM
This revision is now accepted and ready to land.Nov 26 2019, 12:44 AM