Page MenuHomeElementl

Don't pass a dir to our TemporaryDirectory

Authored by jordansanders on Apr 16 2021, 5:09 PM.



In response to:

The step succeeded when retried. I was unable to reproduce the same
failure locally, but I'm incidentally able to produce a different

FileNotFoundError: [Errno 2] No such file or directory: 'user_in_loop/tmpzayi1d_n'

In short, I'm unable to specify a dir that doesn't yet exist. Creating
the TemporaryDirectory works just fine if I pass an existing directory.

Effectively, I'm running into:

>>> import os
>>> os.mkdir("foo/bar")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'foo/bar'
>>> os.mkdir("foo")
>>> os.mkdir("foo/bar")

This is because mktmp doesn't set parents=True when it calls mkdir:

So on a FileNotFoundError, it raises instead of recursively building
the parents:

We don't run into this when running via tox because tox is changing our
working directory to /workdir/examples/user_in_loop so the parent
already exists.

This is the only place in our codebase where we're explicitly specifying
a dir when creating a TemporaryDirectory and it doesn't seem to be
providing any actual value to the test so I'm going to remove the dir
argument. That way, we can invoke this test locally without running it
through tox.

Test Plan


Diff Detail

R1 dagster
Lint Not Applicable
Tests Not Applicable

Event Timeline


Incidentally, I kind of prefer the pytest tmpdir fixture for tests instead of using the standard library's tempfile because I think it can help guard against some of the common tempfile gotchas:

But I want to narrowly scope this diff to just the particular issue I was running into.

This revision is now accepted and ready to land.Apr 16 2021, 5:36 PM