xarray.backends.CachingFileManager¶
-
class
xarray.backends.
CachingFileManager
(opener, *args, **keywords)¶ Wrapper for automatically opening and closing file objects.
Unlike files, CachingFileManager objects can be safely pickled and passed between processes. They should be explicitly closed to release resources, but a per-process least-recently-used cache for open files ensures that you can safely create arbitrarily large numbers of FileManager objects.
Don’t directly close files acquired from a FileManager. Instead, call FileManager.close(), which ensures that closed files are removed from the cache as well.
Example usage:
manager = FileManager(open, ‘example.txt’, mode=’w’) f = manager.acquire() f.write(…) manager.close() # ensures file is closed
Note that as long as previous files are still cached, acquiring a file multiple times from the same FileManager is essentially free:
f1 = manager.acquire() f2 = manager.acquire() assert f1 is f2
-
__init__
(opener, *args, **keywords)¶ Initialize a FileManager.
The cache and ref_counts arguments exist solely to facilitate dependency injection, and should only be set for tests.
- Parameters
- openercallable
Function that when called like
opener(*args, **kwargs)
returns an open file object. The file object must implement aclose()
method.- *args
Positional arguments for opener. A
mode
argument should be provided as a keyword argument (see below). All arguments must be hashable.- modeoptional
If provided, passed as a keyword argument to
opener
along with**kwargs
.mode='w' `` has special treatment: after the first call it is replaced by ``mode='a'
in all subsequent function to avoid overriding the newly created file.- kwargsdict, optional
Keyword arguments for opener, excluding
mode
. All values must be hashable.- lockduck-compatible threading.Lock, optional
Lock to use when modifying the cache inside acquire() and close(). By default, uses a new threading.Lock() object. If set, this object should be pickleable.
- cacheMutableMapping, optional
Mapping to use as a cache for open files. By default, uses xarray’s global LRU file cache. Because
cache
typically points to a global variable and contains non-picklable file objects, an unpickled FileManager objects will be restored with the default cache.- ref_countsdict, optional
Optional dict to use for keeping track the number of references to the same file.
Methods
__init__
(opener, *args, **keywords)Initialize a FileManager.
acquire
([needs_lock])Acquiring a file object from the manager.
close
([needs_lock])Explicitly close any associated file object (if necessary).
-