mystic.cache module documentation
archive module
klepto archive readers and writers, for functions and data
- _prep_dist(distances)
create a dict of mean and max graphical distance
- Parameters:
distances (array[float]) – indicates the graphical distance
- Returns:
dict(mean=distances.mean(), max=distances.max())
- _read_func(name, keymap=None, type=None)
read function db with name ‘name’
- Parameters:
name (string) – filename of the klepto db
keymap (klepto.keymap) – keymap used for key encoding
type (klepto.archive) – type of klepto archive
- Returns:
klepto db object (has dictionary-like interface)
- _write_func(archive, func, dist, keymap=None)
write stored function and distance information to archive
- Parameters:
archive (klepto.archive) – function archive (output of
_read_func
)func (function) – with interface
y = f(x)
,x
is a list of floatsdist (dict) – distance information
keymap (klepto.keymap) – keymap used for key encoding
- Returns:
None
- class dict_archive(name=None, dict=None, cached=True, **kwds)
Bases:
dict_archive
initialize a dictionary archive
- static __new__(dict_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with an in-memory dictionary archive backend
- classmethod from_frame(dataframe)
- class dir_archive(name=None, dict=None, cached=True, **kwds)
Bases:
dir_archive
initialize a file folder with a synchronized dictionary interface
- Parameters:
dirname (str, default='memo') – path of the archive root directory
serialized (bool, default=True) – save python objects in pickled files
compression (int, default=0) – compression level (0 to 9), 0 is None
permissions (octal, default=0o775) – read/write permission indicator
memmode (str, default=None) – mode, one of
{None, 'r+', 'r', 'w+', 'c'}
memsize (int, default=100) – size (MB) of cache for in-memory compression
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
- static __new__(dir_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a file-folder archive backend
- Parameters:
name (str, default='memo') – path of the archive root directory
dict (dict, default={}) – initial dictionary to seed the archive
cached (bool, default=True) – interact through an in-memory cache
serialized (bool, default=True) – save python objects in pickled files
compression (int, default=0) – compression level (0 to 9), 0 is None
permissions (octal, default=0o775) – read/write permission indicator
memmode (str, default=None) – mode, one of
{None, 'r+', 'r', 'w+', 'c'}
memsize (int, default=100) – size (MB) of cache for in-memory compression
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
- classmethod from_frame(dataframe)
- class file_archive(name=None, dict=None, cached=True, **kwds)
Bases:
file_archive
initialize a file with a synchronized dictionary interface
- Parameters:
- static __new__(file_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a single file archive backend
- Parameters:
name (str, default='memo.pkl') – path of the file archive
dict (dict, default={}) – initial dictionary to seed the archive
cached (bool, default=True) – interact through an in-memory cache
serialized (bool, default=True) – save python objects in pickled file
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
- classmethod from_frame(dataframe)
- get_dist(archive, func, keymap=None)
get the graphical distance of func from data in archive
- Parameters:
archive (klepto.archive) – run archive (output of
read
)func (function) – with interface
y = f(x)
,x
is a list of floatskeymap (klepto.keymap) – keymap used for key encoding
- Returns:
array of floats, graphical distance from func to each point in archive
- class hdf_archive(name=None, dict=None, cached=True, **kwds)
Bases:
hdf_archive
initialize an archive
- static __new__(hdf_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a single hdf5 file archive backend
- Parameters:
name (str, default='memo.hdf5') – path of the file archive
dict (dict, default={}) – initial dictionary to seed the archive
cached (bool, default=True) – interact through an in-memory cache
serialized (bool, default=True) – pickle saved python objects
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
meta (bool, default=False) – store in root metadata (not in dataset)
- classmethod from_frame(dataframe)
- class hdfdir_archive(name=None, dict=None, cached=True, **kwds)
Bases:
hdfdir_archive
initialize an archive
- static __new__(hdfdir_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a hdf5 file-folder archive backend
- Parameters:
name (str, default='memo') – path of the archive root directory
dict (dict, default={}) – initial dictionary to seed the archive
cached (bool, default=True) – interact through an in-memory cache
serialized (bool, default=True) – pickle saved python objects
permissions (octal, default=0o775) – read/write permission indicator
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
meta (bool, default=False) – store in root metadata (not in dataset)
- classmethod from_frame(dataframe)
- class null_archive(name=None, dict=None, cached=True, **kwds)
Bases:
null_archive
initialize a permanently-empty dictionary
- static __new__(null_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a permanently-empty archive backend
- classmethod from_frame(dataframe)
- read(name, keys=None, keymap=None, type=None)
read klepto db with name ‘name’
- Parameters:
name (string) – filename of the klepto db
keys (iterable) – keys to load; or boolean to load all/no keys
keymap (klepto.keymap) – used for key encoding
type (klepto.archive) – type of klepto archive
- Returns:
klepto db object (has dictionary-like interface)
Notes
If keys is None, create a direct handle to the db. If a key in keys is not found, it will be ignored.
- read_func(name, keymap=None, type=None, n=0)
read stored function from db with name ‘name’
- Parameters:
name (string) – filename of the klepto db
keymap (klepto.keymap) – keymap used for key encoding
type (klepto.archive) – type of klepto archive
n (int) – db entry in reverse order (i.e. most recent is
0
)
- Returns:
tuple of (stored function, distance information)
Notes
If the db is empty, or
n
produces a bad index, returnsNone
. Alternately,name
can be the relevant klepto.archive instance.
- class sql_archive(name=None, dict=None, cached=True, **kwds)
Bases:
sqltable_archive
initialize a sql database with a synchronized dictionary interface
Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql or postgresql database, sqlalchemy must be installed. When connecting to sqlite, the default database is ‘:memory:’. Storable values are limited to strings, integers, floats, and other basic objects. To store functions, classes, and similar constructs, sqlalchemy must be installed.
- Parameters:
- static __new__(sql_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a sql database archive backend
Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql database ‘foo’ on localhost, database=’mysql://user:pass@localhost/foo’. For postgresql, use database=’postgresql://user:pass@localhost/foo’. When connecting to sqlite, the default database is ‘:memory:’; otherwise, the default database is ‘defaultdb’. If sqlalchemy is not installed, storable values are limited to strings, integers, floats, and other basic objects. If sqlalchemy is installed, additional keyword options can provide database configuration, such as connection pooling. To use a mysql or postgresql database, sqlalchemy must be installed.
- Parameters:
name (str, default=None) – database url (see above note)
dict (dict, default={}) – initial dictionary to seed the archive
cached (bool, default=True) – interact through an in-memory cache
serialized (bool, default=True) – save objects as pickled strings
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
- classmethod from_frame(dataframe)
- class sqltable_archive(name=None, dict=None, cached=True, **kwds)
Bases:
sqltable_archive
initialize a sql database with a synchronized dictionary interface
Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql or postgresql database, sqlalchemy must be installed. When connecting to sqlite, the default database is ‘:memory:’. Storable values are limited to strings, integers, floats, and other basic objects. To store functions, classes, and similar constructs, sqlalchemy must be installed.
- Parameters:
- static __new__(sqltable_archive, name=None, dict=None, cached=True, **kwds)
initialize a dictionary with a sql database table archive backend
Connect to an existing database, or initialize a new database, at the selected database url. For example, to use a sqlite database ‘foo.db’ in the current directory, database=’sqlite:///foo.db’. To use a mysql database ‘foo’ on localhost, database=’mysql://user:pass@localhost/foo’. For postgresql, use database=’postgresql://user:pass@localhost/foo’. When connecting to sqlite, the default database is ‘:memory:’; otherwise, the default database is ‘defaultdb’. Connections should be given as database?table=tablename; for example, name=’sqlite:///foo.db?table=bar’. If not provided, the default tablename is ‘memo’. If sqlalchemy is not installed, storable values are limited to strings, integers, floats, and other basic objects. If sqlalchemy is installed, additional keyword options can provide database configuration, such as connection pooling. To use a mysql or postgresql database, sqlalchemy must be installed.
- Parameters:
name (str, default=None) – url for database table (see above note)
dict (dict, default={}) – initial dictionary to seed the archive
cached (bool, default=True) – interact through an in-memory cache
serialized (bool, default=True) – save objects as pickled strings
protocol (int, default=DEFAULT_PROTOCOL) – pickling protocol
- classmethod from_frame(dataframe)
- write(archive, entries, keymap=None)
write entries to klepto db instance ‘archive’
- Parameters:
archive (klepto.archive) – archive instance
entries (iterable) – iterable/dict to update the archive with
keymap (klepto.keymap) – keymap used for key encoding
- Returns:
None
Examples
>>> write(foo, {x0:y0, x1:y1, ...}) >>> write(foo, ((x0,y0), (x1,y1), ...))
Notes
Within
entries
,xi
is a tuple of floats, andyi
is a float.
function module
higher-level readers and writers for stored functions
- db(name)
get function db with the name ‘name’
- read(archives)
read stored functions from the list of dbs
- Parameters:
archives (list[string]) – list of names of function archives
- Returns:
a klepto.archive instance
Notes
The order of the dbs is important, with the index of
archives
corresponding to the desired axis. If a db is empty, returnsNone
for the empty db. Also, a klepto.archive instance can be provided instead of thename
of the db.
- write(function, archives)
write function to corresponding archives