Reference

h5preserve is a thin wrapper around h5py, providing easier serialisation of native python types.

copyright:
  1. 2016 James Tocknell

license:

3-clause BSD

class h5preserve.DatasetContainer(attrs=None, **kwargs)[source]

Representation of an hdf5 dataset for use in h5preserve.

Parameters:
  • attrs (Mapping) – mapping containing the attributes of the group

  • **kwargs – properties of the group, which get passed to create group

clear() None.  Remove all items from D.
get(k[, d]) D[k] if k in D, else d.  d defaults to None.
items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() an object providing a view on D's values
class h5preserve.DelayedContainer[source]

Helper class for allowing delayed writing of containers to hdf5 files.

write_container(data)[source]

Write data to hdf5 file with the associated located of the DelayedContainer.

class h5preserve.GroupContainer(attrs=None, **kwargs)[source]

Representation of an hdf5 group for use in h5preserve.

Parameters:
  • attrs (Mapping) – mapping containing the attributes of the group

  • **kwargs – datasets or subgroups to add to the group

clear() None.  Remove all items from D.
get(k[, d]) D[k] if k in D, else d.  d defaults to None.
items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() an object providing a view on D's values
class h5preserve.H5PreserveFile(h5py_file, registries)[source]

Thin wrapper around h5py.File to automatically use h5preserve when accessing the file contents.

Acts like h5preserve.H5PreserveGroup, but allows access to the associated h5py.File instance via h5py_file.

Parameters:
  • h5py_file (a h5py.File) – the hdf5 file to wrap

  • registries (RegistryContainer) – the collection of registries that you want to use to read from the hdf5 file

clear() None.  Remove all items from D.
create_group(name)

Creates a new group in the associated hdf5 file

Parameters:

name (string, or other identifier accepted by h5py) – name of the new group

Returns:

The new group wrapped by H5PreserveGroup

Return type:

H5PreserveGroup

get(k[, d]) D[k] if k in D, else d.  d defaults to None.
property h5py_file

the instance of h5py.File which H5PreserveFile wraps

Type:

h5py.File

property h5py_group

the instance of h5py.Group which H5PreserveGroup wraps

Type:

h5py.Group

items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

require_group(name)

Returns the group associated with name, creating it if necessary.

Parameters:

name (string, or other identifier accepted by h5py) – name of the desired group

Returns:

The group wrapped by H5PreserveGroup

Return type:

H5PreserveGroup

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() an object providing a view on D's values
class h5preserve.H5PreserveGroup(h5py_group, registries)[source]

Thin wrapper around h5py.Group to automatically use h5preserve when accessing the group contents.

Parameters:
  • h5py_group (h5py.Group) –

  • registries (RegistryContainer) – the collection of registries that you want to use to read from the hdf5 file

clear() None.  Remove all items from D.
create_group(name)[source]

Creates a new group in the associated hdf5 file

Parameters:

name (string, or other identifier accepted by h5py) – name of the new group

Returns:

The new group wrapped by H5PreserveGroup

Return type:

H5PreserveGroup

get(k[, d]) D[k] if k in D, else d.  d defaults to None.
property h5py_group

the instance of h5py.Group which H5PreserveGroup wraps

Type:

h5py.Group

items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

require_group(name)[source]

Returns the group associated with name, creating it if necessary.

Parameters:

name (string, or other identifier accepted by h5py) – name of the desired group

Returns:

The group wrapped by H5PreserveGroup

Return type:

H5PreserveGroup

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() an object providing a view on D's values

Represent a h5py hard link to be created via h5preserve.

Parameters:

obj (string, h5py.Group or h5py.Dataset) – the h5py object that the hard link points to, can either be an h5py object, or a string with the absolute path of the object

property h5py_obj

the object which the hard link will point to

Type:

h5py.Group or h5py.Dataset

property path

The path this object points to

class h5preserve.OnDemandDatasetContainer(attrs=None, **kwargs)[source]

Subclass of DatasetContainer which supports accessing dataset data on demand, rather that loading immediately.

clear() None.  Remove all items from D.
get(k[, d]) D[k] if k in D, else d.  d defaults to None.
items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() an object providing a view on D's values
class h5preserve.OnDemandGroupContainer(attrs=None, **kwargs)[source]

Subclass of GroupContainer which supports accessing group members on demand, rather that loading immediately.

clear() None.  Remove all items from D.
get(k[, d]) D[k] if k in D, else d.  d defaults to None.
items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

setdefault(k[, d]) D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) None.  Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values() an object providing a view on D's values
class h5preserve.OnDemandWrapper(func)[source]

Wrapper which represents a container which can be accessed on demand.

class h5preserve.Registry(name)[source]

Register of functions for converting between hdf5 and python.

This is the core of h5preserve, containing the information about how to convert to and from hdf5 files, what version to use, and the namespace of created data.

Parameters:

name (string) – name of registry for identification purposes

dumper(cls, label, version)[source]

Decorator function to create a dumper function.

Parameters:
  • cls (any class) – the class which this dumper operates on

  • label (string) – the label or tag associated with this class

  • version (integer, None) – The version of the output that this function returns.

freeze()[source]

Freeze the registry, preventing further changes to the registry.

loader(label, version)[source]

Decorator function to create a loader function.

Parameters:
  • label (string) – the label or tag associated with this class

  • version (integer, any, all, None) – The version of the output that this function reads.

property name

name of the registry

Type:

str

class h5preserve.RegistryContainer(*registries)[source]

Ordered container of registries which manages interaction with the hdf5 file.

Parameters:

*registries (list of Registry) – the list of registries to be associated with this container

append(value)

S.append(value) – append value to the end of the sequence

clear() None -- remove all items from S
count(value) integer -- return number of occurrences of value
dump(obj)[source]

Dump native python object to h5preserve representation

Parameters:

obj – the object to dump

extend(values)

S.extend(iterable) – extend sequence by appending elements from the iterable

from_file(h5py_obj)[source]

Return an representation of a hdf5 object from a hdf5 file

Parameters:

h5py_obj (a h5py object, e.g. group, dataset) –

index(value[, start[, stop]]) integer -- return first index of value.

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

insert(index, value)[source]

S.insert(index, value) – insert value before index

load(obj)[source]

Load native python object from h5preserve representation

Parameters:

obj – the object to load

lock_version(cls, version)[source]

Lock output version for a specific class

Parameters:
  • cls (any class) – the class to lock the version of

  • version (integer, any, all, None) – the version which will always be used

pop([index]) item -- remove and return item at index (default last).

Raise IndexError if list is empty or index is out of range.

property registries

Iterator over the registries contained in the order they were added.

remove(value)

S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.

reverse()

S.reverse() – reverse IN PLACE

to_file(h5py_group, key, val)[source]

Dump h5preserve object to hdf5 file

Parameters:
  • h5py_group (h5py.Group) – the group to add the object to

  • key (string) – the name for the object

  • val – the object to add

h5preserve.new_registry_list(*registries)[source]

Create a new list of registries which includes builtin registries.

Parameters:

*registries (list of Registry) – the list of registries to be associated with this container

h5preserve.open(filename, registries, *, mode, **kwargs)[source]

Open a hdf5 file wrapped with h5preserve.

Parameters:
  • filename (string, or other identifier accepted by h5py.File) –

  • registries (RegistryContainer) – the collection of registries that you want to use to read from the hdf5 file

  • **kwargs – additional keyword arguments to pass to h5py.File

h5preserve.wrap_on_demand(obj, key, val)[source]

Wrap val such that it can be used on demand.

wrap_on_demand returns either the original val if obj has not yet been dumped, or a wrapped version of val if obj has been dumped.

wrap_on_demand automatically deals with wrapping/unwrapping if needed, so it is save to repeatedly call on the same object.

Parameters:
  • obj (any dumpable object) – the object which val is a member of or an attribute of

  • key (string) – the key to be used when writing out val

  • val (any dumpable object) – the object to be wrapped