Simple Authenticating Store

This module provides a simple wrapper for a store that implements a simple authentication scheme. This may be used as a base for more complex and fine-grained authentication.

By default it authenticates by computing a (salted) hash of the user’s password and validates it against the hash stored in an appropriate key. Authenticated users then have full access to all keys.

Subclasses can refine this behaviour by overriding the check_permissions() method to provide different or more controlled permissioning.

encore.storage.simple_auth_store.make_encoder(salt, hasher=None)

Create a moderately secure salted encoder

Parameters:
  • salt (bytes) – A salt that is added to the user-supplied password before hashing. This salt should be kept secret, but needs to be remembered across invocations (ie. the same salt needs to be used every time the password is encoded).
  • hasher (callable) – A callable that takes a string and returns a cryptographic hash of the string. The default is sha1_hasher().
encore.storage.simple_auth_store.sha1_hasher(s)

A simple utility function for producing a sha1 digest of a string.

class encore.storage.simple_auth_store.SimpleAuthStore(store, encoder, user_key_path='.user_', user_key_store=None)

A key-value store that wraps another store and implements simple authentication

This wraps an existing store with no notion of authentication and provides simple username/password authentication, storing a hash of the password in the wrapped store.

The base implementation has all-or-nothing

Parameters:
  • event_manager – An event_manager which implements the BaseEventManager API.
  • store (AbstractStore instance) – The wrapped store that actually holds the data.
  • encoder (callable) – A callable that computes the password hash.
  • user_key_path (str) – The prefix to put before the username for the keys that store the user’s information. At present these keys must simply hold the encoded hash of the user’s password.
  • user_key_store (AbstractStore instance) – The store to store the user keys in. Defaults to the wrapped store.
check_permissions(key=None)

Return permissions that the user has for the provided key

The default behaviour gives all authenticated users full access to all keys. Subclasses may implement finer-grained controls based on user groups or other permissioning systems.

Parameters:key (str or None) – The key which the permissions are being requested for, or the global permissions if the key is None.
Returns:permissions (set) - A set of strings chosen from ‘connect’, ‘exists’, ‘get’, ‘set’, and/or ‘delete’ which express the permissions that the user has on that particular key.
connect(credentials=None)

Connect to the key-value store, optionally with authentication

This method creates or connects to any long-lived resources that the store requires.

Parameters:credentials – A dictionary with keys ‘username’ and ‘password’.
delete(key)

Delete a key from the repsository.

This may be left unimplemented by subclasses that represent a read-only key-value store.

Parameters:key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
Events:StoreDeleteEvent - On successful completion of a transaction, a StoreDeleteEvent should be emitted with the key.
Raises:AuthenticationError - If the user has no rights to delete the key, then an Authentication error is raised.
disconnect()

Disconnect from the key-value store

This method disposes or disconnects to any long-lived resources that the store requires.

exists(key)

Test whether or not a key exists in the key-value store

If a user does not have ‘exists’ permissions for this key, then it will return False, even if the key exists in the underlying store.

Parameters:key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
Returns:exists (bool) - Whether or not the key exists in the key-value store.
from_bytes(key, data, buffer_size=1048576)

Efficiently store a bytes object as the data associated with a key.

This method can be optionally overriden by subclasses to proved a more efficient way of copy the data from a bytes object to the underlying data store. The default implementation uses the set() method together with a cStringIO.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • data (bytes) – The data as a bytes object.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
from_file(key, path, buffer_size=1048576)

Efficiently read data from a file into a key in the key-value store.

This method can be optionally overriden by subclasses to proved a more efficient way of copy the data from a path in the filesystem to the underlying data store. The default implementation uses the set() method together with chunked reads from the disk which are fed into the data stream.

This makes no attempt to set metadata.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • path (string) – A file system path to read the data from.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
get(key)

Retrieve a stream of data and metdata from a given key in the key-value store.

Parameters:

key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
Returns:

  • data (file-like) - A readable file-like object that provides stream of data from the key-value store
  • metadata (dictionary) - A dictionary of metadata for the key.
Raises:
  • KeyError - If the key is not found in the store, or does not exist for the user, a KeyError is raised.
  • AuthenticationError - If the user has no rights to get the key, then an Authentication error is raised.
get_data(key)

Retrieve a stream from a given key in the key-value store.

Parameters:

key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
Returns:

data (file-like) - A readable file-like object the that provides stream of data from the key-value store.
Raises:
  • KeyError - This will raise a key error if the key is not present in the store.
  • AuthenticationError - If the user has no rights to get the key, then an Authentication error is raised.
get_metadata(key, select=None)

Retrieve the metadata for a given key in the key-value store.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • select (iterable of strings or None) – Which metadata keys to populate in the result. If unspecified, then return the entire metadata dictionary.
Returns:

metadata (dict) - A dictionary of metadata associated with the key. The dictionary has keys as specified by the select argument. If a key specified in select is not present in the metadata, then it will not be present in the returned value.
Raises:
  • KeyError - This will raise a key error if the key is not present in the store.
  • AuthenticationError - If the user has no rights to get the key, then an Authentication error is raised.
glob(pattern)

Return keys which match glob-style patterns

Parameters:pattern (string) – Glob-style pattern to match keys with.
Returns:result (iterable) - A iterable of keys which match the glob pattern.
is_connected()

Whether or not the store is currently connected

Returns:connected (bool) - Whether or not the store is currently connected.
multiget(keys)

Retrieve the data and metadata for a collection of keys.

Parameters:keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
Returns:result (iterator of (file-like, dict) tuples) - An iterator of (data, metadata) pairs.
Raises:KeyError - This will raise a key error if the key is not present in the store.
multiget_data(keys)

Retrieve the data for a collection of keys.

Parameters:keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
Returns:result (iterator of file-like) - An iterator of file-like data objects corresponding to the keys.
Raises:KeyError - This will raise a key error if the key is not present in the store.
multiget_metadata(keys, select=None)

Retrieve the metadata for a collection of keys in the key-value store.

Parameters:
  • keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
  • select (iterable of strings or None) – Which metadata keys to populate in the results. If unspecified, then return the entire metadata dictionary.
Returns:

metadatas (iterator of dicts) - An iterator of dictionaries of metadata associated with the key. The dictionaries have keys as specified by the select argument. If a key specified in select is not present in the metadata, then it will not be present in the returned value.
Raises:

KeyError - This will raise a key error if the key is not present in the store.
multiset(keys, values, buffer_size=1048576)

Set the data and metadata for a collection of keys.

Where supported by an implementation, this should perform the whole collection of sets as a single transaction.

Like zip() if keys and values have different lengths, then any excess values in the longer list should be silently ignored.

Parameters:
  • keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
  • values (iterable of (file-like, dict) tuples) – An iterator that provides the (data, metadata) pairs for the corresponding keys.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
Events:
  • StoreProgressStartEvent - For buffering implementations, this event should be emitted prior to writing any data to the underlying store.
  • StoreProgressStepEvent - For buffering implementations, this event should be emitted periodically as data is written to the underlying store.
  • StoreProgressEndEvent - For buffering implementations, this event should be emitted after finishing writing to the underlying store.
  • StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata for each key that was set.
multiset_data(keys, datas, buffer_size=1048576)

Set the data for a collection of keys.

Where supported by an implementation, this should perform the whole collection of sets as a single transaction.

Like zip() if keys and datas have different lengths, then any excess values in the longer list should be silently ignored.

Parameters:
  • keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
  • datas (iterable of file-like objects) – An iterator that provides the data file-like objects for the corresponding keys.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
Events:
  • StoreProgressStartEvent - For buffering implementations, this event should be emitted prior to writing any data to the underlying store.
  • StoreProgressStepEvent - For buffering implementations, this event should be emitted periodically as data is written to the underlying store.
  • StoreProgressEndEvent - For buffering implementations, this event should be emitted after finishing writing to the underlying store.
  • StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata for each key that was set.
multiset_metadata(keys, metadatas)

Set the metadata for a collection of keys.

Where supported by an implementation, this should perform the whole collection of sets as a single transaction.

Like zip() if keys and metadatas have different lengths, then any excess values in the longer list should be silently ignored.

Parameters:
  • keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
  • metadatas (iterable of dicts) – An iterator that provides the metadata dictionaries for the corresponding keys.
Events:

StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata for each key that was set.
multiupdate_metadata(keys, metadatas)

Update the metadata for a collection of keys.

Where supported by an implementation, this should perform the whole collection of sets as a single transaction.

Like zip() if keys and metadatas have different lengths, then any excess values in the longer list should be silently ignored.

Parameters:
  • keys (iterable of strings) – The keys for the resources in the key-value store. Each key is a unique identifier for a resource within the key-value store.
  • metadatas (iterable of dicts) – An iterator that provides the metadata dictionaries for the corresponding keys.
Events:

StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata for each key that was set.
set(key, value, buffer_size=1048576)

Store a stream of data into a given key in the key-value store.

This may be left unimplemented by subclasses that represent a read-only key-value store.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • value (tuple of file-like, dict) – A pair of objects, the first being a readable file-like object that provides stream of data from the key-value store. The second is a dictionary of metadata for the key.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
Events:
  • StoreProgressStartEvent - For buffering implementations, this event should be emitted prior to writing any data to the underlying store.
  • StoreProgressStepEvent - For buffering implementations, this event should be emitted periodically as data is written to the underlying store.
  • StoreProgressEndEvent - For buffering implementations, this event should be emitted after finishing writing to the underlying store.
  • StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata
Raises:

AuthenticationError - If the user has no rights to set the key, then an Authentication error is raised.
set_data(key, data, buffer_size=1048576)

Replace the data for a given key in the key-value store.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • data (file-like) – A readable file-like object the that provides stream of data from the key-value store.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
Events:
  • StoreProgressStartEvent - For buffering implementations, this event should be emitted prior to writing any data to the underlying store.
  • StoreProgressStepEvent - For buffering implementations, this event should be emitted periodically as data is written to the underlying store.
  • StoreProgressEndEvent - For buffering implementations, this event should be emitted after finishing writing to the underlying store.
  • StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata
Raises:

AuthenticationError - If the user has no rights to set the key, then an Authentication error is raised.
set_metadata(key, metadata)

Set new metadata for a given key in the key-value store.

This replaces the existing metadata set for the key with a new set of metadata.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • metadata (dict) – A dictionary of metadata to associate with the key. The dictionary keys should be strings which are valid Python identifiers.
Events:

StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata
Raises:

AuthenticationError - If the user has no rights to set the key, then an Authentication error is raised.
to_bytes(key, buffer_size=1048576)

Efficiently store the data associated with a key into a bytes object.

This method can be optionally overriden by subclasses to proved a more efficient way of copy the data from the underlying data store to a bytes object. The default implementation uses the get() method together with chunked reads from the returned data stream and join.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
Returns:

bytes - The contents of the file-like object as bytes.
Events:
  • StoreProgressStartEvent - For buffering implementations, this event should be emitted prior to extracting the data.
  • StoreProgressStepEvent - For buffering implementations, this event should be emitted periodically as data is extracted.
  • StoreProgressEndEvent - For buffering implementations, this event should be emitted after extracting the data.
to_file(key, path, buffer_size=1048576)

Efficiently store the data associated with a key into a file.

This method can be optionally overriden by subclasses to proved a more efficient way of copy the data from the underlying data store to a path in the filesystem. The default implementation uses the get() method together with chunked reads from the returned data stream to the disk.

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • path (string) – A file system path to store the data to.
  • buffer_size (int) – An optional indicator of the number of bytes to read at a time. Implementations are free to ignore this hint or use a different default if they need to. The default is 1048576 bytes (1 MiB).
Events:
  • StoreProgressStartEvent - For buffering implementations, this event should be emitted prior to writing any data to disk.
  • StoreProgressStepEvent - For buffering implementations, this event should be emitted periodically as data is written to disk.
  • StoreProgressEndEvent - For buffering implementations, this event should be emitted after finishing writing to disk.
update_metadata(key, metadata)

Update the metadata for a given key in the key-value store.

This performs a dictionary update on the existing metadata with the provided metadata keys and values

Parameters:
  • key (string) – The key for the resource in the key-value store. They key is a unique identifier for the resource within the key-value store.
  • metadata (dict) – A dictionary of metadata to associate with the key. The dictionary keys should be strings which are valid Python identifiers.
Events:

StoreSetEvent - On successful completion of a transaction, a StoreSetEvent should be emitted with the key & metadata
Raises:

AuthenticationError - If the user has no rights to set the key, then an Authentication error is raised.