ducks.frozen package

Submodules

ducks.frozen.frozen_attr module

Performs object lookup for a single attribute in a FrozenDex.

class ducks.frozen.frozen_attr.FrozenAttrIndex(attr: str | Callable, objs: ndarray, dtype: str)

Bases: object

Stores data and handles requests that are relevant to a single attribute of a FrozenDex.

There are three places where object indexes are stored.

none_ids stores all indexes for with the attribute value None
val_to_obj_ids stores object ids for attribute values that have many objects
val_arr + obj_id_arr store all the rest.

__init__(attr: str | Callable, objs: ndarray, dtype: str)

get(val) → ndarray: Get indexes of objects whose attribute is val.

get_all() → ndarray: Get indexes of every object with this attribute. Used when matching ANY.

get_ids_by_range(lo, hi, include_lo=False, include_hi=False) → ndarray: Get the object IDs associated with this value range as an Int64Set. Only usable when self.d is a tree.

get_values() → Set: Get each value we have objects for.

ducks.frozen.init_helpers module

ducks.frozen.init_helpers.get_vals(objs: ndarray, obj_id_arr: ndarray, attr: Callable | str): Gets vals by attribute. Returned arrays will be shorter than input if objects are missing attributes.

ducks.frozen.init_helpers.run_length_encode(arr: ndarray)

Find counts of each element in the arr (sorted) via run-length encoding.

Takes 10ms for 1M objs.

ducks.frozen.main module

class ducks.frozen.main.FrozenDex(objs: Iterable[Any], on: Iterable[str | Callable])

Bases: object

__getitem__(query: Dict) → ndarray

Find objects in the FrozenDex that satisfy the constraints.

Parameters:

query –

Dict of {attribute: expression} defining the subset of objects that match. If {}, all objects will match.

Each attribute is a string or Callable. Must be one of the attributes specified in the constructor.

The expression can be any of the following:

A dict of {operator: value}, such as {'==': 1} {'>': 5}, or {'in': [1, 2, 3]}.
A single value, which is a shorthand for {‘==’: value}.
A list of values, which is a shorthand for {'in': [list_of_values]}.

The expression {'==': ducks.ANY} will match all objects having the attribute. The expression {'!=': ducks.ANY} will match all objects without the attribute.

Valid operators are ‘==’, ‘!=’, ‘in’, ‘not in’, ‘<’, ‘<=’, ‘>’, ‘>=’. The aliases ‘eq’, ‘ne’, ‘lt’, ‘le’, ‘lte’, ‘gt’, ‘ge’, and ‘gte’ work too. To match a None value, use {'==': None}. There is no separate operator for None values.

Returns:

Numpy array of objects matching the constraints. Array will be in the same order as the original objects.

__init__(objs: Iterable[Any], on: Iterable[str | Callable])

Create a FrozenDex containing the objs, queryable by the on attributes.

Parameters:

objs – The objects that FrozenDex will contain.
on – The attributes that will be used for finding objects. Must contain at least one.

It’s OK if the objects in objs are missing some or all of the attributes in on.

For the objects that do contain the attributes on on, those attribute values must be hashable and sortable. Most Python objects are hashable. Implement the function __lt__(self, other) to make a class sortable. An attribute value of None is acceptable as well, even though None is not sortable.

get_values(attr: str | Callable) → Set

Get the set of unique values we have for the given attribute.

Parameters:: attr – The attribute to get values for.
Returns:: Set of all unique values for this attribute.

ducks.frozen.main.load(box: FrozenDex): Creates a FrozenDex from the pickle file contents.

ducks.frozen.main.save(box: FrozenDex, filepath: str): Saves this object to a pickle file.

ducks.frozen.utils module

ducks.frozen.utils.snp_difference(left: ndarray, right: ndarray)