Quick Start
Basic Usage
The main container in ducks is called Dex.
from ducks import Dex
# make some objects
objects = [
{'x': 3, 'y': 'a'},
{'x': 6, 'y': 'b'},
{'x': 9, 'y': 'c'}
]
# Create a Dex containing the objects.
# Index on x and y.
dex = Dex(objects, ['x', 'y'])
# match objects
dex[{
'x': {'>': 5, '<': 10}, # where 5 < x < 10
'y': {'in': ['a', 'b']} # and y is 'a' or 'b'
}]
# result: [{'x': 6, 'y': 'b'}]
This is a Dex of dicts, but the objects can be any type.
Dex supports ==, !=, in, not in, <, <=, >, >=.
The indexes can be dict keys, object attributes, or custom functions.
Alternative forms:
* {'a': 1}
may be used in place of {'a': {'==': 1}}
* {'a': [1, 2, 3]}
may be used in place of {'a': {'in': [1, 2, 3]}}
* eq
, ge
, gt
, and so on can be used in place of ==
, >=
, >
Add, remove, update
Dex supports add, remove, and update of objects.
from ducks import Dex
class Thing:
def __init__(self):
self.x = 1
self.y = 1
def __repr__(self):
return f"Thing(x: {self.x}, y: {self.y})"
# make an empty Dex
dex = Dex([], ['x', 'y'])
# add an object
obj = Thing()
dex.add(obj)
print(dex[{'x': 1}]) # find it
# update it
obj.x = 2
dex.update(obj)
print(dex[{'x': 2}]) # find updated obj
# remove it
dex.remove(obj)
print(list(dex)) # dex now contains no objects
Update notifies Dex that an object’s attributes have changed, so the index can be updated accordingly. There’s an example in Demos of how to automatically update Dex when objects change.
FrozenDex
If you don’t need add, remove, or update, use a FrozenDex instead. It is used just like a Dex, but it’s faster and more memory-efficient.
from ducks import FrozenDex
dex = FrozenDex([{'a': 1, 'b': 2}], ['a'])
dex[{'a': 1}] # result: [{'a': 1, 'b': 2}]
FrozenDex is thread-safe because it does not allow writes.
ConcurrentDex
For multithreaded cases where writes are needed, use ConcurrentDex. It is a thin wrapper around a Dex that uses a lock to provide thread-safety.
from ducks import ConcurrentDex, FAIR, READERS, WRITERS
objects = [{'a': 1, 'b': 2}]
dex = ConcurrentDex(objects, ['a'], priority=READERS)
dex[{'a': 1}] # result: [{'a': 1, 'b': 2}]
The ConcurrentDex API is the same as Dex. An optional kwarg ‘priority’ allows prioritization of readers, writers, or neither; the default is to prioritize reads.
Function attributes
Ducks can also index using functions evaluated on the objects. This allows indexing of object types such as strings.
Let’s find strings that are palindromes of length 3:
from ducks import Dex
strings = [
'ooh', 'wow',
'kayak', 'bob'
]
# define a function that
# takes the object as input
def is_palindrome(s):
return s == s[::-1]
# make a Dex
dex = Dex(strings, [is_palindrome, len])
dex[{
is_palindrome: True,
len: 3
}]
# result: ['wow', 'bob']
Functions are evaluated on the object when it is added to the Dex.
Nested data
Use functions to get values from nested data structures.
from ducks import Dex
objs = [
{'a': {'b': [1, 2, 3]}},
{'a': {'b': [4, 5, 6]}}
]
def get_nested(obj):
return obj['a']['b'][0]
dex = Dex(objs, [get_nested])
dex[{get_nested: 4}]
# result: {'a': {'b': [4, 5, 6]}}
Missing attributes
Objects don’t need to have every attribute.
Indexes are sparse. Objects that are missing an attribute will not be stored under that attribute. This saves lots of memory.
To find all objects that have an attribute, match the special value
ANY
.To find objects missing the attribute, do
{'!=': ANY}
.In functions, raise
MissingAttribute
to tell ducks the attribute is missing.
Example:
from ducks import Dex, ANY, MissingAttribute
objs = [{'a': 1}, {'a': 2}, {}]
def get_a(obj):
try:
return obj['a']
except KeyError:
raise MissingAttribute # tell Dex this attribute is missing
dex = Dex(objs, ['a', get_a])
print(dex[{'a': ANY}]) # [{'a': 1}, {'a': 2}]
print(dex[{get_a: ANY}]) # [{'a': 1}, {'a': 2}]
print(dex[{'a': {'!=': ANY}}]) # [{}]
Note that None
is treated as a normal attribute value and is stored.
Pickling
Dex, ConcurrentDex, and FrozenDex can be pickled using the special functions save
and load
.
from ducks import Dex, save, load
dex = Dex([1.2, 1.8, 2.7], [round])
save(dex, 'numbers.dex')
loaded_dex = load('numbers.dex')
loaded_dex[{round: 2}]
# result: 1.8
Objects inside the dex will be saved along with it.
Class APIs
There are three container classes: