9.2. functools
— Higher order functions and operations on callable objects¶
Source code: Lib/functools.py
The functools
module is for higher-order functions: functions that act on
or return other functions. In general, any callable object can be treated as a
function for the purposes of this module.
The functools
module defines the following functions:
-
functools.
cmp_to_key
(func)¶ Transform an old-style comparison function to a key-function. Used with tools that accept key functions (such as
sorted()
,min()
,max()
,heapq.nlargest()
,heapq.nsmallest()
,itertools.groupby()
). This function is primarily used as a transition tool for programs being converted from Py2.x which supported the use of comparison functions.A compare function is any callable that accept two arguments, compares them, and returns a negative number for less-than, zero for equality, or a positive number for greater-than. A key function is a callable that accepts one argument and returns another value indicating the position in the desired collation sequence.
Example:
sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order
New in version 3.2:
New in version 3.2.
-
@
functools.
lru_cache
(maxsize=100)¶ Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
If maxsize is set to None, the LRU feature is disabled and the cache can grow without bound.
To help measure the effectiveness of the cache and tune the maxsize parameter, the wrapped function is instrumented with a
cache_info()
function that returns a named tuple showing hits, misses, maxsize and currsize. In a multi-threaded environment, the hits and misses are approximate.The decorator also provides a
cache_clear()
function for clearing or invalidating the cache.The original underlying function is accessible through the
__wrapped__
attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.An LRU (least recently used) cache works best when more recent calls are the best predictors of upcoming calls (for example, the most popular articles on a news server tend to change daily). The cache’s size limit assures that the cache does not grow without bound on long-running processes such as web servers.
Example of an LRU cache for static web content:
@lru_cache(maxsize=20) def get_pep(num): 'Retrieve text of a Python Enhancement Proposal' resource = 'http://www.python.org/dev/peps/pep-%04d/' % num try: with urllib.request.urlopen(resource) as s: return s.read() except urllib.error.HTTPError: return 'Not Found' >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991: ... pep = get_pep(n) ... print(n, len(pep)) >>> print(get_pep.cache_info()) CacheInfo(hits=3, misses=8, maxsize=20, currsize=8)
Example of efficiently computing Fibonacci numbers using a cache to implement a dynamic programming technique:
@lru_cache(maxsize=None) def fib(n): if n < 2: return n return fib(n-1) + fib(n-2) >>> print([fib(n) for n in range(16)]) [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610] >>> print(fib.cache_info()) CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)
New in version 3.2:
New in version 3.2.
-
@
functools.
total_ordering
¶ Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:
The class must define one of
__lt__()
,__le__()
,__gt__()
, or__ge__()
. In addition, the class should supply an__eq__()
method.For example:
@total_ordering class Student: def __eq__(self, other): return ((self.lastname.lower(), self.firstname.lower()) == (other.lastname.lower(), other.firstname.lower())) def __lt__(self, other): return ((self.lastname.lower(), self.firstname.lower()) < (other.lastname.lower(), other.firstname.lower()))
New in version 3.2:
New in version 3.2.
-
functools.
partial
(func, *args, **keywords)¶ Return a new
partial
object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords. Roughly equivalent to:def partial(func, *args, **keywords): def newfunc(*fargs, **fkeywords): newkeywords = keywords.copy() newkeywords.update(fkeywords) return func(*(args + fargs), **newkeywords) newfunc.func = func newfunc.args = args newfunc.keywords = keywords return newfunc
The
partial()
is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature. For example,partial()
can be used to create a callable that behaves like theint()
function where the base argument defaults to two:>>> from functools import partial >>> basetwo = partial(int, base=2) >>> basetwo.__doc__ = 'Convert base 2 string to an int.' >>> basetwo('10010') 18
-
functools.
reduce
(function, iterable[, initializer])¶ Apply function of two arguments cumulatively to the items of sequence, from left to right, so as to reduce the sequence to a single value. For example,
reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])
calculates((((1+2)+3)+4)+5)
. The left argument, x, is the accumulated value and the right argument, y, is the update value from the sequence. If the optional initializer is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. If initializer is not given and sequence contains only one item, the first item is returned.
-
functools.
update_wrapper
(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)¶ Update a wrapper function to look like the wrapped function. The optional arguments are tuples to specify which attributes of the original function are assigned directly to the matching attributes on the wrapper function and which attributes of the wrapper function are updated with the corresponding attributes from the original function. The default values for these arguments are the module level constants WRAPPER_ASSIGNMENTS (which assigns to the wrapper function’s __name__, __module__, __annotations__ and __doc__, the documentation string) and WRAPPER_UPDATES (which updates the wrapper function’s __dict__, i.e. the instance dictionary).
To allow access to the original function for introspection and other purposes (e.g. bypassing a caching decorator such as
lru_cache()
), this function automatically adds a __wrapped__ attribute to the wrapper that refers to the original function.The main intended use for this function is in decorator functions which wrap the decorated function and return the wrapper. If the wrapper function is not updated, the metadata of the returned function will reflect the wrapper definition rather than the original function definition, which is typically less than helpful.
update_wrapper()
may be used with callables other than functions. Any attributes named in assigned or updated that are missing from the object being wrapped are ignored (i.e. this function will not attempt to set them on the wrapper function).AttributeError
is still raised if the wrapper function itself is missing any attributes named in updated.New in version 3.2:
New in version 3.2: Automatic addition of the
__wrapped__
attribute.
New in version 3.2:
New in version 3.2: Copying of the __annotations__
attribute by default.
Changed in version 3.2:
Changed in version 3.2: Missing attributes no longer trigger an AttributeError
.