16.2. threading
— Thread-based parallelism¶
Source code: Lib/threading.py
This module constructs higher-level threading interfaces on top of the lower
level _thread
module. See also the queue
module.
The dummy_threading
module is provided for situations where
threading
cannot be used because _thread
is missing.
Note
While they are not listed below, the camelCase
names used for some
methods and functions in this module in the Python 2.x series are still
supported by this module.
CPython implementation detail: Due to the Global Interpreter Lock, in CPython only one thread
can execute Python code at once (even though certain performance-oriented
libraries might overcome this limitation).
If you want your application to make better of use of the computational
resources of multi-core machines, you are advised to use
multiprocessing
or concurrent.futures.ProcessPoolExecutor
.
However, threading is still an appropriate model if you want to run
multiple I/O-bound tasks simultaneously.
This module defines the following functions and objects:
-
threading.
active_count
()¶ Return the number of
Thread
objects currently alive. The returned count is equal to the length of the list returned byenumerate()
.
-
threading.
Condition
() A factory function that returns a new condition variable object. A condition variable allows one or more threads to wait until they are notified by another thread.
See Condition Objects.
-
threading.
current_thread
()¶ Return the current
Thread
object, corresponding to the caller’s thread of control. If the caller’s thread of control was not created through thethreading
module, a dummy thread object with limited functionality is returned.
-
threading.
enumerate
()¶ Return a list of all
Thread
objects currently alive. The list includes daemonic threads, dummy thread objects created bycurrent_thread()
, and the main thread. It excludes terminated threads and threads that have not yet been started.
-
threading.
Event
() A factory function that returns a new event object. An event manages a flag that can be set to true with the
set()
method and reset to false with theclear()
method. Thewait()
method blocks until the flag is true.See Event Objects.
-
class
threading.
local
¶ A class that represents thread-local data. Thread-local data are data whose values are thread specific. To manage thread-local data, just create an instance of
local
(or a subclass) and store attributes on it:mydata = threading.local() mydata.x = 1
The instance’s values will be different for separate threads.
For more details and extensive examples, see the documentation string of the
_threading_local
module.
-
threading.
Lock
()¶ A factory function that returns a new primitive lock object. Once a thread has acquired it, subsequent attempts to acquire it block, until it is released; any thread may release it.
See Lock Objects.
-
threading.
RLock
()¶ A factory function that returns a new reentrant lock object. A reentrant lock must be released by the thread that acquired it. Once a thread has acquired a reentrant lock, the same thread may acquire it again without blocking; the thread must release it once for each time it has acquired it.
See RLock Objects.
-
threading.
Semaphore
(value=1) A factory function that returns a new semaphore object. A semaphore manages a counter representing the number of
release()
calls minus the number ofacquire()
calls, plus an initial value. Theacquire()
method blocks if necessary until it can return without making the counter negative. If not given, value defaults to 1.See Semaphore Objects.
-
threading.
BoundedSemaphore
(value=1)¶ A factory function that returns a new bounded semaphore object. A bounded semaphore checks to make sure its current value doesn’t exceed its initial value. If it does,
ValueError
is raised. In most situations semaphores are used to guard resources with limited capacity. If the semaphore is released too many times it’s a sign of a bug. If not given, value defaults to 1.
-
class
threading.
Thread
A class that represents a thread of control. This class can be safely subclassed in a limited fashion.
See Thread Objects.
-
class
threading.
Timer
A thread that executes a function after a specified interval has passed.
See Timer Objects.
-
threading.
settrace
(func)¶ Set a trace function for all threads started from the
threading
module. The func will be passed tosys.settrace()
for each thread, before itsrun()
method is called.
-
threading.
setprofile
(func)¶ Set a profile function for all threads started from the
threading
module. The func will be passed tosys.setprofile()
for each thread, before itsrun()
method is called.
-
threading.
stack_size
([size])¶ Return the thread stack size used when creating new threads. The optional size argument specifies the stack size to be used for subsequently created threads, and must be 0 (use platform or configured default) or a positive integer value of at least 32,768 (32kB). If changing the thread stack size is unsupported, a
ThreadError
is raised. If the specified stack size is invalid, aValueError
is raised and the stack size is unmodified. 32kB is currently the minimum supported stack size value to guarantee sufficient stack space for the interpreter itself. Note that some platforms may have particular restrictions on values for the stack size, such as requiring a minimum stack size > 32kB or requiring allocation in multiples of the system memory page size - platform documentation should be referred to for more information (4kB pages are common; using multiples of 4096 for the stack size is the suggested approach in the absence of more specific information). Availability: Windows, systems with POSIX threads.
This module also defines the following constant:
-
threading.
TIMEOUT_MAX
¶ The maximum value allowed for the timeout parameter of blocking functions (
Lock.acquire()
,RLock.acquire()
,Condition.wait()
, etc.). Specifying a timeout greater than this value will raise anOverflowError
.New in version 3.2:
New in version 3.2.
Detailed interfaces for the objects are documented below.
The design of this module is loosely based on Java’s threading model. However,
where Java makes locks and condition variables basic behavior of every object,
they are separate objects in Python. Python’s Thread
class supports a
subset of the behavior of Java’s Thread class; currently, there are no
priorities, no thread groups, and threads cannot be destroyed, stopped,
suspended, resumed, or interrupted. The static methods of Java’s Thread class,
when implemented, are mapped to module-level functions.
All of the methods described below are executed atomically.
16.2.1. Thread Objects¶
This class represents an activity that is run in a separate thread of control.
There are two ways to specify the activity: by passing a callable object to the
constructor, or by overriding the run()
method in a subclass. No other
methods (except for the constructor) should be overridden in a subclass. In
other words, only override the __init__()
and run()
methods of
this class.
Once a thread object is created, its activity must be started by calling the
thread’s start()
method. This invokes the run()
method in a
separate thread of control.
Once the thread’s activity is started, the thread is considered ‘alive’. It
stops being alive when its run()
method terminates – either normally, or
by raising an unhandled exception. The is_alive()
method tests whether the
thread is alive.
Other threads can call a thread’s join()
method. This blocks the calling
thread until the thread whose join()
method is called is terminated.
A thread has a name. The name can be passed to the constructor, and read or
changed through the name
attribute.
A thread can be flagged as a “daemon thread”. The significance of this flag is
that the entire Python program exits when only daemon threads are left. The
initial value is inherited from the creating thread. The flag can be set
through the daemon
property.
There is a “main thread” object; this corresponds to the initial thread of control in the Python program. It is not a daemon thread.
There is the possibility that “dummy thread objects” are created. These are
thread objects corresponding to “alien threads”, which are threads of control
started outside the threading module, such as directly from C code. Dummy
thread objects have limited functionality; they are always considered alive and
daemonic, and cannot be join()
ed. They are never deleted, since it is
impossible to detect the termination of alien threads.
-
class
threading.
Thread
(group=None, target=None, name=None, args=(), kwargs={})¶ This constructor should always be called with keyword arguments. Arguments are:
group should be
None
; reserved for future extension when aThreadGroup
class is implemented.target is the callable object to be invoked by the
run()
method. Defaults toNone
, meaning nothing is called.name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.
args is the argument tuple for the target invocation. Defaults to
()
.kwargs is a dictionary of keyword arguments for the target invocation. Defaults to
{}
.If the subclass overrides the constructor, it must make sure to invoke the base class constructor (
Thread.__init__()
) before doing anything else to the thread.-
start
()¶ Start the thread’s activity.
It must be called at most once per thread object. It arranges for the object’s
run()
method to be invoked in a separate thread of control.This method will raise a
RuntimeError
if called more than once on the same thread object.
-
run
()¶ Method representing the thread’s activity.
You may override this method in a subclass. The standard
run()
method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.
-
join
(timeout=None)¶ Wait until the thread terminates. This blocks the calling thread until the thread whose
join()
method is called terminates – either normally or through an unhandled exception – or until the optional timeout occurs.When the timeout argument is present and not
None
, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). Asjoin()
always returnsNone
, you must callis_alive()
afterjoin()
to decide whether a timeout happened – if the thread is still alive, thejoin()
call timed out.When the timeout argument is not present or
None
, the operation will block until the thread terminates.A thread can be
join()
ed many times.join()
raises aRuntimeError
if an attempt is made to join the current thread as that would cause a deadlock. It is also an error tojoin()
a thread before it has been started and attempts to do so raises the same exception.
-
name
¶ A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.
-
ident
¶ The ‘thread identifier’ of this thread or
None
if the thread has not been started. This is a nonzero integer. See thethread.get_ident()
function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.
-
is_alive
()¶ Return whether the thread is alive.
This method returns
True
just before therun()
method starts until just after therun()
method terminates. The module functionenumerate()
returns a list of all alive threads.
-
daemon
¶ A boolean value indicating whether this thread is a daemon thread (True) or not (False). This must be set before
start()
is called, otherwiseRuntimeError
is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default todaemon
=False
.The entire Python program exits when no alive non-daemon threads are left.
-
16.2.2. Lock Objects¶
A primitive lock is a synchronization primitive that is not owned by a
particular thread when locked. In Python, it is currently the lowest level
synchronization primitive available, implemented directly by the _thread
extension module.
A primitive lock is in one of two states, “locked” or “unlocked”. It is created
in the unlocked state. It has two basic methods, acquire()
and
release()
. When the state is unlocked, acquire()
changes the state
to locked and returns immediately. When the state is locked, acquire()
blocks until a call to release()
in another thread changes it to unlocked,
then the acquire()
call resets it to locked and returns. The
release()
method should only be called in the locked state; it changes the
state to unlocked and returns immediately. If an attempt is made to release an
unlocked lock, a RuntimeError
will be raised.
When more than one thread is blocked in acquire()
waiting for the state to
turn to unlocked, only one thread proceeds when a release()
call resets
the state to unlocked; which one of the waiting threads proceeds is not defined,
and may vary across implementations.
All methods are executed atomically.
-
Lock.
acquire
(blocking=True, timeout=-1)¶ Acquire a lock, blocking or non-blocking.
When invoked without arguments, block until the lock is unlocked, then set it to locked, and return true.
When invoked with the blocking argument set to true, do the same thing as when called without arguments, and return true.
When invoked with the blocking argument set to false, do not block. If a call without an argument would block, return false immediately; otherwise, do the same thing as when called without arguments, and return true.
When invoked with the floating-point timeout argument set to a positive value, block for at most the number of seconds specified by timeout and as long as the lock cannot be acquired. A negative timeout argument specifies an unbounded wait. It is forbidden to specify a timeout when blocking is false.
The return value is
True
if the lock is acquired successfully,False
if not (for example if the timeout expired).Changed in version 3.2:
Changed in version 3.2: The timeout parameter is new.
Changed in version 3.2:
Changed in version 3.2: Lock acquires can now be interrupted by signals on POSIX.
-
Lock.
release
()¶ Release a lock.
When the lock is locked, reset it to unlocked, and return. If any other threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed.
Do not call this method when the lock is unlocked.
There is no return value.
16.2.3. RLock Objects¶
A reentrant lock is a synchronization primitive that may be acquired multiple times by the same thread. Internally, it uses the concepts of “owning thread” and “recursion level” in addition to the locked/unlocked state used by primitive locks. In the locked state, some thread owns the lock; in the unlocked state, no thread owns it.
To lock the lock, a thread calls its acquire()
method; this returns once
the thread owns the lock. To unlock the lock, a thread calls its
release()
method. acquire()
/release()
call pairs may be
nested; only the final release()
(the release()
of the outermost
pair) resets the lock to unlocked and allows another thread blocked in
acquire()
to proceed.
-
RLock.
acquire
(blocking=True, timeout=-1)¶ Acquire a lock, blocking or non-blocking.
When invoked without arguments: if this thread already owns the lock, increment the recursion level by one, and return immediately. Otherwise, if another thread owns the lock, block until the lock is unlocked. Once the lock is unlocked (not owned by any thread), then grab ownership, set the recursion level to one, and return. If more than one thread is blocked waiting until the lock is unlocked, only one at a time will be able to grab ownership of the lock. There is no return value in this case.
When invoked with the blocking argument set to true, do the same thing as when called without arguments, and return true.
When invoked with the blocking argument set to false, do not block. If a call without an argument would block, return false immediately; otherwise, do the same thing as when called without arguments, and return true.
When invoked with the floating-point timeout argument set to a positive value, block for at most the number of seconds specified by timeout and as long as the lock cannot be acquired. Return true if the lock has been acquired, false if the timeout has elapsed.
Changed in version 3.2:
Changed in version 3.2: The timeout parameter is new.
-
RLock.
release
()¶ Release a lock, decrementing the recursion level. If after the decrement it is zero, reset the lock to unlocked (not owned by any thread), and if any other threads are blocked waiting for the lock to become unlocked, allow exactly one of them to proceed. If after the decrement the recursion level is still nonzero, the lock remains locked and owned by the calling thread.
Only call this method when the calling thread owns the lock. A
RuntimeError
is raised if this method is called when the lock is unlocked.There is no return value.
16.2.4. Condition Objects¶
A condition variable is always associated with some kind of lock; this can be passed in or one will be created by default. (Passing one in is useful when several condition variables must share the same lock.)
A condition variable has acquire()
and release()
methods that call
the corresponding methods of the associated lock. It also has a wait()
method, and notify()
and notify_all()
methods. These three must only
be called when the calling thread has acquired the lock, otherwise a
RuntimeError
is raised.
The wait()
method releases the lock, and then blocks until it is awakened
by a notify()
or notify_all()
call for the same condition variable in
another thread. Once awakened, it re-acquires the lock and returns. It is also
possible to specify a timeout.
The notify()
method wakes up one of the threads waiting for the condition
variable, if any are waiting. The notify_all()
method wakes up all threads
waiting for the condition variable.
Note: the notify()
and notify_all()
methods don’t release the lock;
this means that the thread or threads awakened will not return from their
wait()
call immediately, but only when the thread that called
notify()
or notify_all()
finally relinquishes ownership of the lock.
Tip: the typical programming style using condition variables uses the lock to
synchronize access to some shared state; threads that are interested in a
particular change of state call wait()
repeatedly until they see the
desired state, while threads that modify the state call notify()
or
notify_all()
when they change the state in such a way that it could
possibly be a desired state for one of the waiters. For example, the following
code is a generic producer-consumer situation with unlimited buffer capacity:
# Consume one item
cv.acquire()
while not an_item_is_available():
cv.wait()
get_an_available_item()
cv.release()
# Produce one item
cv.acquire()
make_an_item_available()
cv.notify()
cv.release()
To choose between notify()
and notify_all()
, consider whether one
state change can be interesting for only one or several waiting threads. E.g.
in a typical producer-consumer situation, adding one item to the buffer only
needs to wake up one consumer thread.
Note: Condition variables can be, depending on the implementation, subject
to both spurious wakeups (when wait()
returns without a notify()
call) and stolen wakeups (when another thread acquires the lock before the
awoken thread.) For this reason, it is always necessary to verify the state
the thread is waiting for when wait()
returns and optionally repeat
the call as often as necessary.
-
class
threading.
Condition
(lock=None)¶ If the lock argument is given and not
None
, it must be aLock
orRLock
object, and it is used as the underlying lock. Otherwise, a newRLock
object is created and used as the underlying lock.-
acquire
(*args)¶ Acquire the underlying lock. This method calls the corresponding method on the underlying lock; the return value is whatever that method returns.
-
release
()¶ Release the underlying lock. This method calls the corresponding method on the underlying lock; there is no return value.
-
wait
(timeout=None)¶ Wait until notified or until a timeout occurs. If the calling thread has not acquired the lock when this method is called, a
RuntimeError
is raised.This method releases the underlying lock, and then blocks until it is awakened by a
notify()
ornotify_all()
call for the same condition variable in another thread, or until the optional timeout occurs. Once awakened or timed out, it re-acquires the lock and returns.When the timeout argument is present and not
None
, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof).When the underlying lock is an
RLock
, it is not released using itsrelease()
method, since this may not actually unlock the lock when it was acquired multiple times recursively. Instead, an internal interface of theRLock
class is used, which really unlocks it even when it has been recursively acquired several times. Another internal interface is then used to restore the recursion level when the lock is reacquired.The return value is
True
unless a given timeout expired, in which case it isFalse
.Changed in version 3.2:
Changed in version 3.2: Previously, the method always returned
None
.
-
-
wait_for
(predicate, timeout=None)¶ Wait until a condition evaluates to True. predicate should be a callable which result will be interpreted as a boolean value. A timeout may be provided giving the maximum time to wait.
This utility method may call
wait()
repeatedly until the predicate is satisfied, or until a timeout occurs. The return value is the last return value of the predicate and will evaluate toFalse
if the method timed out.Ignoring the timeout feature, calling this method is roughly equivalent to writing:
while not predicate(): cv.wait()
Therefore, the same rules apply as with
wait()
: The lock must be held when called and is re-aquired on return. The predicate is evaluated with the lock held.Using this method, the consumer example above can be written thus:
with cv: cv.wait_for(an_item_is_available) get_an_available_item()
New in version 3.2:
New in version 3.2.
-
notify
()¶ Wake up a thread waiting on this condition, if any. If the calling thread has not acquired the lock when this method is called, a
RuntimeError
is raised.This method wakes up one of the threads waiting for the condition variable, if any are waiting; it is a no-op if no threads are waiting.
The current implementation wakes up exactly one thread, if any are waiting. However, it’s not safe to rely on this behavior. A future, optimized implementation may occasionally wake up more than one thread.
Note: the awakened thread does not actually return from its
wait()
call until it can reacquire the lock. Sincenotify()
does not release the lock, its caller should.
-
notify_all
()¶ Wake up all threads waiting on this condition. This method acts like
notify()
, but wakes up all waiting threads instead of one. If the calling thread has not acquired the lock when this method is called, aRuntimeError
is raised.