Python | Communicating Between Threads | Set-1
Given multiple threads in the program and one wants to safely communicate or exchange data between them.
Perhaps the safest way to send data from one thread to another is to use a Queue from the queue library. To do this, create a Queue instance that is shared by the threads. Threads then use put()
or get()
operations to add or remove items from the queue as shown in the code given below.
Code #1 :
from queue import Queue
from threading import Thread
def producer(out_q):
while True :
...
out_q.put(data)
def consumer(in_q):
while True :
data = in_q.get()
...
q = Queue()
t1 = Thread(target = consumer, args = (q, ))
t2 = Thread(target = producer, args = (q, ))
t1.start()
t2.start()
|
Queue instances already have all of the required locking, so they can be safely shared by as many threads as per requirement. When using queues, it can be somewhat tricky to coordinate the shutdown of the producer and consumer.
A common solution to this problem is to rely on a special sentinel value, which when placed in the queue, causes consumers to terminate as shown in the code below:
Code #2 :
from queue import Queue
from threading import Thread
_sentinel = object ()
def producer(out_q):
while running:
...
out_q.put(data)
out_q.put(_sentinel)
def consumer(in_q):
while True :
data = in_q.get()
if data is _sentinel:
in_q.put(_sentinel)
break
...
|
A subtle feature of the code above is that the consumer, upon receiving the special sentinel value, immediately places it back onto the queue. This propagates the sentinel to other consumers threads that might be listening on the same queue—thus shutting them all down one after the other.
Although queues are the most common thread communication mechanism, one can build own data structures as long as one adds the required locking and synchronization. The most common way to do this is to wrap your data structures with a condition variable.
Code #3 : Building a thread-safe priority queue
import heapq
import threading
class PriorityQueue:
def __init__( self ):
self ._queue = []
self ._count = 0
self ._cv = threading.Condition()
def put( self , item, priority):
with self ._cv:
heapq.heappush( self ._queue, ( - priority, self ._count, item))
self ._count + = 1
self ._cv.notify()
def get( self ):
with self ._cv:
while len ( self ._queue) = = 0 :
self ._cv.wait()
return heapq.heappop( self ._queue)[ - 1 ]
|
Thread communication with a queue is a one-way and non-deterministic process. In general, there is no way to know when the receiving thread has actually received a message and worked on it. However, Queue objects do provide some basic completion features, as illustrated by the task_done()
and join()
methods in the example given below –
Code #4 :
from queue import Queue
from threading import Thread
def producer(out_q):
while running:
...
out_q.put(data)
def consumer(in_q):
while True :
data = in_q.get()
...
in_q.task_done()
q = Queue()
t1 = Thread(target = consumer, args = (q, ))
t2 = Thread(target = producer, args = (q, ))
t1.start()
t2.start()
q.join()
|
Last Updated :
12 Jun, 2019
Like Article
Save Article
Share your thoughts in the comments
Please Login to comment...