SoFunction
Updated on 2025-03-03

Python uses thread module to implement multi-threading operations

introduce:

Threads are a lightweight execution unit provided by the operating system, which can execute multiple tasks concurrently within a process. Each thread has its own execution context, including stack, registers, and program counters.

In Python, threading can be created and managed using the threading module. Threads can execute multiple tasks at the same time, and can perform time-consuming operations in one thread without blocking the execution of other threads. The resources of the process are shared among threads, such as memory space, so you need to pay attention to thread safety issues.

However, Python threads may be restricted by Global Interpreter Lock (GIL) in certain situations. GIL is a mechanism that ensures that only one thread can execute Python bytecode at the same time. This means that in a multi-threaded scenario, even if there are multiple threads, parallel execution cannot be truly implemented. Therefore, in CPU-intensive tasks, Python threads do not fully utilize the capabilities of multi-core processors.

1. Import threading module

Before using Python threads, you need to importthreadingModule. The module can be imported using the following statement:

import threading

2. Create thread

useClass creates thread objects. A thread object can be instantiated by passing a callable objective function and other parameters. The objective function is a task that the thread actually executes.

# Define an objective function as the execution task of a threaddef my_task(arg1, arg2):
    # Code for executing tasks 
# Create thread objectmy_thread = (target=my_task, args=(arg1, arg2))

3. Start the thread

By calling thread objectstart()Method to start the thread. After starting the thread, it will run in the background, executing the code in the objective function.

my_thread.start()

4. Wait for the thread to complete

Can be usedjoin()The method waits for the thread to execute. Calljoin()The method will block the current thread until the target thread execution is completed.

my_thread.join()

5. Thread synchronization

In multithreaded programming, synchronization between threads is an important task designed to ensure that threads are executed in the expected order and avoid the problems of race conditions and data inconsistencies. Python provides several synchronization primitives, commonly used include locks, semaphores, events, and condition variables. The following details of the characteristics and usage methods of these synchronization primitives:

Lock

Lock is a basic synchronization primitive, implemented by classes in Python. It provides two main methods: acquire() and release(). A thread can acquire the lock by calling acquire() . If the lock is not currently held by another thread, the thread will acquire the lock and continue to execute, otherwise it will be blocked until the lock is released. When the thread completes access to the critical section, release() should be called to release the lock so that other threads can obtain it.

import threading
 
# Create a lock objectlock = ()
 
# Thread Functiondef thread_function():
    ()
    # Critical area code    ()

The lock also supports the use of context managers, which can be usedwithStatement to automatically acquire and release locks:

import threading
 
# Create a lock objectlock = ()
 
# Thread Functiondef thread_function():
    with lock:
        # Critical area code

Semaphore

Semaphore is a more advanced synchronization primitive used to control concurrent access to shared resources. Semaphores in Python are implemented by classes. Semaphore maintains an internal counter, and the thread can reduce the counter's value by calling acquire() , and if the counter is zero, the thread will be blocked. After the thread completes access to the shared resource, it should call release() to increase the counter value so that other threads can obtain the semaphore.

import threading
 
# Create a semaphore objectsemaphore = (value=3)  # Set the initial counter value to 3 
# Thread Functiondef thread_function():
    ()
    # Access shared resources    ()

A counter of semaphore can control the number of threads that access shared resources simultaneously.

Event

Events are a synchronization primitive for inter-thread communication, implemented by classes. There are two states for events: set and not set. Threads can set events by calling set() and set their status to set; by calling clear(), the event status can be set to not set. The thread can wait for the event to be set by calling wait() . If the event has been set, the thread can continue to execute, otherwise it will be blocked.

import threading
 
# Create event objectevent = ()
 
# Thread Functiondef thread_function():
    ()  # Wait for event settings    # Perform an action 
# Main thread setting event()

Events can also use the is_set() method to check the status of the event.

Condition variables

Conditional variables are complex synchronization primitives implemented by classes. It provides a conditional queue that allows threads to wait for a condition to occur. Condition variables are used in conjunction with locks to achieve more complex inter-thread synchronization.

import threading
 
# Create a conditional variable objectcondition = ()
 
# Thread function Adef thread_function_a():
    with condition:
        while not condition_predicate():
            ()
        # Perform an action 
# Thread function Bdef thread_function_b():
    with condition:
        # Modify the conditions        ()  # Notify waiting threads 
# Main threadwith condition:
    # Modify the conditions    ()  # Notify waiting threads

In thread function A, the thread will wait for the condition predicate to be established before continuing to execute, otherwise it will be called.wait()Method suspends the thread. Thread function B can be called when a certain condition changes.notify()Method to notify waiting threads.

6. Share data

Shared data refers to data accessed and modified by multiple threads at the same time. When multiple threads read and write shared data at the same time, race conditions and data corruption may occur. To ensure thread safety, appropriate synchronization measures are required to protect shared data. Here are some commonly used synchronization mechanisms and technologies:

Lock

Lock is the most common synchronization primitive used to protect mutually exclusive access to shared data. In a multi-threaded environment, one thread can access shared data exclusively by acquiring a lock, while other threads must wait for the lock to be released before accessing it. The lock can be usedClasses are implemented by callingacquire()andrelease()Method to acquire and release the lock.

import threading
 
# Create a lock objectlock = ()
 
# Share datashared_data = 0
 
# Thread Functiondef thread_function():
    global shared_data
    ()
    # Access and modify shared data    shared_data += 1
    ()

Acquire the lock before accessing shared data, ensuring that only one thread can modify the data at the same time, thus avoiding race conditions.

Semaphore

Semaphores can also be used to protect access to shared data and control the number of concurrent accesses in a multi-threaded environment. Semaphore maintains an internal counter where threads reduce the value of the counter by getting the semaphore before accessing shared data. If the counter is zero, the thread will be blocked. After the thread completes access to the shared data, it increases the counter value by releasing the semaphore, allowing other threads to continue accessing.

import threading
 
# Create a semaphore objectsemaphore = ()
 
# Share datashared_data = 0
 
# Thread Functiondef thread_function():
    global shared_data
    ()
    # Access and modify shared data    shared_data += 1
    ()

By appropriately setting the initial value of the semaphore, the number of threads that access shared data simultaneously can be controlled.

Other synchronous primitives

Python also provides some other synchronization primitives, such as condition variables and events. They can be used for more complex synchronization requirements, such as communication between threads and waiting for specific conditions to occur.

import threading
 
# Create a conditional variable objectcondition = ()
 
# Share datashared_data = []
 
# Thread function Adef thread_function_a():
    with condition:
        while not condition_predicate():
            ()
        # Access and modify shared data 
# Thread function Bdef thread_function_b():
    with condition:
        # Modify the conditions        ()  # Notify waiting threads

In the above example, thread function A can only access shared data when the condition predicate is established, and thread function B passes when the condition changesnotify()Method notifies the waiting thread.

7. Thread status

Thread state refers to the state of the thread at different points in time, which reflects the execution and availability of the thread. In multithreading programming, threads can be in the following different states:

New status

When a thread object is created but the thread has not been started, the thread is in a new state. At this time, the thread object has been created, but the system resources and code execution have not been allocated. A new thread can be created by instantiating a thread class or by getting a thread from a thread pool.

import threading
 
# Create a new thread objectthread = (target=thread_function)

Runnable status

The thread is in ready state when the thread is ready to execute but has not started due to system scheduling. The thread has allocated the system resources and is waiting for the scheduler to put it into the run queue. Multiple ready-state threads may compete for CPU resources, and the scheduler will decide which thread is selected to execute based on the scheduling algorithm.

Running status

When the thread obtains CPU resources and starts executing thread functions, the thread is in a running state. At this time, the thread's code is being executed, and it may be executed concurrently with other threads or switched through time slice rotation. Only one thread can be run.

Blocked state

When the thread is suspended for execution and waiting for a certain condition to occur, the thread is in a blocking state. In the blocking state, the thread will not occupy CPU resources and cannot continue execution until a specific condition is met. Common reasons for blocking include waiting for I/O operations, failing to acquire locks, waiting for notifications from other threads, etc.

Terminated status

The thread is in a terminated state when the thread completes its execution task or is terminated explicitly. When the thread function is executed or an exception occurs, the thread will automatically terminate. You can also call thread objectsjoin()Method to wait for the thread to execute.

# Wait for the thread to complete execution()

Thread states can be converted to each other, and the state transition of threads is usually determined by the execution of the operating system scheduler and thread. For example, when the thread is in ready state and obtains CPU resources, it will enter the running state; when the thread blocks during execution, it will enter the blocking state; when the thread is executed or terminated, it will enter the terminated state.

8. Thread properties and methods

Classes provide some properties and methods to manage and operate threads. Some of these commonly used properties and methods include:

  • name: Get or set the name of the thread.
  • ident: Get the identifier of the thread.
  • is_alive(): Check whether the thread is active.
  • setDaemon(daemonic): Set the thread as a daemon thread, and when the main thread exits, the daemon thread will also be terminated.
  • start(): Start the thread.
  • join(timeout): Wait for the thread execution to complete, optionally set the timeout time.
  • run(): The thread's execution entry point, which is called when the thread starts.
  • sleep(secs): The number of seconds specified for thread sleep.

9. Communication between threads

Inter-thread communication refers to the process of data transmission and sharing between multiple threads in multi-thread programming. The purpose of inter-thread communication is to realize collaboration and data exchange between threads to complete complex tasks. In Python, you can usequeueThe queue provided by the module is to realize thread-safe data transfer.

queueThe module provides several queue types, and the following three commonly used ones are:

Queue (first-in first-out queue)

QueueIt is the most commonly used thread-safe queue, which uses first-in-first-out (FIFO) method to store and obtain data. Multiple threads can safely put data into the queue and get data from the queue.QueueThe class provides the following commonly used methods:

  • put(item[, block[, timeout]]): Put data into the queue to specify whether to block and timeout.
  • get([block[, timeout]]): Get data from the queue to specify whether to block and timeout.
  • empty(): Determine whether the queue is empty.
  • full(): Determine whether the queue is full.
  • qsize(): Returns the number of elements in the queue.
import queue
 
# Create queue objectq = ()
 
# Thread function Adef thread_function_a():
    while True:
        item = ()
        # Process data 
# Thread function Bdef thread_function_b():
    while True:
        # Generate data        (item)

In the above example, thread function A obtains data from the queue and processes it, thread function B generates data and puts it into the queue, and two threads exchange data through the queue.

LifoQueue (Last in first out queue)

LifoQueue is a late-in-first-out (LIFO) queue type. Unlike Queue, its order of acquisition is the opposite of the order of placement. Other methods are the same as the Queue class.

import queue
 
# Create a last-in first-out queue objectq = ()

The last-in-first-out queue is suitable for certain specific scenarios, such as the need to process data in the reverse order.

PriorityQueue (priority queue)

PriorityQueue is a queue type sorted by priority and can specify a priority for each element in the queue. Elements with high priority are obtained first. The priority of an element can be a number, a tuple, or a custom object. Other methods are the same as the Queue class.

import queue
 
# Create a priority queue objectq = ()

Priority queues are suitable for scenarios where data needs to be processed in order of priority.

10. Thread pool

Thread pooling is a mechanism for managing and reusing threads, which can effectively manage the life cycle of a large number of threads and provide a simplified interface to submit and manage tasks. In Python, you can use In the module ThreadPoolExecutorClass to create thread pool.

Features of thread pool

  • Thread reuse: Threads in thread pools can be reused, avoiding the overhead of frequent thread creation and destruction.
  • Thread management: The thread pool is responsible for managing the life cycle of threads, including thread creation, destruction and recycling.
  • Concurrency control: The thread pool can limit the number of threads executed concurrently to prevent system resources from being over-occupied.
  • Asynchronous commit: The thread pool provides a method to submit tasks asynchronously, which can execute tasks in the background and return results.

Create a thread pool

Can be usedThreadPoolExecutorClass to create thread pool. You can specify the size of the thread pool (number of threads that can be executed simultaneously) and other related parameters.

from  import ThreadPoolExecutor
 
# Create a thread poolpool = ThreadPoolExecutor(max_workers=5)

In the above example, a thread pool is created that has a maximum of 5 threads executed simultaneously.

Submit a task

You can use thread poolsubmit()The method submits a task, and the method will return aFutureObject, used to obtain the execution result of the task.

# Define task functionsdef task_function():
    # Task Logic 
# Submit tasks to thread poolfuture = (task_function)

In the above example, the task function task_function is submitted to the thread pool and a Future object is obtained.

Get task results

You can use the result() method of the Future object to get the execution result of the task. If the task has not been completed, the result() method will block until the task is completed and returns the result.

# Get task resultsresult = ()

Close the thread pool

After using the thread pool, you should callshutdown()Method to close the thread pool. After closing the thread pool, new task submissions will no longer be accepted, and all submitted tasks will be waited before exiting.

# Close the thread pool()

The above is the detailed content of Python's use of thread module to implement multi-threading operations. For more information about Python thread multi-threading, please pay attention to my other related articles!