Python implements sharing variables between multiple processes

Share variables between multiple processes in Python

It is very simple to share variables between multiple threads in Python, just define the global variable directly. As multiple processes are independent execution units, this method is not feasible.

However, the Python standard library has provided us with such capabilities and is very simple to use.

But it depends on two situations. One is Process multi-process and the other is Pool process pooling.

Process Multi-process

Using Process-defined variables to share between multiple processes can directly use Value, Array, Queue, etc. under multiprocessing. If you want to share list and dict, you can use powerful Manager modules.

import multiprocessing

def func(num):
    # Shared numerical variables    #  = 2

    # Share array variables    num[2] = 9999


if __name__ == '__main__':
    # Shared numerical variables    # num = ('d', 1)
    # print()

    # Share array variables    num = ('i', [1, 2, 3, 4, 5])
    print(num[:])

    p = (target=func, args=(num,))
    ()
    ()

    # Shared numerical variables    # print()

    # Share array variables    print(num[:])

Pool Process Pool

The above method cannot be used to share variables between process pools, because the process relationship in the process pool is not a parent-child process. If you want to share, you must use the Manager module to define it.

from multiprocessing import Pool, Manager

def func(my_list, my_dict):
    my_list.append(10)
    my_list.append(11)
    my_dict['a'] = 1
    my_dict['b'] = 2


if __name__ == '__main__':
    manager = Manager()
    my_list = ()
    my_dict = ()

    pool = Pool(processes=2)
    for i in range(0, 2):
        pool.apply_async(func, (my_list, my_dict))
    ()
    ()

    print(my_list)
    print(my_dict)

Another thing to note is that when sharing lists, writing func like the one below does not work.

def func(my_list, my_dict):
    my_list = [10, 11]
    my_dict['a'] = 1
    my_dict['b'] = 2

Writing this way is equivalent to redefining a local variable and not acting on the original list. Append, extend and other methods must be used.

Summarize

The above is personal experience. I hope you can give you a reference and I hope you can support me more.