Anatomy of python's mechanism for cleaning up child processes

set in motion

I was under the impression that python's mechanism would automatically clean up child processes that have completed their tasks. I've really seen zombie processes through the net.

import multiprocessing as mp
import os
import time
def pro():
 print (" is ", ())
if __name__ == '__main__':
 print ("parent ", ())
 while True:
  p = (target = pro)
  ()
  (1)

So I thought I'd take a fresh look at the process.

Timing of zombie process destruction

Inherited from BaseProcess file in Lib/mutilprossing/, let's look at its start method.

_children = set()
class BaseProcess(object):
 def start(self):
  self._check_closed()
  _cleanup()
  self._popen = self._Popen(self)
  self._sentinel = self._popen.sentinel
  # Avoid a refcycle if the target function holds an indirect
  # reference to the process object (see bpo-30775)
  del self._target, self._args, self._kwargs
  _children.add(self)

_children is a global collection variable that holds all BaseProcess instances, and _children.add(self) at the end of the start function puts the process objects in. Notice also the _cleanup() function:

def _cleanup():
 # check for processes which have finished
 for p in list(_children):
  if p._popen.poll() is not None:
   _children.discard(p)

_popen is a Popen object, the code is in multiprossing/popen_fork.py, and its poll function has an id, sts = (, flag) a function that recycles the child process. The BaseProcess subclass instance is removed from _children after recycling.

Now it's clear, python puts processes into a collection in sub-process start, sub-processes may run for a long time, so there will be many states of processes on this collection, and in order to prevent too many zombie processes leading to resource utilization, python will clean up the zombie processes when the next sub-process start. So the last child process becomes a zombie process after its own program finishes running, and it waits for the next child process to start to be cleaned up. So there is always a zombie process on ps, but the process id of that zombie process is always changing.