Skip to content

ProcessPoolExecutor executor manager thread crashes if task fails in some circumstances #119592

Open
@batterseapower

Description

@batterseapower

Bug report

Bug description:

import time
from concurrent.futures import ProcessPoolExecutor

def fail():
    time.sleep(1)
    raise ValueError()

executor = ProcessPoolExecutor(max_workers=1, max_tasks_per_child=1)
try:
    print(executor.submit(fail).exception() is not None)
finally:
    executor.shutdown(wait=False)

If you run this, then the following is printed to the console:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "python3.11/concurrent/futures/process.py", line 364, in run
    executor._adjust_process_count()
  File "python3.11/concurrent/futures/process.py", line 759, in _adjust_process_count
    process_count = len(self._processes)
                    ^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

If you change the code to using wait=True, or remove the max_tasks_per_child=1 option then the message is not printed.

Probably the fix is just to check if self.processes is None before accessing it at https://github.com/python/cpython/blob/main/Lib/concurrent/futures/process.py#L762

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions