Open
Description
Bug report
Bug description:
Summary
ProcessPoolExecutor.map
may hang forever when the submitted iterable is far greater than max_tasks_per_child
Internally ProcessPoolExecutor.map
submits to the executor all items from the iterable (via submit
). The worker process exits when the first max_tasks_per_child
work items are processed but a new worker process isn't created because ProcessPoolExecutor._adjust_process_count
can be invoked only in submit
Reproducible example
import time
from concurrent.futures import ProcessPoolExecutor
def work(i):
time.sleep(1)
return i * i
def main():
with ProcessPoolExecutor(max_workers=1) as executor:
_ = list(executor.map(work, range(100)))
# completes
with ProcessPoolExecutor(max_workers=1, max_tasks_per_child=5) as executor:
_ = list(executor.map(work, range(100)))
# hangs forever
if __name__ == '__main__':
main()
Expected behavior
Program exits
Actual behavor
Program hangs forever
CPython versions tested on:
3.11
Operating systems tested on:
Windows