An easy way to handle this is to just catch SIGTERM and raise a KeyboardInterrupt, so that it has the same effect as a SIGINT.ĭef do_task ( sleep_sec : int ) -> int : "Dummy task" sleep ( sleep_sec ) logger. If the process is still running after the "grace period" (30 seconds by default), it is killed the hard way with SIGKILL (which cannot be caught).
![python exit program and quit all processes python exit program and quit all processes](https://img-blog.csdnimg.cn/20200113152751425.png)
If you want your application to run in Kubernetes (or other systems that communicate with your process using signals), you'll need to catch SIGTERM, which is k8s' way to say "please shut down gracefully". This is subsequently caught by Python after which a KeyboardInterrupt is raised.
![python exit program and quit all processes python exit program and quit all processes](https://i.stack.imgur.com/QEmXw.png)
When running in a shell, one will tend to stop the main process and all its worker processes/threads by doing CTRL-C which sends a SIGINT to the main process. Stop signals and k8s: 3 ways to handle SIGTERM ¶ The overhead of starting processes is negligible as these scripts usually run for minutes, not milliseconds. Most of the work I do involves not only I/O but a good amount of compute so I'll use processes to avoid the GIL. However, that's only the first item on my list, the rest still need to be addressed so we'll happily continue making our own Executor. It must be said that a new feature has landed in Python 3.9 that allows to cancel all pending futures on exit. To do that, I'll build a simple demonstration that exhibits these behaviors. Implement more sophisticated retry handling if desired.Not do the work to submit all tasks at once (this might involve some API calls or whatever for 1M files that I'd rather not do all at once).I often use the Process/ThreadPoolExecutor from the concurrent.futures standard library module to parallelize workloads,īut have trouble exiting gracefully as the default behavior is to finish all pending futures (either using as_completed or during exit of the with block).