Sam Hooke

Debugging a Python process not terminating due to active threads

Situation §

A multi-threaded Python process is not terminating because a non-daemon thread is still alive. Given a running Python process, we want to identify the name of this thread.

Resolution §

Identify the PID of the Python process:

(virtualenv) [user@hostname ~]$ ps aux | grep python
root       859  0.0  0.4 553080  2288 ?        Ssl  Apr26   1:59 /usr/bin/python -Es /usr/sbin/tuned -l -P
appuser  14738  0.2  2.9 825472 14868 ?        Sl   10:11   0:02 /opt/myapp/virtualenv/bin/python -m myapp_package

Connect Pyrasite to the Python process:

(virtualenv) [user@hostname ~]$ pyrasite-shell 14738
Pyrasite Shell 2.0
Connected to '/opt/myapp/virtualenv/bin/python -m myapp_package'
Python 2.7.5 (default, Nov  6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(DistantInteractiveConsole)

>>> 

Use REPL to identify extrenuous threads:

>>> import threading
>>> threading.active_count()
3

>>> threading.current_thread()
<ReversePythonShell(Thread-4, started 140499363800832)>

>>> threading.enumerate()
[<_MainThread(MainThread, stopped 140499644737344)>, <ReversePythonShell(Thread-4, started 140499363800832)>, <NaughtyThread(Thread-2, started 140498966664960)>]
>>>

In the above we identify that there are three threads: _MainThread, which is the main thread and has stopped; ReversePythonShell, which is the thread Pyrasite is running in and so is still active; and NaughtyThread, which by process of elimination is the extrenuous thread which is keeping the process alive. The name corresponds to the name of the Python class.

These are rough notes from whatever I was working on, interested in or thinking about at the time. They vary greatly in quality and length, but prove useful to me, and hopefully to you too!

← Previous: Lint Bash on Windows
Next: The Important Files: Part 1 →