Debugging a Python process not terminating due to active threads
Situation §
A multi-threaded Python process is not terminating because a non-daemon thread is still alive. Given a running Python process, we want to identify the name of this thread.
Resolution §
Identify the PID of the Python process:
(virtualenv) [user@hostname ~]$ ps aux | grep python
root 859 0.0 0.4 553080 2288 ? Ssl Apr26 1:59 /usr/bin/python -Es /usr/sbin/tuned -l -P
appuser 14738 0.2 2.9 825472 14868 ? Sl 10:11 0:02 /opt/myapp/virtualenv/bin/python -m myapp_package
Connect Pyrasite to the Python process:
(virtualenv) [user@hostname ~]$ pyrasite-shell 14738
Pyrasite Shell 2.0
Connected to '/opt/myapp/virtualenv/bin/python -m myapp_package'
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(DistantInteractiveConsole)
>>>
Use REPL to identify extrenuous threads:
>>> import threading
>>> threading.active_count()
3
>>> threading.current_thread()
<ReversePythonShell(Thread-4, started 140499363800832)>
>>> threading.enumerate()
[<_MainThread(MainThread, stopped 140499644737344)>, <ReversePythonShell(Thread-4, started 140499363800832)>, <NaughtyThread(Thread-2, started 140498966664960)>]
>>>
In the above we identify that there are three threads: _MainThread, which is the main thread and has stopped; ReversePythonShell, which is the thread Pyrasite is running in and so is still active; and NaughtyThread, which by process of elimination is the extrenuous thread which is keeping the process alive. The name corresponds to the name of the Python class.