These are rough notes from whatever I was working on, interested in or thinking about at the time. They vary greatly in quality and length, but prove useful to me, and hopefully to you too!
Debugging a Python process not terminating due to active threads
- software engineering
A multi-threaded Python process is not terminating because a non-daemon thread is still alive. Given a running Python process, we want to identify the name of this thread.
Identify the PID of the Python process:
(virtualenv) [user@hostname ~]$ ps aux | grep python root 859 0.0 0.4 553080 2288 ? Ssl Apr26 1:59 /usr/bin/python -Es /usr/sbin/tuned -l -P appuser 14738 0.2 2.9 825472 14868 ? Sl 10:11 0:02 /opt/myapp/virtualenv/bin/python -m myapp_package
Connect Pyrasite to the Python process:
(virtualenv) [user@hostname ~]$ pyrasite-shell 14738 Pyrasite Shell 2.0 Connected to '/opt/myapp/virtualenv/bin/python -m myapp_package' Python 2.7.5 (default, Nov 6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2 Type "help", "copyright", "credits" or "license" for more information. (DistantInteractiveConsole) >>>
Use REPL to identify extrenuous threads:
>>> import threading >>> threading.active_count() 3 >>> threading.current_thread() <ReversePythonShell(Thread-4, started 140499363800832)> >>> threading.enumerate() [<_MainThread(MainThread, stopped 140499644737344)>, <ReversePythonShell(Thread-4, started 140499363800832)>, <NaughtyThread(Thread-2, started 140498966664960)>] >>>
In the above we identify that there are three threads:
_MainThread, which is the main thread and has stopped;
ReversePythonShell, which is the thread Pyrasite is running in and so is still active; and
NaughtyThread, which by process of elimination is the extrenuous thread which is keeping the process alive. The name corresponds to the name of the Python class.